AI for Customer Support: A Practitioner's Playbook (2026)

In February 2024 Klarna announced its OpenAI assistant was handling two thirds of all chat support, doing the work of 700 agents. The press release was everywhere. Eighteen months later the same company was hiring humans back, the CEO publicly saying quality had slipped. Both things are true. The bot did handle that volume. It also did not handle it well enough.

The takeaway most operators drew was the wrong one. It was not that LLMs cannot do support. It was that resolution rate is a vanity metric the moment a customer's choice is auto-resolve or escalate-and-suffer. If you measure only the first column you ship a system optimised for closing tickets, not solving problems. Klarna's number was real. The thing that number measured was not what the board thought they were buying.

The model is the cheapest decision you make. Claude Sonnet 4.5 at $3 in and $15 out per million tokens, GPT-5 via Azure OpenAI for compliance-bound buyers, Gemini 2.5 Pro if you are already a Google Workspace shop. They all clear the bar. None of them will save you from a bad retrieval layer or a refund tool with no spending cap. I have watched two teams spend six weeks A/B testing Claude against GPT and ship a worse product than the team that picked Sonnet on day one and spent those six weeks writing eval cases.

The same logic applies to vector stores. Pinecone is fine. pgvector inside the Postgres you already run is also fine and cheaper. Cohere Rerank in front of either one buys you more accuracy than swapping the embedding model. The interesting work is in tool definitions and the JSON schemas you hand the model, not in benchmark battles. Read Anthropic's tool-use docs once, then write your refund tool five times before you write it well.

Intercom Fin is good. It is good in a narrow way: B2B SaaS with clean help-centre docs, low SKU complexity and a CSAT bar that tolerates the occasional shrug. The pricing per resolved conversation looks reasonable until you model it against your real ticket mix and realise you are paying premium for the easy ones the bot was going to win anyway. For consumer brands with messy product catalogues, multilingual returns flows or anything regulated, Fin is the wrong shape of tool. Zendesk AI agents are in roughly the same bucket, with the additional friction that you are now married to Zendesk's data model.

In January 2024 Intercom reported that customers of its Fin AI agent resolve an average of 41 percent of conversations end to end. That figure is the honest baseline to plan around, whatever a louder deck claims.

Build only when the off-the-shelf path forces you to bend your business around its limits. The honest middle ground for most teams I talk to is a thin custom orchestration layer over a managed model and a managed vector store, talking to your existing helpdesk through its API. You write maybe four thousand lines of code. You own the bits that matter, which are the tool definitions, the eval harness and the escalation routing. You rent the bits that do not, which are the model weights and the storage.

The work that decides whether your bot survives contact with real traffic is the part nobody puts in the proposal deck. None of it is the model.

The eval set is the unglamorous part. Fifty conversations is the floor, not the ceiling. They have to be real, with the spelling mistakes and the conjoined questions and the customer who started in Polish and switched to English halfway through. OpenAI's cookbook has decent starting templates if you have never built one. The point of the eval is not to score the bot. The point is to notice when a model update or a knowledge change quietly breaks something that used to work.

The capability shift that matters in 2026 is tool calling that mutates state. The bot is not answering questions, it is issuing refunds and changing addresses and pausing subscriptions. This is where the money is and also where the lawsuits live. The right design is three tiers, the wrong design is one big bucket called assistant permissions.

If your refund tool does not have a per-call spending cap and a daily aggregate cap, you do not have a refund tool, you have an incident waiting for a Hacker News thread. The cap is not there because the model is dumb. It is there because a determined customer with a working browser and a willingness to retry will find a phrasing that gets past your guardrails, and the cap is what limits the blast radius when they do.

I keep seeing teams treat multilingual as a model problem. It is not. Sonnet 4.5 and GPT-5 both speak twenty plus languages well enough to fool a casual listener. The problem is that your help articles exist in English, your returns policy exists in English with a half-translated German version from 2022 and your Polish escalation team uses a different ticket schema than your Spanish one. The bot speaks the language fluently and gives confidently wrong answers because the underlying knowledge is wrong or missing.

“

The bot is as smart as the worst-translated paragraph in your knowledge base. Fix that paragraph before you change the model.

Launch in two languages. Get your CSAT, escalation rate and refund-disputed rate roughly equal across both. Only then add the third. Teams that launch in seven on day one are choosing the same dashboard screenshot Klarna chose.

For a mid-market business doing this for the first time, twelve weeks is honest. The first three are knowledge audit and the eval set, which is the part everyone wants to skip. Weeks four through six are the tool layer and the orchestration, picking a model and writing the refund and lookup tools with their caps and their audit trails. Weeks seven and eight are a closed pilot with internal traffic only. Weeks nine and ten run five percent of real traffic with a human reviewing every output, which is annoying and necessary. Weeks eleven and twelve expand to forty or fifty percent if and only if the eval set has stayed green for two consecutive weeks. Anyone selling you four weeks for the same scope is selling you the demo, not the deployment. The ninety day roadmap covers the sequencing in more depth, and the implementation cost breakdown covers what each stage actually runs.