How much of your customer service can you hand to an AI agent?

Three paths, not two

Almost every discussion about AI agents in customer service starts as a binary choice: the AI answers or it doesn't. In practice there are three paths for every incoming message. The AI answers on its own. The AI writes a draft and someone on the team reviews it before it goes out. Or the message goes straight to a person, with no draft.

Each path has a cost. An unreviewed answer takes nobody's time, but when it's wrong, the error reaches the customer. A review takes a few minutes per message and should catch the error before it goes out. A human answer takes more minutes, and it's what's left when the AI's draft doesn't help.

The right decision depends on what an error costs for that question. Getting a viewing time wrong costs one extra contact. Getting a rent amount or a contract clause wrong costs a customer, or a legal problem.

Almost all of the gain comes from the simplest step: splitting by type. In that setup, the AI answers every viewing-time question on its own and drafts the rest for review. Errors still reach customers, about 1,050 a month, all of them on viewing-time questions, where each error costs one repeat contact. The goal isn't zero errors. It's putting the errors where they're cheap. And refining answer by answer only pays off with well-measured confidence: with the confidence the AI reports, refining costs more than not refining.

What usually goes wrong

• Treating automation as all or nothing. The pilot measures the AI's average accuracy, say 80%, and the company decides whether "80% is enough". It's neither enough nor too little: 80% is great for viewing times and unacceptable for contract clauses.
• Trusting the confidence the AI reports. Language models tend to overstate how sure they are when asked to put it in words. The confidence that counts is the one the company measures, on a set of real questions with known right answers, split by type of question.
• Reviewing everything "to be safe". It sounds prudent, but it spends the team on messages the AI would get right on its own, and it wears reviewers out. A tired reviewer approves without reading, and then the review costs the time and doesn't catch the error.
• Forgetting that accuracy changes. Inventory changes, the pricing policy changes, the vendor's AI model changes. The threshold set during the pilot stops being valid, and nobody notices until the error shows up at the customer.
• Nobody owns the threshold. The vendor delivers the agent, customer service uses it, and the rule for who answers what stays hidden in a configuration file.

How to decide, in five steps

1. Split the questions by type and put a price on the error for each type: what happens when the wrong answer reaches the customer (repeat contact, missed viewing, lost lead, legal risk).
2. Put a price on the team's time: what it costs to review a draft and what it costs to answer from scratch.
3. Measure the AI's accuracy by type and by confidence band, on real questions that have already been answered. A few hundred per type are enough to start; to let the AI answer on its own where errors are expensive, you need thousands, because the acceptable error margin is small. This measured accuracy, not the number the AI states, is what goes into the calculation.
4. Set two thresholds per type: above the upper threshold, the AI answers on its own; between the two, the AI drafts and someone reviews; below the lower threshold, the question goes straight to a person. The calculator below does the math for the four types in the example, and you can change the numbers.
5. Audit every month: a sample of automatic answers reviewed after sending, to check that accuracy is still where it was, and a sample of reviews, to check that reviewers still catch the errors.

How to use this in your company

The same calculation applies to any workflow where an AI produces an output and someone can check it first. Four examples of how the threshold takes shape:

• E-commerce customer service. "Where is my order?" the AI answers on its own, from tracking data. Exchanges and returns within policy get a draft and a review. Complaints that mention a consumer-protection agency, a lawsuit or the press go straight to a person.
• Reading invoices and documents. Fields read with high confidence go straight into the system. Amount or tax fields with intermediate confidence go to a check. Illegible or non-standard documents go to manual entry.
• Expense reimbursement. Small amounts within policy, the AI approves. Large amounts or unusual categories, the AI prepares the analysis and a manager approves. Signs of fraud go to audit.
• Sales inquiries (the case that inspired this piece). In a rental assistant, the AI qualified prospects (occupancy, price range, number of bedrooms) against real inventory and passed only the matching leads to the agent. The design that worked was "the AI qualifies, the person closes".

In every case, the deciding question is the same: what does this error cost if it reaches the customer, compared with what it costs to have someone look first.

References

• Chow, C. K. (1970). On Optimum Recognition Error and Reject Tradeoff. IEEE Transactions on Information Theory, 16(1), 41–46.
• Geifman, Y. & El-Yaniv, R. (2017). Selective Classification for Deep Neural Networks. NIPS 2017.
• Xiong, M. et al. (2024). Can LLMs Express Their Uncertainty? An Empirical Evaluation of Confidence Elicitation in LLMs. ICLR 2024.
• Elkan, C. (2001). The Foundations of Cost-Sensitive Learning. IJCAI.