01 the business
Online pet supplies store
40 staff, five support agents
Five agents answer email and chat for a store shipping thousands of orders a week. Most messages are the same few questions about order status, returns and changes. Customers with real problems wait behind them, and weekends pile up until Monday.
02 before
The problem
Most of the inbox is the same handful of questions: where is my order, how do I reset my password, can I change my booking. Your support people answer them one at a time, so the customers with real problems wait behind them. Evenings and weekends pile up until Monday.
- Routine questions take most of the team's day.
- Weekend messages wait up to two days for a reply.
- New agents take weeks to learn every policy.
03 the build
What the AI does
- Step 1: Reads each new email or chat message and works out what the customer is asking for.
- Step 2: Looks up the answer in your help center, policies and past replies, and checks live details such as order status in Shopify or a subscription in Stripe.
- Step 3: Replies in your tone of voice when the answer is clear and inside the rules you set.
- Step 4: Asks a short follow-up question when the message is vague, instead of guessing.
- Step 5: Hands the conversation to a person in Zendesk or Intercom with a summary, the customer's history and a suggested reply when it is unsure or the topic is sensitive.
paused: waiting for a person to approve
How it was put together
- Grounded answers in the help centre, the returns policy and approved past replies.
- Connected live order and tracking data so answers use real status, not guesses.
- Let the support lead choose which topics the agent may resolve alone, starting with order status.
- Handed over everything else in the help desk with a summary, the history and a suggested reply.
04 human in the loop
Where a person stays in charge
Your support lead decides which topics the agent may answer on its own. Refunds, complaints, legal questions and anything outside those topics always go to a person before a reply is sent.
- Refunds, complaints, legal questions and anything outside approved topics always go to a person.
- The agent says it is automated, and customers can ask for a person at any time.
- Every automated reply is logged and sampled weekly for quality.
05 rollout
From kickoff to live in 5–6 weeks
- Weeks 1–2Tag the last three months of tickets by topic and pick the first three topics.
- Weeks 3–4Run in draft mode: the agent suggests, people send. Measure accuracy.
- Weeks 5–6Allow automatic replies on topics above the agreed accuracy bar; expand monthly.
06 after
What changes
Simple questions get answered within minutes, day or night. Your team spends its time on the conversations that need judgement, and every hand-off arrives with the context already gathered.
Assume 6,000 conversations a month, 45% of them order-status and simple policy questions, at 4 minutes each for a person.
6,000 × 45% = 2,700 routine conversations. 2,700 × 4 minutes = 180 hours a month that can move to complex cases.
What to measure
- Share of conversations resolved without a person, by topic.
- First response time, including nights and weekends.
- Customer satisfaction on automated versus human replies.
- Re-opened conversations after an automated answer.
07 evidence
What the research says
Across 5,179 customer support agents, an AI assistant raised issues resolved per hour by 14% on average and by 34% for novice and lower-skilled agents.
NBER, Generative AI at Work (Brynjolfsson, Li and Raymond) (opens in a new tab) · 2023
From August 2026, AI systems that talk directly to people in the EU must make clear they are AI, unless that is obvious.
EU AI Act, Article 50: transparency obligations (opens in a new tab) · 2024
About 75% of generative AI's potential value sits in four areas: customer operations, marketing and sales, software engineering, and R&D.
08 lessons
Mistakes to avoid
- Launching on every topic at once. Start narrow and expand on evidence.
- Letting it answer from the open web. Answers must come from your own policies and data.
- Hiding the handover. Customers forgive automation; they do not forgive being stuck.
09 questions
Common questions
- How accurate does it need to be before going live?
- Your support lead sets the bar per topic, based on draft-mode results. Topics that miss it stay with people.
- Does research back this up?
- A large study of 5,179 support agents found an AI assistant raised issues resolved per hour by 14% on average, and by 34% for newer agents.