What Should Never Be Automated
Want to talk about this essay? Email me: amitkvint@gmail.comcopied· or message me on LinkedIn
Most of the decisions in an AI support project are about what the agent will do.
But what about what it shouldn't do?
Every report that came in was classified before anything else happened: AI Ready, or Needs Human. Around 60% came back AI Ready in the first months. That is the number people ask about, and it is the boring one.
I have written about the mechanics of that rollout elsewhere, so one line is enough here: Needs Human reports went straight to a person with the debug data already attached, so the supporter could start solving instead of collecting.
This is about the line itself, and who it was drawn for.
Nobody draws that line from the customer's side
The obvious way to draw it is from the agent's side. What can it handle? You look at your documented issues, your canned diagnostics, your resolution history, and you push the line as far out as the evidence supports.
That is a reasonable way to start, and it is how we started.
It is also how you automate something you shouldn't, because "the agent can produce a correct answer here" and "a correct answer is what this person needs" are not the same sentence.
Somebody who has just lost a week of work on a live site does not have an information problem. The reply can be accurate, fast, polite, and still be the wrong thing to send. The customer is not reading it for the steps.
So the second pass has to come from the other direction: which reports does a person need to be on, whatever the agent is capable of that quarter.
The categories that should be held back on the human side are angry customers, refunds, data loss, security, and the like. What they have in common is quite obvious: they are not technical anymore. They have a technical origin, but the client does not care about that now. They are emotional. Someone is actually angry and disappointed, and an angry client can cause a lot of damage. If you have worked in support for as many years as I have, you know that very well.
What automation actually buys
Around 30% of the total workload was absorbed in the first weeks, mostly known issues and documented how-tos. After the full rollout, average resolution time fell from roughly 24 hours to around 10, and satisfaction stayed above 95%.
Those are the numbers that go in the deck, and they are real.
What they do not show is where the freed time went, and that is a decision, not a result. Automation does not create space in a support organisation by itself. It removes a category of work, and something else expands to fill the room.
If nobody decides what expands, volume does.
The job that was left
Once the easy and predictable tickets were gone, what stayed with people was the ambiguous and the genuinely difficult.
That is easy to say and easy to underestimate. A queue where almost every remaining report is hard is a different job from a queue where most of them are quick. It is more interesting and considerably more tiring, and you feel the difference by Thursday.
It also breaks your measurement. Speed stopped being a fair way to judge a supporter, not because anyone got slower, but because the fast work had left the queue. Keep reporting average handling time after that and you are mostly measuring how nasty this week's remaining problems happened to be, while quietly penalising whoever took the worst ones.
HR and management kept looking at headcount. That is the simplistic view.
Before the agent, a good supporter could take 9 or 10 new tickets and chats a day. After the rollout that was not possible anymore, because every issue that came through the agent, escalated or not, was one of the hard ones. No more "do this, then do that". Now it was "I need to debug this, make a copy of your site, and look at your code or your database structure".
The hiring bar moved too. The people we brought in afterwards were more technical than the people we had hired before, because the work that was left needed it.
A line is not a launch decision
The line moves. The agent gets better, the documentation improves, a category that needed a person in March does not need one in September. That is fine, as long as somebody owns the moving.
In practice we found out the line was in the wrong place by reading what the agent had actually said to customers, week after week. That habit deserves its own piece, so I will only say here that it is where the evidence comes from. Nothing else told us.
The failure mode is not that the line is drawn badly on day one. Everyone draws it badly on day one. The failure mode is that nobody is responsible for it afterwards, so it drifts outward by default, one small "the agent handles that now" at a time, and no single step in that drift is ever wrong enough to argue about.
The part to remember
If you are about to draw this line for your own queue, try drawing it backwards.
Not "what can the agent take off us", but "what do we want a person to have time for". Then automate towards that.
The first question has a different answer every few months, because it is really a question about the model. The second one shouldn't change that often, because it is a question about your customers, and they have not been upgraded.
The human part is why customers stay. Automation earns its place when it buys time for that part. It stops earning it when it starts taking that part instead, and the reporting will look identical either way.
So when the next release moves the line further out, the question I would ask first is not whether the agent can handle it.
It is whether anyone would notice what we quietly stopped doing by hand.