· Rakesh Gangwar
The demo goes well. Someone wires a capable model to a few tools, points it at real work — reviewing a document, reconciling a statement, triaging an exception — and it does the job. Not perfectly, but well enough that everyone in the room can see the shape of the thing.
Then it goes to the operations team, and it stops.
This happens often enough now to be a pattern rather than bad luck, and the cause is usually not model quality. A demo and an operation are different kinds of object. The demo succeeds precisely because it never had to answer the questions the operation runs on.
Four questions a demo never has to answer
Who approves this, and what do they see? In most regulated operations, some decisions cannot be made by software at all. Someone with a name and a job title has to accept them. That person needs the case in front of them, the reasoning that produced it, and enough context to disagree. “The model decided” is not an approval, and a chat transcript is not a review queue.
What happens when step six of nine fails at 2am? Real work spans days, systems, and people. A counterparty is slow, an API times out, a document arrives corrupted. The question is not whether that happens but what state the work is in afterwards, whether it resumes from step six or restarts from step one, and who finds out.
Six months from now, why did it decide that? This is the question that arrives with an auditor, a regulator, or a dispute — long after the context is gone and the model version has changed. Answering it requires that the inputs, the policy in force at the time, the steps taken, and the person who approved them were all recorded as they happened. There is no way to reconstruct it afterwards.
What happens when the policy changes? Thresholds move. A new rule takes effect on the first of the month. If that policy lives in a prompt, the change is a text edit with no review, no version, and no way to say which cases were decided under which rule.
None of these are AI problems. They are the ordinary requirements of operational work, and they were there before anyone added a model.
Three ways teams respond
Improve the model. Better prompts, a larger context, a stronger model, more tools. This helps with quality and does nothing for the four questions above. A more capable model with no approval step is a more capable model with no approval step.
Write the orchestration by hand. Retries, state, queues, and an approvals table. This works, and it is how most teams reach their first production deployment. The cost shows up later: the process is now spread across application code, and the people who own the policy — risk, compliance, operations — cannot see it, let alone change it. Every threshold change becomes an engineering ticket.
Make the process explicit. Model the work as a process in its own right, with the judgment steps marked as judgment, the approvals as real tasks assigned to real people, and execution durable enough to survive a failure at step six. The model becomes one participant in that process rather than the thing holding it together.
We build the third one, so weigh the recommendation accordingly. But the reason we build it is worth stating plainly: the first approach fails the operation, the second works and then calcifies, and the third is the only one where the people accountable for a decision can see and change the rule that produced it.
The part that is genuinely hard
Marking a step as needing judgment is easy to say and hard to do honestly. Most processes contain steps that look mechanical and are not — a classification that carries a legal consequence, a threshold with an unwritten exception everyone on the team knows about. Finding those is not a modelling exercise. It means sitting with the people who currently do the work and asking what they would never let software decide alone.
That is the work we find most interesting, and most of what we write here will be about it.