“Human in the loop” is said in almost every conversation about AI agents, usually as reassurance. A person will check it. But where exactly does the person sit, what do they check, and how do you stop that check from becoming a click nobody thinks about? Getting this right decides whether an agent is safe and whether it actually saves time.
The three places a person can sit
Before the action: approval
The agent prepares something and waits. A person reads it and approves, edits or rejects it. Nothing happens until they do. This suits anything that moves money, reaches a customer or cannot be undone: a refund, a message to a buyer, an order sent to a supplier, a change to a customer’s account.
After the action: review
The agent acts, and a person looks through a sample or a daily summary afterwards. This suits low-risk actions that are easy to reverse: tagging emails, filing documents, updating an internal status field. The review catches patterns of mistakes rather than single errors.
On exception: escalation
The agent works on its own for normal cases and hands over to a person when something falls outside its rules: a message it cannot understand, an amount above a limit, a customer who is clearly upset, a document that does not match. This suits high-volume work where most cases are routine.
Most well-designed agents use all three, for different actions within the same task.
Deciding which check goes where
For each action the agent could take, ask three questions:
- Can it be undone easily? If not, approval before.
- Does it reach someone outside the business? A customer, supplier, bank or regulator. If yes, approval before, at least until there is a long record of it getting that action right.
- What does one mistake cost? If the cost is small and the action reversible, review after is usually enough.
Take an agent handling WhatsApp orders for a small clothing shop. Reading the message and filling in an order draft: no approval needed, a person sees the draft anyway. Checking stock: no approval, it only reads. Sending the confirmation with price and delivery charge: approval before, because it reaches the customer and states a price. Marking the order as ready for the courier: approval before, because a rider will be sent. Adding a tag to the customer record: review after.
Keeping the check real
The biggest risk with human in the loop is not the agent. It is the person approving forty drafts in a row without reading them. After a few weeks of the agent being right, attention fades. Then the one bad draft goes out with an approval stamp on it.
Ways to keep checking meaningful:
- Show what matters, not everything. The approval screen should put the key facts up front: customer, amount, what will happen. Not a wall of text.
- Highlight the unusual. If the amount is higher than this customer’s normal order, or the delivery address has changed, mark it clearly.
- Keep batches small. Twenty approvals at a time is easier to do carefully than two hundred.
- Record who approved. When approvals carry a name, people read more carefully.
- Measure edits. If staff never change anything, either the agent is excellent or nobody is reading. Spot-check to find out which.
Moving the person over time
Human in the loop is not fixed forever. As the record grows, some approvals can move to review, and some reviews can become occasional samples. The key is that each move is based on evidence and decided by a person with authority, not drifted into because approving felt tedious.
A sensible rule: an action can move from approval to review only after a set number of consecutive approvals with no edits, for example a few hundred, and only for actions that are reversible. Actions that move money or reach customers with commitments should stay under approval much longer, and some should stay there permanently. A refund, for instance, is a decision a person should own.
Moving back matters too. If the agent’s inputs change, such as a new product range or a new type of customer, bring the approvals back for a while.
What human in the loop does not solve
A person approving an action does not make the agent’s reasoning correct. It only catches the mistakes the person notices. If the agent is working from the wrong price list, a busy approver may not spot that every draft carries an old price. So human checks need to sit alongside other safeguards: correct source data, hard limits in the software, and logs that let you trace problems back.
It also costs time. If the approval takes nearly as long as doing the task by hand, the agent is not saving much. In that case, either the approval screen needs redesigning or the task was not a good fit.
Take one agent or one process you are planning and list every action it could take. Next to each, write “approve before”, “review after” or “escalate on exception”. That single page will shape the design more than any choice of model.
