Human-in-the-loop automation: why a person should approve every step

What "human in the loop" means at your order desk

Human-in-the-loop automation is simple to describe. Software does the slow part of the job: it reads the customer's PO from email, finds the customer and items in your ERP, and drafts the sales order. Then it stops. A person on your team looks at the draft, fixes anything that's off, and clicks approve. Only then does the order go into the ERP, the confirmation go out, or the invoice get paid.

The opposite is "fully automatic," where the software posts orders or pays bills on its own and a person only finds out when something goes wrong. Plenty of vendors sell that idea. This guide explains why we think a person should approve every step for the work most mid-size companies want to automate first: orders, invoices, purchasing and customer emails. It also covers when that rule can relax, and how to keep the approval from eating the time you meant to save.

People trust AI more than they check it

The best reason for a human check is how people actually use AI today. KPMG and the University of Melbourne surveyed more than 48,000 people in 47 countries for Trust, attitudes and use of AI: a global study 2025. Among employees, the full report found:

How employees use AI at work

KPMG and University of Melbourne, global survey of 48,000+ people in 47 countries, 2025. Share of employees.

1. Relied on AI without checkingUsed AI output without critically evaluating it66%
2. Use AI regularlyIntentionally use AI at work on a regular basis58%
3. Not open about itUsed AI in non-transparent ways57%
4. Made mistakesMade mistakes in their work because of AI56%

Source: KPMG and the University of Melbourne, Trust, attitudes and use of AI: a global study 2025 (full report).

That pattern has a name. Researchers call it automation bias: the tendency to over-rely on automation. A 2012 systematic review in the Journal of the American Medical Informatics Association found people accept computer output as a shortcut instead of checking it themselves, which leads to both doing the wrong thing and missing the right thing. The U.S. National Institute of Standards and Technology lists automation bias as a risk in its Generative AI Profile (July 2024), and warns it can make AI's made-up answers more dangerous.

So the human check isn't there because your people aren't smart. It's there because everyone, smart people included, starts trusting a tool that's right most of the time. A good approval step is designed to fight that habit.

Even professional AI tools get things wrong

It's tempting to think the errors come from cheap tools. They don't. Stanford researchers tested AI legal research tools sold to lawyers, from companies that spent heavily on getting them right. Lexis+ AI and Ask Practical Law AI gave incorrect information more than 17% of the time. Westlaw's AI-Assisted Research did so more than 34% of the time. Those were hard benchmark questions in a preprint study, and the tools have changed since May 2024. The lesson still holds: a polished product can be confidently wrong.

On an order desk, most work is easier than legal research. Reading a PO number off a clean PDF is not hard. But the hard cases are the ones that cost money: a customer's part number that maps to two of yours, a unit of measure that changed, a ship-to that isn't on file, a price that doesn't match the contract. Those are where a person catches what software misses.

The approval isn't overhead. It's the part of the job that keeps one bad read from becoming a bad shipment.

What the rule makers and the AI makers say

You don't have to take our word for it. Three very different sources land in the same place.

• Regulators. The EU AI Act's Article 14 on human oversight says high-risk AI must be built so people can oversee it. Those people must be able to override or reverse its output, and to stop it with a "stop" button or a similar step. It names automation bias as something overseers need to watch for. Most order-entry or AP automation is probably not "high-risk" under the Act, and it's EU law, not Texas law. But it's a clear picture of what good oversight looks like.
• OpenAI. OpenAI's practical guide to building agents says actions that are sensitive, irreversible or high-stakes should trigger human oversight. Its examples are the ones a business owner cares about: canceling orders, authorizing large refunds and making payments.
• Anthropic. Anthropic's guide to building effective agents describes agents that pause for human feedback at checkpoints, and says they add the most value where there is meaningful human oversight.

Add one more reason that has nothing to do with AI. Business email compromise, where a fake email asks you to pay a new account or change a vendor's bank details, cost victims who reported it more than $3 billion in 2025, according to the FBI's 2025 Internet Crime Report. Businesses also reported more than $30 million in losses to these scams when AI was involved. Software that pays bills on its own is a gift to those scammers. A person approving payments is a cheap defense.

Where the other side has a point

The fair objection is cost. If someone has to check every draft, have you saved anything? G2's 2026 AI Agents Report, a survey of more than 1,000 business software buyers, found that 45% said the time spent overseeing and correcting agents ate more than a quarter of the productivity gains they expected. 31% said employees spent six or more hours a week watching over agent work. G2 calls it a "distrust tax." That's a vendor survey, but it matches what we hear.

There's a second risk: approval fatigue. When people are asked to approve too many things, they stop looking. In hospitals, a 2020 review of drug-alert studies found doctors overrode between 46% and 96% of computer alerts on average. That's medicine, not order entry, but the lesson carries: a check people click through without reading protects nothing.

And OpenAI's own guide frames oversight as something you keep "until confidence in the agent's reliability grows." Approval doesn't have to stay at the same level forever.

So the answer isn't fewer checks. It's checks that are fast, focused and hard to rubber-stamp.

How to make the approval fast

1. Show the evidence. The approver should see what the software read and where it read it, not just the final answer.
2. Flag, don't hide, doubt. When a customer part number doesn't map cleanly, or a price is off the contract, the software should say so and stop, not guess.
3. Keep money and master data with people. Payments, credit holds, price changes and vendor bank details always get a person, no matter how good the software gets.
4. Measure it. Track how often approvers change a draft and what they change. If edits fall to near zero on a customer's routine orders, that's the case for a lighter check there. If approvers never change anything, check whether they're actually looking.
5. Name the owner. Approving drafts should be a named part of someone's job, with time set aside, not a pile of extra clicks on top of the old workload.

What one approval looks like

Say a customer emails a PO with eight lines. The software reads it and drafts the sales order. On the approval screen, the PDF sits on the left and the draft on the right. Seven lines match past orders and the contract price, so they're marked as matched. One line is highlighted: the customer used a new part number the software hasn't seen, and it found two of your items that could fit. The order desk picks the right one and clicks approve. The software remembers the choice for next time. The whole check takes well under a minute, and the one decision that needed a person got one.

When can a person step back?

Some steps can run without a click once they've earned it: filing a PDF, logging an order acknowledgment, sending an order-status reply that only repeats what's in the ERP. The test is simple. Is it reversible, low-stakes and easy to check later? Then a person can review in batches, or spot-check, instead of approving each one. If it moves money, commits inventory, changes a price or speaks for the company in a new way, a person approves it first.

How Peekgentic builds it

Every build we do works this way. Software reads and drafts; a person on your team approves before anything is sent, entered or filed; every action is logged and can be reversed; and when something doesn't match, it stops and asks instead of guessing. See how an order-processing build works, or how we handle security and approvals.