Human in the loop, or on the loop?

There are two ways to keep a person in charge of an agent. In the loop, the person approves a step before it happens. On the loop, the agent works inside written limits while the person watches, can see every step, and can stop it.

Neither is the safer choice in general. The NIST AI Risk Management Framework notes that human-AI configurations can span from fully autonomous to fully manual, and that human roles in overseeing AI systems need to be clearly defined. The useful move is to make the choice per action, not per system, and to write it down.

Two questions decide

For each action the agent can take, ask:

1. What does a wrong step cost? Money, a customer's outcome, a regulatory position, a production line.
2. Can it be undone, and by whom? A draft can be deleted. A posted payment can sometimes be reversed. A message sent to a customer cannot be unsent.

The answers sort actions into four groups:

• Cheap and reversible, such as tagging a document, drafting an internal summary, or requesting a missing file from the case owner: on the loop. The record shows each step, and a person can correct it later.
• Cheap but irreversible, such as sending an external message: approve it, or redesign it so it becomes reversible, for example by queuing it for release after a short review window.
• Costly but reversible, such as a hold on a shipment or a change to a case's priority: on the loop only inside tight written limits, with review of a sample and an alert on anything near the limit.
• Costly and irreversible: in the loop, always. Some actions never move out of a person's hands at all. An agent never holds credit authority, never declines an insurance claim, never closes a sanctions match, and never files a suspicious-activity report. It prepares each of these for the person who decides.

Limits are written, and the institution writes them

On the loop works only when the limits are explicit. The institution writes them: which actions, on which systems, up to which amount or threshold, for how many steps, within what time. The agent reads its limits. It never sets them and never widens them, including when a case seems to call for it.

Limits are enforced outside the model, in the harness and in the permissions of each tool, not by asking the model to behave. A tool with read-only access cannot write, whatever the model produces. The OWASP guidance on excessive agency makes the same two points: give tools the minimum permissions they need, and require a person to approve high-impact actions before they are taken.

Stop conditions

An agent on the loop needs to know when to stop. Write the conditions before it runs:

1. A written limit would be crossed by the next step.
2. Evidence is missing or contradicts itself.
3. A tool fails and its retry budget is spent.
4. The step budget is reached, such as a maximum number of iterations.
5. The request is outside the agent's written scope.
6. A person presses stop.

Stopping is not failure. When the agent stops, it hands the case to a named person with what it found, what it did, and why it stopped. The NIST framework asks for exactly this capability: mechanisms in place, with responsibilities assigned and understood, to supersede, disengage, or deactivate AI systems that show performance or outcomes inconsistent with intended use.

Make watching real

On the loop is only oversight if someone can actually watch. That means a live view of open cases, the record of every step and tool result, alerts on cases near a limit, and a regular review of a sample of completed cases by someone who knows the work.

In the loop has its own failure. A person asked to approve every small step soon stops reading and approves by habit. A few approvals placed at the steps that matter, with the evidence laid out, protect more than many approvals spread across steps that do not.

Moving an action between modes

Start an action in the loop. Move it on the loop only on evidence: the approval record shows the approvers agreed with the agent's proposals, and the errors they caught were cheap and reversible. The decision to move belongs to the institution, which writes the new limit. The agent can report the record; it does not propose or apply a wider limit for itself.

Moving back is immediate. If something changes, such as a new product, a new rule, or an unexplained error, the action returns to approval while the cause is found.

The checklist

• Every action the agent can take is listed, with its cost if wrong and whether it can be undone.
• Each action is marked in the loop or on the loop, and the mark is written down.
• Limits are written by the institution and enforced in the harness and tool permissions.
• Stop conditions are written, tested, and hand over to a named person.
• Every step is recorded, and someone reviews the record.