Wiki · concept

Agents need human confirmation for consequential actions

Asking for human confirmation on every action causes prompt fatigue: users click yes to everything, which defeats the purpose. The better pattern is a policy engine that evaluates risk dynamically. An action that is read-only or low-cost proceeds automatically. An action that is irreversible, involves money, or follows exposure to untrusted content requires explicit confirmation.

This maps to the principle of least privilege applied to agents: grant the minimum capability needed for the current task, and escalate only with human approval for high-stakes operations. The policy engine should track the data flow: if an agent (see: agent actions and planning must be observable) recently processed content from an untrusted source, subsequent write actions should require confirmation regardless of their normal risk level.


References

  1. Simon Willison. The lethal trifecta for AI agents