Wiki · concept

Agent actions and planning must be observable

Systems that hide what they are doing from the operator are inherently harder to evaluate and more vulnerable to subversion. Every action an agent takes (tool call, file write, API request) and every reasoning step should be logged, timestamped, and auditable. The metadata matters: whether an action is read-only or state-changing, whether it handles sensitive data, whether it was triggered after processing untrusted content.

The security argument reinforces the engineering one. An opaque agent that silently modifies state is not just frustrating to debug; it is a security liability. The more transparent the system, the better your chance of catching a prompt injection before it causes damage.

Observable actions are also what make an output checkable for the user, turning a hard-to-eval product into a verifiable one (see: a hard-to-eval product is hard for users to verify).


References

  1. Simon Willison. The lethal trifecta for AI agents