Sai Varun Kodathala, Prashanth Pollishetty and Jaylen Cargill test what an AI agent’s approval button actually protects. Their paper, Approve What, Exactly?, separates two requirements: the confirmation must show the complete action, and execution must stay tied to that exact action afterward.
Their reference system, Agent Step-Up, puts a gateway between the agent and consequential tools. The gateway builds the confirmation from the stored tool request, obtains a passkey signature and executes that stored request once. The agent’s explanation appears separately, marked untrusted.
- The display changes the decision. Across 15 language-model reviewers, attack detection rose from 30% with an agent-written description to 98% with a trusted confirmation. The test used 23 attack examples and 11 harmless near misses, shown in three formats.
- Showing more also creates false alarms. Reviewers rejected 16% of harmless near misses on the trusted screen. Most of those refusals concerned placeholder message bodies used in the experiment.
- Approval can come apart from execution. In eight runtime-audit rows where the authors could test binding, four allowed arguments to change after approval. Three resisted it; one resisted the resume interface but remained vulnerable to changes in stored session data.
- A phone is no shortcut. Trusted on-host confirmation matched the phone-based design in every AgentDojo replay cell. The experiments left the host display uncompromised, so they could not measure the phone’s value against host compromise.
The design work is deliberately conventional; the contribution is its attacks, audit and measurements. A confirmation built from the actual request can expose an extra email recipient that a reassuring summary omits. Storing that same request prevents a later substitution.
Caveat: these are author-reported local tests with simulated or model reviewers, with code and data available. No people were tested; flooding still succeeded 45–60% of the time under assumed approver behavior.