Alex Wauters turned his “approve or deny the AI coding agent’s commands” browser game into a dataset: over 40,000 runs and 409,000 decisions, and the results are a bleak audit of the human-in-the-loop as a security control. The average player missed 1 in 3 threats, a third of sessions finished with a negative score, and 7% of players just approved everything. The category breakdown is the uncomfortable part: blatantly destructive commands like rm -rf / were caught 88% of the time, but the commands that actually steal credentials (cat ~/.aws/credentials) were missed three times as often. The single most-missed threat was npm run analyze — approved 64.7% of the time — because a familiar script name hides whatever arbitrary code lives in package.json, even when the payload is displayed in the history log right above the prompt. Wauters’ argument is structural, not just statistical: command-by-command approval asks users to validate commands that are almost always safe but stop being safe the moment the agent edits a file, and it demands a vigilance humans demonstrably don’t have (miss rates climb at the end of sessions; 59% of players blocked a benign internal registry config). His takeaway, echoing Anthropic’s own admission about permission fatigue: sandboxing and separating secrets beat vigilance.
Humans Missed 1 in 3 Threats Approving AI Agent Commands — Alex Wauters
40,000 game runs show the average human-in-the-loop missed a third of real threats from AI coding agents — and the commands that actually exfiltrate credentials were missed three times as often as the scary ones.