Anirudh Veeraragavan, lead security engineer at Fieldguide, walks through a bank-reconciliation agent in this 13-minute AI Engineer talk.

Make each decision inspectable

  • Bank reconciliation compares a company’s cash books with its bank balance and explains differences through outstanding checks and deposits in transit.
  • An inspector needs the evidence and reasoning behind a specific decision, even when aggregate tests and user reviews look good.
  • Design the record so an experienced auditor joining later can follow the agent’s work.

Four controls that work together

  • Authorization: permit an agent to draft unreviewed rows under its own identity; preserve human-reviewed work and present later changes as suggestions.
  • Execution records: give each step an identifier, inputs, and outputs; distinguish ordinary code from model-dependent extraction or judgment.
    • Replay individual steps and compare old and new runs to locate changed evidence, behavior, or conclusions.
  • Evidence provenance: tie citations to a specific document ID, version, and content hash, with rules for acceptable sources.
  • Evaluation integrity: audit the expected answers in your test set for superseded decisions, stale evidence, and changed methodology.

Give reviewers a concrete exception

  • Stop when an agent would alter reviewed work, a new run contradicts an old one, or evidence fails the required rules.
  • Show what differs, the supporting evidence, and what needs investigation; repeated approval prompts encourage rubber-stamping.
  • The talk’s illustrative 97% score shows how incorrect expected answers can both penalize correct behavior and conceal real misses.
  • Veeraragavan extends the same controls to loan decisions and anti-money-laundering alerts.

“An experienced auditor new to the work can follow the agent.”

Caveat: This is a design walkthrough with fictional audit examples. It supplies no measured deployment improvement or guarantee of identical model reruns.