John Feminella, who works on AI architecture and engineering at Two Sigma, explains five composable controls in this 19-minute AI Engineer talk. He speaks in a personal capacity.

A warning and three approvers still missed the mistake

  • Feminella recounts Citibank’s 2020 Revlon payment error: roughly $900 million went out when about $8 million in interest was intended.
  • The workflow simulated a full loan payoff, with principal supposed to stay inside the bank in an internal wash account.
  • A generic warning that money would leave the bank also described the intended interest payment, so three approvers accepted it.
  • Agents need controls tied to their actual authority and actions; well-intentioned operators can misunderstand a valid-looking workflow.

Make agents visible and review them by risk

  • Registry: record each agent’s owner, product, tools and endpoints automatically through approved frameworks, with manual registration available.
  • Risk-based review: separate a ticket summarizer from an agent authorized to deploy production changes.
  • One approval queue for every experiment consumes reviewer attention and encourages teams to work around the process.

Give tools to a declared purpose

  • Authors declare the capabilities their agents need; a research summarizer and a release deployer get different access.
  • Managed environments read registry policy at initialization and translate it into enforceable endpoint, host and resource restrictions.
  • A natural-language policy in the prompt leaves enforcement dependent on the model following it.

Share tests and concentrate human attention

  • Shared evaluations: provide common infrastructure for testing agent behavior while teams define what correct results mean for their own agents.
  • Treat model changes as changes to a runtime dependency, with tests to catch effects on output quality.
  • Selective approvals: use deterministic rules for clear cases, conservative classifiers for ambiguity and human review for consequential uncertainty.
  • Repeated low-value approval prompts create review fatigue; a blanket “yes” habit can hide the action that deserves scrutiny.

Controls need evidence too

  • The talk describes an architecture and operating approach, without measured classifier error rates or before-and-after incident results.
  • Its bank story illustrates control failure; the bank incident itself involved conventional software and human operators.

“You can’t govern what you can’t see.” — John Feminella, 9:00