Mitchell Troyanovsky, co-founder of accounting-agent company Basis, presents a 19-minute talk on AI Engineer about making agents dependable across a company.

Personal gains need shared processes

  • General-purpose agents leave users to manage skills, memory, context and the choice of when to delegate.
  • Companies also need a shared way of doing work, with approved tools and consistent handling of exceptions.
  • Troyanovsky calls these the competency gap and the process gap; simplifying the interface addresses only the first.
  • The title’s “100x” comes from his personal productivity estimate, without a measured company-wide comparison.

Define the behavior you want

  • Choose explicit methods: PDF extraction could use a vision model, optical character recognition, or a checked combination.
  • Use domain expertise to set expectations, such as consulting primary sources before returning tax advice.
  • Trade accuracy against latency deliberately; rendering and inspecting a slide deck adds time but can catch layout errors.
  • Prefer stable capabilities and clear limits over an agent that alternates between brilliance and basic mistakes.

Turn opinions into behavior specs

  • Write the important intentions before changing prompts, tools or other context; avoid duplicating every implementation detail.
  • In his bank-reconciliation example, explain a discrepancy instead of inserting an unsupported entry just to balance the books.
  • Inspect a trajectory, the recorded sequence of agent actions: did the relevant situation arise, and did the agent follow the rule?
  • Keep outcome evaluations, tests of the final result, while adding checks that work even when no reference answer exists.
  • Behavior emerges from system prompts, progressively revealed instructions, tool descriptions and the underlying model together.

Debug with forks and simulations

  • Fork a recorded run near a bad decision, change the context and compare the resulting behavior.
  • Recreating databases, prompts and the surrounding environment makes cloud replay expensive to implement faithfully.
  • For faster iteration, give a sub-agent the production instructions and ask it to act through the failing scenario.
  • Troyanovsky describes simulations as cheaper and less faithful than full forks; confirm proposed fixes in the real system.

Earn wider delegation

  • Start with narrow work and expand responsibility as users observe consistent performance.
  • Adjust human review to the consequences of the task and the agent’s demonstrated competence.
  • Use behavior-check failures to guide changes to the agent’s surrounding software, then evaluate again.
  • Caveat: Basis presents an engineering approach, without comparative reliability results. Process compliance alone cannot guarantee a correct outcome.

“Nobody wants their agent to move 37 their tax return.” — Mitchell Troyanovsky, 18:29