For a repeated operating procedure, ReTrace uses a few logged agent runs to compile a decision list. It extracts the conditions and tool inputs from the agent’s explanations and actual calls, merges them into cited rules, then generates executable code. At runtime, code chooses the next step; a particular tool can still call a model.
- On the public telecom subset of τ²-Bench, the unrepaired workflow succeeded on 75.8% of test tasks versus 80.6% for the original agent. It took 1.3 seconds per task rather than 170 seconds. The test set has only 40 inputs, and its advantage over a simpler few-shot workflow was not statistically clear.
- Across 79 proprietary procedures, the authors report 93.8% completion versus 96.8% for the agent and 24% lower cost per task. These closed-platform outcomes are not independently reproducible. The platform can also mark a run complete when its web-tool calls failed.
- Removing the agent’s reasoning hurts on the telecom tasks and some enterprise procedures, but improves the result on the short SOP-Bench procedures. Repairing rules with labeled training failures raises success; it changes the feedback budget.
Mean enterprise time actually rose from 153 to 163 seconds, although median time fell. An illustrative 75-run break-even uses a public-benchmark construction estimate, not measured enterprise construction cost. A compiled rule still needs review before it is repeated at scale.