Blackstone CTO John Stecher and Norm AI CTO Scott Worland discuss where to invest as models improve in this 21-minute AI Engineer conversation.

Fix the data and automate the stable path

  • Stecher puts accessible, well-managed data ahead of adding agents to a business process.
  • Keep repeatable work in ordinary code, then use agents where the workflow needs judgment.
  • Test model, tool-running software, and data changes against the same task-quality checks.

Keep the tests when the workflow becomes obsolete

  • Worland says Norm accumulated tens of thousands of expert-labeled examples while building its earlier structured workflows.
  • Testing newer agent approaches against those examples showed that much of the old workflow code was getting in the way.
  • The team reset its platform design around agentic client experiences and AI-assisted development.
  • Invest deeply in task evidence, benchmarks, and security boundaries; keep model-specific workflow choices replaceable.

Decide whether to build, prototype, or wait

  • Stecher sometimes delays nonurgent work when expected model improvements could make it cheaper without sacrificing business value.
  • For uncertain ideas, he favors fast, disposable proofs of value that users can try.
  • Production still needs cybersecurity review, supported infrastructure, traceable execution, and monitoring.
  • Norm requires human oversight for changes to core security-sensitive primitives, even as surrounding applications become easier to rebuild.

Reconsider the cost of keeping old software

  • Compare rewriting a small internal application with continuing to maintain it as development costs change.
  • Worland’s metaphor is to build conduits for improving models, while keeping core boundaries locked down.
  • Caveat: these are the speakers’ operating judgments; the discussion supplies no controlled ROI study or numerical comparison of old and new systems.

“Straight through processing still matters.” — John Stecher, 2:27