Matthew Jewkes of Standard Cybernetics describes a Clarivate biopharma-indexing automation and its effect on expert attention in this AI Engineer talk.

Separate three different problems

  • Ordinary errors have an agreed correct answer; unresolved cases need an organizational definition; unusual cases need fresh expert judgment.
  • Bring concrete evaluation examples to experts rather than asking them to populate a blank spreadsheet.
  • The team reviewed about 1,000 examples after an initial agent-like approach reached roughly 80–83% performance, according to Jewkes.

Keep routine work predictable

  • Use input screening and routing to determine whether a case belongs on the normal successful path.
  • Route ambiguity and unusual inputs out of that path instead of forcing an answer.
  • Give business-critical cases a heavily tested route, with human escalation when uncertainty remains.
  • Jewkes says routing required at least as much effort as improving the final critical pipeline.

Scope the accuracy claim

  • After separating ambiguous and exceptional cases, Jewkes reports moving selected critical tasks from about 95% to 99.95% accuracy.
  • He says a random sample outperformed historical human work at scale, while an expert examining one case carefully could still do better.
  • The talk does not supply the final sample size or the fraction of inputs excluded through escalation.

Organize what remains for human attention

  • Group critical exceptions for focused review rather than scattering interruptions across an expert’s day.
  • An ambiguous backlog can justify freezing low-value work until there is a business case to resolve it.
  • Recurring unusual cases can reveal a new product need before a customer articulates it.
  • Jewkes describes an expert forming an investment thesis from grouped exceptions before an eight-figure account raised the same need.

Choose a process worth the investment

  • Favor scaled, detail-oriented work with an existing business case and rules that can be specified.
  • Budget for evaluation work, ownership changes, and trust-building alongside implementation.
  • Preserve human flexibility during early product discovery and for work dominated by subjective judgment.

“This is a form of organizational refactoring.” — Matthew Jewkes

Caveat: The accuracy and customer outcomes are speaker-reported. The 99.95% figure applies to selected business-critical tasks, not every incoming case.