Lightfield’s Ves Stoyanov proposes a useful division of labor: Claude builds and revises a readable classification policy; Jev applies it repeatedly. The article reports experiments rather than just a product pitch. I read the full writeup; I did not run its released tools or reproduce the results.

Rules as the deployable artifact

  • The builder edits question instructions and option descriptions, often splitting broad classes into finer options and mapping those back to the required labels.
  • With all training labels available, Stoyanov reports its Opus-built classifiers matched or exceeded the study’s fine-tuned RoBERTa baseline across six tasks. These comparisons use one run per cell, not a demonstrated state-of-the-art result.
  • Under a simulated-user budget worth roughly 50 labels, the article reports Banking77 routing accuracy of 91.3, against 73.7 for a description-only Jev classifier; these are author-reported scores on a 0–100 scale, not reproduced here.

Ask where the missing knowledge lives

  • When intent is evident in the text, labeling uncertain examples helps refine the boundary.
  • When team assignments follow a private policy, a wrong mapping can remain highly confident: uncertainty sampling never selects the records that would expose it.
  • Ask directly about routing rules, then check the answers against labels. The study’s simulated user initially described stricter rules than its own annotations followed, changing which strategy looked best.

Turn examples into rules, not a scrapbook

  • The builder frequently pasted purchased examples into its policy. A small matched experiment replacing these with general rules scored similarly or slightly better, according to Stoyanov; the author treats this as evidence that examples are unnecessary, not proof that rules always win.
  • Donation outcome and deal completion offered little or no improvement over zero-shot classification. More interaction is not automatically useful.
  • Typed answers do not guarantee correct decisions. Before deployment, use real human labels, random audits alongside uncertainty sampling, and held-out examples to test policy revisions.

The released classifier-builder recipes and tools make this more inspectable than a generic “agents will maintain classifiers” claim. Continuous monitoring and updating under changing policies remain future work, not a production loop established by these experiments.