Amazon engineers say a small decision model took over labeling internal tickets from Claude Sonnet 4.6. The task has 20 categories, so the replacement can score fixed choices instead of generating a long answer. They ran both models on real tickets before switching, with Sonnet’s label still applied during the comparison.

  • Cost: At published list prices and their usage levels, the team estimates about $22,200 per million tickets for Sonnet versus $500 for Strands Decider E4B. The GPU estimate depends on keeping the server busy enough; neither figure includes Lambda costs.
  • Speed: Average call time fell from 4,215 to 897 milliseconds. A three-way configuration switch let them compare both paths and, they say, roll back in 45 seconds.
  • Input changed too: The team shortened label descriptions and dropped the oldest comments on long tickets to fit the smaller model. Rearranging JSON keys shifted some predicted probabilities by as much as 0.19, so they now sort the keys.

The engineers say Decider beat Sonnet on a separate ticket set, but publish no accuracy table, sample size or breakdown of costly mistakes. This is a vendor-reported production case, not an independent test. Side-by-side disagreement review is worth borrowing even if a simpler classifier ultimately wins.

The 213-comment thread on Hacker News adds two baseline questions the article leaves open.

What the thread adds

  • armcat — describes a previous ticket-classification pipeline trained with support-vector machines: “it took a couple of minutes to train, and the ticket classification was just a few milliseconds.” That is a different system, not a head-to-head comparison.
  • ryankrage77 — reports a former workplace’s conventional classifier trained on about 30,000 examples and reached roughly 85% accuracy; again, a different dataset.
  • Kaedon — asks whether the annual savings claim implies around 92 million tickets to save $2 million. Amazon does not give its annual ticket volume.

What was the right baseline? psnehanshu asks why a cheaper general model was not tried, while 3eb7988a1663 asks whether anyone tried scikit-learn. The article does not report either comparison.

From a 213-comment thread, in HN’s own order. Handles are pseudonymous; HN publishes no per-comment scores, and this slice is not a consensus.