Why AI Rating Scales Make It Harder to Ship — Hamel Husain
Hamel Husain argues that ROUGE/BLEU string-similarity scores and 1-to-10 rating scales are holdovers from traditional NLP that make evals harder to act on — nobody knows what a 4.2 means, and binary pass/fail evals are what actually let you ship.