huemorgan2/say-hi asks a useful absurd question: can a model that only selects from supplied options talk if every keyboard key becomes an option? The result is not a practical chatbot. It is a compact experiment showing that a decision model’s knowledge can be stranded behind a bad action space.

The three-stage loop

  • A first Jev call chooses the reply’s goal: answer, greet, repeat, correct, support, or another bounded intent.
  • A tournament over roughly 10,000 candidate words selects the answer keyword, splitting candidates into groups of 250 before a final among group winners.
  • Jev then chooses one key per request from letters, digits, punctuation, backspace, whole-word completions, and SEND.
  • Every option describes the resulting reply, while deterministic rules remove obvious loops such as double spaces, repeated words, and retyping a deleted character.
  • The server caps a reply at 400 keystrokes and $0.50, exposes each request and probability distribution in the UI, and keeps the API key server-side.

The revealing failures

  • The author reports that Jev could select 12, 19, 2000, or Einstein when offered whole answers, yet could not reliably reach them one character at a time.
  • A first letter is semantically far from a complete answer. Once the correct prefix exists, spelling becomes much easier; the repository reports 96–99% choices for the remaining letters after inv in “invented.”
  • Character hints—especially the real words a prefix can lead to—reportedly moved spelling from about 70% toward 100%.
  • The approach remains weak on open-ended advice, out-of-vocabulary words, and answers that are not one word or a small number.

What was actually measured

  • The repository reports a score improvement from 2.47 to 2.77 out of 3 across 16 iterations of an 18-turn scripted conversation.
  • That score combines whether Jev pressed SEND, produced recognized words, and matched a strict answer regex. It is a small development set, not a general chatbot evaluation.
  • The author’s “Hi” example required four Jev requests and 10,639 reported input tokens. The cost comparison uses provider list prices; it does not establish equivalent quality across models.
  • I inspected the implementation and live-evaluation harness at commit 7b67dd8637d410bda80ef20fb4f5fa07706c5417. I ran the mocked test suite locally: 22 tests passed. I did not run the paid 18-turn Jev evaluation, so the model outputs, score progression, token count, and cost figures remain author-reported.

The actual design lesson

  • A model can contain the relevant fact yet fail when the available next action does not align with that fact.
  • Option descriptions and intermediate abstractions are part of the model system, not neutral packaging.
  • The answer tournament repairs the character-level mismatch by moving the important decision back to a semantically meaningful option set.
  • Whole-word completion, planning, loop guards, and hard budgets do most of the product work around the model.
  • For real software, canned responses or domain actions are a better fit than forcing a chooser to imitate free-form generation.

“Jev knows answers but can’t spell its way to them.”

Say-hi is valuable precisely because it fails in public. It turns “typed decisions” from an API feature into an interface-design constraint: a valid choice is useful only when the choices preserve the meaning of the task.