Sean Chen of Sean’s AI Stories uses a whiteboard and his Waku Agent dashboard to show where a structured decision model differs from a conversational LLM. This is a presenter-run demo, not an independent benchmark.
What goes into the model [06:01–10:30]
- Jev takes a state (the situation) plus named questions.
noulasks yes/no,choiceselects among labeled options, andscoregives an ordered level. - Its response includes probability distributions; an output in the required shape is not evidence that the chosen action is correct. Chen says he could not establish precisely what the separate
confidencefield means.
The live examples [10:41–19:46]
- A fundraising-message example asks whether the sender seeks investment, what action they seek, and how urgent it is. The answer follows the supplied wording; no real investment decision was validated.
- Chen’s dashboard compares Jev with Claude Opus, Haiku 4.5, and GPT-5.4 Mini using 15 human-labeled questions per category, spanning inbound investor messages, memory retrieval, support, refunds, and leads. The video walks through some runs, not an independently replicated study.
- In the shown investor-message run, Chen says Jev gets 13 of 15 right, behind Opus. He says many disagreements involve ordered scores; he also shows examples where the smaller chat models disagree on a yes/no classification.
Where it fits [19:47–22:10]
- Chen reports Jev returning a shown batch much faster and cheaper than Opus in his harness; these are his measurements and pricing assumptions, not reproduced here. His own takeaway is routing, retrieval decisions, and quick bounded checks—not replacing research or multi-step reasoning.
- The small, presenter-labeled set cannot establish calibration, reliability in deployment, or the vendor’s much broader speed claims.
“I don’t think it’s always performing the best in terms of the final results.” — Sean Chen [20:43]