The AI Automators uses Jev’s experimental image inputs to check whether a support ticket’s screenshot actually supports its reported problem. This short demo is more useful for its review policy and error breakdown than its speed headline.

I read the complete transcript and checked TypeSafe’s image and Box documentation. The demo’s performance, cost, and quality figures are the presenter’s reports, not independently reproduced here.

0:34 — Check the evidence, not just the attachment

  • Ask whether the screenshot supports the claim, locate the relevant region, and send weak or disagreeing answers for review.
  • On 200 public bug reports, the presenter reports 112 sent for review and 79 accepted; routing counts do not establish accuracy.
  • The claimed run takes about four seconds and US$0.02; that is a reported demo cost, not the total cost of a support workflow.

4:01 — A returned box does not mean “found”

  • Box always returns coordinates, even when the requested target is absent; TypeSafe’s documentation confirms this limitation.
  • Ask a separate yes/no presence question in the same request and ignore the box when presence is not sufficiently supported.
  • Images can also define answer options or reference examples; a coarse tile selection followed by a crop can help with small targets.

“There is no not found answer for the box question type.” — The AI Automators, 4:08

6:25 — The controlled test needs its denominator

  • On 1,000 synthetic screenshot/ticket cases, the presenter reports 934 correct decisions and 66 errors.
  • Its 65% confidence gate leaves 899 automatic decisions with no observed errors and 101 review cases containing all 66 errors.
  • “No observed errors” applies to that selected synthetic subset—not real support tickets or future production traffic.
  • The reported batch takes about 20 seconds at US$0.11, with 16 concurrent tickets; this is not single-request latency.

8:38 — Confidence and calibration answer different questions

  • Confidence controls this demo’s review gate; calibration asks whether stated answer probabilities match observed correctness.
  • The presenter reports 95.3% average probability on the error-free automatic subset, but roughly 48% on the review subset, where correctness is only about 35%.
  • Those are two subgroup comparisons, not a full calibration study, and the video does not establish an independently tested threshold-selection procedure.

Treat this as a prototype pattern. TypeSafe explicitly says images and Box are not ready for production, requires the TypeSafe-Danger-Zone-Images: true header, and warns that the interface and accuracy can change. Typed output is not a correct decision; a review gate must be validated on your own cases before it authorizes consequential action.