Justin Tan uses Jev to check meeting notes, retain important email and analyze coding sessions. His month of experiments exposes a familiar problem: a probability cutoff that looks right on one batch can fail on the next.

Choose the cutoff for the cost of a mistake

Tan’s cleanup process moves old promotional mail to the bin. A safety check tries to retain receipts, invoices and personal messages.

  • He reports that a 0.40 keep-probability cutoff worked on his first email batch but missed important messages in another.
  • Lowering it to 0.30 retained 96% of wanted messages across both batches, according to his account.
  • The lower cutoff favors retaining an extra newsletter over binning an invoice.

That combined result includes examples used to adjust the cutoff. Sample counts and unwanted-mail retention are missing, and the revised setting still needs an untouched test batch.

Ask separate questions for separate intentions

An email requests workshop slides and proposes another workshop. Tan reports that Jev selects Delivery with 96% confidence, while a separate new-work question returns 92% probability.

Both answers can help. A single department choice would hide the second intent. Application code has to combine the judgments into appropriate actions.

Send the evidence each question needs

  • Email retention: Tan says sender, subject and Gmail’s short preview worked as well as full messages in his experiment.
  • Meeting commitments: selected excerpts performed worse than whole transcripts, where earlier discussion could qualify an apparent promise.
  • Sensitive remarks: replacing “would not want repeated” with concrete inclusion and exclusion criteria improved his confidentiality check.

These are workload-specific observations. Shorter context helps only when it preserves the evidence needed for the answer.

Keep pilot results in proportion

Tan reports catching 20 planted mistakes across eight meetings, plus one genuine invented item and two false alarms. He also reports about 5,800 session judgments for $0.19, without an independently checked accuracy result for that analysis.

Typed answers make workflow rules straightforward to implement. Correct labels and dependable probabilities still require tests on local data.

“The real test is a fresh batch the model has never seen.”

Verification: read the complete essay and checked its first-party author/date metadata. All experiment and cost figures are Tan’s reports; no private data, live Jev calls or performance results were independently reproduced.