GC AI’s Bardia Pourvakil describes a day of Jev experiments across legal-product features, developer tools and a Slack pet. The useful contribution is the failure map and published design guide—not the project count. I read the full article and guide; none of its measurements were reproduced here.

Batch questions, not unrelated records

  • Ask several narrow questions about one document together; send separate documents through separate requests.
  • GC AI reports 96% accuracy with individual requests versus 72% with similar items packed into an array, attributing the drop to answers bleeding across rows.
  • Let code compute dates, counts and observed system measurements. The team reports Jev failed four of five weekday-calculation questions.

Fix extraction and criteria before changing models

  • The terms-of-service scanner initially flagged cookie banners as contractual data-sharing clauses. Excluding modal dialogs fixed the extraction mistake.
  • A broad liability question also flagged refund policies. Asking specifically about damages, losses and harm made the criterion more useful.
  • For citation checks, “does the source support this?” missed a may → must overstatement; separate checks for support, mismatched details and overstatement caught it.

Concrete signals beat a holistic verdict

  • GC AI reports a useless-test detector improved from 0.54 to 0.72 AUC after replacing one broad judgment with factual questions about test structure. AUC measures separation of the labeled classes, not production accuracy.
  • Its alert-triage prototype combines classification with facts computed by code; its PR Stamper retains hard rules for sensitive paths and requires a human label. It approves but never merges.
  • Jeventus, the Slack sloth, illustrates the same pattern: classify an action, update state in ordinary code, choose a deterministic reply.

Keep the uncertainty and permission boundaries honest

  • Add “none” when no option fits, and a separate check that there is any basis to decide. The guide reports confident forced choices even without supporting context.
  • Tune thresholds on labeled examples and test on held-out files or PRs; a typed answer is not a correct decision or a calibrated probability.
  • The article’s scanner reports 89% F1 versus 57% for keyword matching on synthetic held-out clauses. These are GC AI’s small-study results, not demonstrated legal-review reliability.
  • The guide recommends fail-open behavior for some guards. Do not generalize that to permissions: failures around irreversible actions should require explicit application policy or human approval.

“Numbers come from small, mostly synthetic sets, usually one run. Treat them as direction, and rerun on real data before a Jev call gates anything.”

The published pattern matrix is a useful next stop; its underlying project links and measurements were not independently validated in this capture.