Clew Labs’ README Clew 2 checks public JavaScript and TypeScript repositories for documentation drift. Its useful boundary: Jev decides which check applies to a README line; deterministic code decides whether the specific package, command, environment variable or path agrees with the repository.

I inspected commit 943209e25a03220140675349fd55fd6e6d21659d, ran the offline suite on Node 24.21.0, and recomputed the analysis of the author’s stored evaluation answers. I made no live Jev or Haiku calls; this is not a security audit.

Routing is not verification

  • Code preserves the actual README line as the quote, instead of asking a model to reconstruct it.
  • Rules settle obvious lines; Jev labels the remainder and selects a name when code finds several candidates.
  • Jev-sourced checkable claims must clear a 0.8 confidence gate; when a name is selected separately, the weaker confidence controls admission.
  • A prose-only package name cannot become “Contradicted” merely because Jev is confidently wrong.
  • Haiku receives structured findings for explanation, not authority over the verdict.

The small evaluation measures the right failure

The author’s stored held-out results contain 32 distinct lines, tested twice after configuration was frozen. My offline reanalysis confirms nine exact-label errors per run, but zero wrong verifier routes passing the gate, with nine of 11 eligible lines reaching the correct verifier. The separate synthetic end-to-end cases record zero false contradictions and three of three planted drifts found. These are checked calculations on supplied records—not independently reproduced live inference.

The distinction matters: confusing “not a claim” with “unverifiable” adds noise; confidently sending a claim to the wrong checker can falsely accuse a correct README. The sample does not establish general calibration, a zero error rate or correctness on unseen repositories.

Code-owned checks still need complete evidence

  • The author documents earlier false contradictions caused by missed pnpm workspaces and misparsed --filter arguments, then adds regression fixtures.
  • Unread source/config files make absent environment variables “Unverifiable,” not proof of a contradiction.
  • “Missing” packages are documentation omissions, not necessarily undeclared dependencies; that label embodies a product policy.
  • Build-output paths and unsupported workspace patterns limit what the scan can establish.

My rerun found a small failure

The offline suite returned 473 passed, one failed, zero cancelled and zero skipped, rather than the README’s all-green claim. The failure is in withoutRepoParam(null): the helper returns ?null= instead of an empty string. It is a URL-cleanup edge case, not evidence that Jev routing failed.

The broader lesson survives: typed output is not a correct decision, and deterministic checks are only as sound as their evidence and rules. Confidence gates, explicit abstention and source-linked receipts make that boundary inspectable; they do not turn a narrow README scan into a proof that the repository is correct.