gurn3k’s AI PM Job Radar uses Jev to sort product and program-management openings. Its strongest contribution is an inspectable test: saved model answers, human labels, a scoring script, and an account of where the labels changed.

Separate location rules from semantic questions

The author reports that Jev misread Vancouver locations as US-only. Location handling moved into ordinary code, alongside pay extraction and title filtering.

Jev answers questions about job type, the team’s AI work, seniority, and required skills. The specification separates an ML background from practical experience building with AI. Those requirements had initially been conflated during labeling.

The location rules still need testing. Reading a location field cannot establish a candidate’s work eligibility or an employer’s remote-work policy.

Keep the denominators visible

The project reports results on 40 hand-labeled postings, sampled to include each predicted bucket. An offline rerun of its scorer and a separate recount reproduced these saved-data totals:

  • 26/36 accepted postings match the human’s exact bucket; four additional postings go to review.
  • 32/36 accepted postings agree on the broader question: AI target role or not.
  • In the originally blind half, exact agreement is 11/18 accepted postings; broader agreement is 15/18. Two go to review.

These are comparisons against the author’s labels. Fresh Jev calls and the underlying job descriptions were not independently evaluated.

Revised labels narrow the claim

The author revised labels in the first half after seeing Jev’s answers. The strict ML-background question was relabeled across the set. The broader hands-on-AI question was written after the author had seen their own labels.

That makes the originally blind half informative, but leaves no pristine held-out test for every reported metric. The bucket-balanced sample also cannot estimate accuracy on a random job opening.

Check what the confidence threshold actually gates

The inspected rules apply a 0.6 confidence floor to job type. They do not require that confidence for AI scope or the ML-background decision. Typed output makes the answers easier to route; it does not make them correct.

The author reports 1,018 labels in 15 seconds for $0.107 on an earlier run. Those speed and cost claims were not reproduced. The runner uses concurrent requests and a hard-coded input-token rate, so the headline describes that setup.

Caveat: README, implementation, and frozen evaluation files were inspected at commit b676399cbfc40c60d0cc42b9e60ffff696dae18b; only offline scoring was run, not live classification or a security audit.