Luigi’s survey of four Pi coding-agent extensions is unusually concrete about where a decision model fits in an agent loop—and where it does not. The strongest material is the execution gate’s measured fixtures and a threshold rule that can become less safe when “tightened.”

Before a tool runs

  • Deterministic rules immediately allow known read-only commands and block obvious destructive patterns; only the ambiguous middle goes to Jev.
  • The request includes command text, working directory, recent user intent, and local policy notes—not file contents, diffs, or terminal output.
  • Conditions cover intent, secret egress, irreversible damage, protected paths, fetched-code execution, and prompt injection.
  • Clear violations block, while uncertain results pass by default unless the operator selects an uncertainty-deny mode.
  • If Jev is unavailable, unvouched commands fail closed rather than silently running.

The threshold trap

  • The extension uses three bands: satisfied at p ≥ t, violated at p ≤ 1 − t, and uncertain between them.
  • On 18 real command fixtures, an attempted private-key upload scored 0.02 on no_secret_egress.
  • At a threshold of 0.97, the rejection cutoff is 0.03, so the command blocks.
  • Raising the threshold to 0.99 moves the cutoff down to 0.01; the same 0.02 becomes “uncertain” and passes under the default policy.
  • These are the extension author’s reported fixture results. I reviewed the article but did not run the extensions or reproduce the Jev calls.

After a tool runs

  • A post-call judge checks tool output for leaked secrets and classifies failures as transient, environmental, code, permission, user, or no failure.
  • Advice comes from deterministic lookup tables; Jev chooses the class but does not write remediation prose.
  • The default scope is shell-like tools rather than every file read, limiting both request count and latency.
  • This complements the pre-call gate: intent can be judged before execution, but actual output and failure type only exist afterward.

Tools, skills, and context

  • jev_find_tools and jev_find_skill rank inactive capabilities and load only those above a threshold, keeping Pi’s base prompt small.
  • One compaction extension scores which tool outputs to retain verbatim; another uses Jev to identify important history before a conventional summarizer runs.
  • The article also describes routing among fast, reasoning, long-context, and vision models, though one implementation uses local heuristics rather than Jev for that decision.
  • A standalone pi-jev-gate turns a rubric check into exit codes, so CI or a subagent workflow can stop when a diff fails a typed criterion.

What Jev cannot make safe

  • The gate inspects command text, not behavior. An approved process still runs directly on the host and can do more than its command line reveals.
  • A decision model is not a sandbox; least privilege, network restrictions, and isolated execution remain the real containment boundary.
  • Passing uncertain calls by default is a product tradeoff against approval fatigue, not a security guarantee.
  • Typed output prevents malformed answers, but Jev can still select the wrong allowed answer or be influenced by adversarial state.
  • Thresholds need labeled local fixtures and explicit false-allow costs; returned probabilities should not be treated as trustworthy merely because they look precise.

When a decision model is the wrong tool

  • Dependent fields do not fit a single parallel call because every question sees the same state, not the other answers.
  • Lookup-then-compare tasks need tools and code, while prose generation still needs a generative model.
  • The article argues that very short outputs may see little latency gain beyond schema reliability.
  • It also warns that open implementations can reproduce the scoring mechanism without reproducing TypeSafe’s probability calibration.

“Tightening the gate let the credential upload through.”

The useful pattern is not “ask Jev before everything.” It is deterministic policy at the edges, a measured semantic classifier in the middle, and hard containment underneath both.