Pusu’s local-decider puts deterministic checks around model-selected actions and lets a browser agent swap hosted Jev for a local decision endpoint. The useful evidence is a failure, not a speed pitch: the local model filled a booking form and declared completion without submitting it.
I inspected the implementation and published logs, and ran the rules-only verifier. I did not run either neural model or the live browser experiment; this is not a security audit.
The model proposes; code checks
policy.pysends the goal, observed page elements and recent actions to/v1/systemone;TYPESAFE_BASE_URLselects hosted or local serving.- One request asks for the operation and possible targets; the executor consumes the target for the selected operation.
- A target head with only one candidate is resolved locally, avoiding a documented local-server schema mismatch.
- If the model says
DONEwhile a requested submission control is visible, enabled and untouched,done_guard()substitutes a click. Completion is checked against page state, not just the agent’s report.
A reproduced result—with a narrow boundary
- On the repository’s 112 labeled cases, our execution of its verifier resolved 61 without a model: 58 correct, three unnecessary confirmations, and zero false allows among those resolved cases.
- Pusu reports full-pipeline results of 105/112 locally and 106/112 with Jev, with zero leaks in both. Those neural results were not reproduced here.
- The README corrects an earlier “61/61” printing bug;
docs/RESULTS.mdstill contains the stale figure. Use the corrected README and actual verifier output.
Typed output is not decision correctness
- Pusu’s logs show the local
winnow:e4bendpoint stopping before submission; the guarded runs completed the same fixture. One page and one goal do not establish general browser reliability. - The Hammer chat-model shim emits confidence 1.0 and probabilities 0/1 by construction. These are response formatting, not measured uncertainty or calibration.
- The guard matches submission words in the goal and control names. It can over-click irrelevant controls, and it does not prove a server accepted the submission.
Do not deploy the gate as a general permission boundary
- The safety harness recognizes a small set of dangerous tool names; unknown names are allowed as “not irreversible.” Renaming a sending tool can bypass that classification.
- Its whitelist includes
write_fileandcron_create, and runs before the negation check. That ordering is inappropriate for a broad authorization boundary. - Negation and approval detection are largely Romanian-specific; email matching accepts loose recipient-name matches rather than proving every recipient was authorized.
- Keep explicit tool schemas and application-level permissions outside the model. Retest multilingual requests, recipient lists, malformed arguments and real completion evidence before any consequential use.
“DONE is a claim, not evidence.”
The practical contribution is an inspectable migration experiment: endpoint compatibility preserves the client, but it does not preserve behavior, confidence semantics or permission safety.