An invoice can supply a payment amount. The user’s instruction still determines who should receive it. Bo Yan and colleagues build PROVISO around that separation between outside evidence and authority to act.
The controller sits between an agent’s proposal and its executor. It keeps the trusted request, external observations and candidate actions in separate roles. Learned queries compare the proposal with refusal or another execution, then choose an action using outcome-trained scores.
The authors report:
- On 150 malicious and 1,500 benign skill packages, F1 reaches 85.81 versus SkillGate’s 71.97. F1 balances detection against false alarms; 1.00% of benign packages were flagged.
- On 707 ToolSafe calls, F1 reaches 99.12 versus TS-Guard’s 92.05. PROVISO misses one unsafe call and blocks five benign calls.
- On AgentDojo, task completion under attack reaches 85.41%, versus SecAlign’s 83.35%. Both candidate executions are acquired before the controller chooses between them.
Its 17.40 million trainable parameters are fitted separately for each test setting. These are author-reported results, with no demonstrated transfer of one trained controller across all those settings.
Evidence selection matters: replacing the selected package windows sharply reduces detection quality. Removing role embeddings alone has a small, statistically unresolved effect on that package test. The architecture’s authority boundary is a useful design idea, while the ablations support a more specific claim about the complete method.
Caveat: the two-execution experiment requires a safely restorable environment and pays for both candidates. It cannot be treated as a cheap check before an irreversible live action, and controller-adapted attacks remain untested.