Jeff Crume examines trust in automated AI decisions in this 13-minute IBM Technology explainer.

Confidence needs evidence

  • Crume contrasts a language model’s inspectable text with Jev’s compact, structured decisions and confidence scores.
  • A security system saying “contain this host, 97% confidence” leaves the operator with questions about the evidence and consequences.
  • Known malware, a malicious network address and unusual behavior would each justify different follow-up checks.
  • Caveat: the 97% score is an illustration. The video presents security design concerns, not a demonstrated Jev vulnerability or accuracy comparison.

Protect the information feeding the decision

  • An attacker could alter logs, metadata, training data or the workflow supplying the model.
  • A model can respond consistently to corrupted information while still producing a confident, harmful answer.
  • Record the inputs, consulted data, rule triggers, final action and responsible actor in an independent audit trail.

Break big decisions into smaller checks

  • Separate authorization, supporting evidence, risk assessment and the final action into observable steps.
  • Type checking confirms that an answer has the expected format; logging records what happened around it.
  • Neither check independently establishes that the prediction is correct.

Watch for plausible mistakes over time

  • Crume imagines a compromised system becoming just 1% more likely to approve harmful requests.
  • Individual answers might still look reasonable, while past success makes operators less critical.
  • Monitor changes in behavior and compare outcomes with independent signals.

Keep controls outside the model

  • Validate inputs, maintain audit logs and investigate anomalies.
  • Escalate high-risk decisions to people who can check the evidence independently.
  • Apply stronger checks when AI can block an account, pay an invoice or shut down a server.

“Don’t let Jev be the only thing that knows what happened.” — Jeff Crume, 11:22