AI safety researcher Ramakrishnan Lokanathan explains how he approaches evaluations for high-stakes AI applications on AI Engineer.
Start with the job
- Define what success requires, what must never happen, and when the system should stop.
- An evaluation is one bounded signal, not proof of a system’s overall ability.
- A perfect score can signal a test that is too narrow or easy; it is not, by itself, proof the test is wrong.
Test the whole workflow
- Check intermediate steps and tool calls as well as the final answer.
- A requested image background change might appear successful while accidentally adding a second handle.
- Combine automated checks during deployment with human review and adversarial tests.
Keep learning from failures
- Retain regression cases for tasks that worked before a model or product upgrade.
- Collect user feedback, reproduce failures, identify the failing step, and add cases to the test suite.
- Add challenging cases as capabilities improve, while keeping stable tests for comparison.
Games as one additional lens
- Lokanathan introduces his open-source eSlams project, which has models play games including chess, Go, and poker.
- He proposes examining sequences of decisions, including responses to changing conditions and mistakes.
- Game performance is an exploratory signal, not a demonstrated substitute for testing the actual product.
“Test the job and not just the answer.”