Milind Sunil Shah of Brainbase Labs explains how his team improved a Domino’s competitor-pricing agent in this 13-minute AI Engineer talk.
Measure a small, real job
- The agent collects local competitors’ prices across websites and delivery apps, then fills a reviewable spreadsheet.
- Shah reports accuracy rising from 41% in version 1 to 97% in version 11 through instructions, skills, and playbooks.
- Reported cost falls from $2 to $1 per source, defined as an item–competitor–website combination; runtime falls from 129 to 71 minutes.
The six-step loop
- Kickoff: agree on the output, human sign-off, baseline effort, and forbidden assumptions.
- Ontology: define items, delivery versus takeaway, bundles, checkout prices, discounts, and unresolved gaps.
- Tests: combine code checks, a model reading traces, and an agent revisiting flagged prices.
- First agent: start with one app, one item, and one city so its failures are inspectable.
- Iteration: score, fix, and ship one change at a time; add discovered failures to the test set.
- Handoff: assign ownership of runs, exception review, rule changes, tests, and release decisions.
Two fixes from actual traces
- A linked pizza section hid a cheaper deal farther down the page; the team added a whole-page search rule.
- Another run kept zooming into 26 promotional images after finding the right card.
- Checking page structure and image metadata first, plus a zoom limit, cut image reads to one-third and cost by over half in that example.
Keep the limits visible
- Archived websites could not reproduce delivery-app states, so the team retained browser screenshots from live runs.
- Analysts still review exceptions, and the talk describes expansion into additional markets as ongoing.
- Caveat: these are Brainbase’s project results; Shah does not disclose the accuracy denominator, evaluation-set size, or held-out testing protocol.
“Version one’s job is to be measured, it’s not to be good.” — Milind Sunil Shah, 8:05