Jordy Zomer does program analysis for a living — figuring out how software actually behaves. When he started using LLM agents for multi-hour vulnerability research, they were great at early exploration, then kept losing the plot. Hours in, the model would re-suggest approaches already ruled out, or keep reasoning from assumptions that had been disproven. Dumping more of the old conversation into the prompt didn’t fix it; the model would just re-derive the same stale conclusions.

His reframe: stop asking the LLM to remember, and maintain the knowledge outside the model. He built Lemmalog, a small logic database where you store facts plus rules that derive new facts. The LLM becomes the front-end — it reads code, debugger output, and messy notes, and converts them into structured facts. The database handles the rest: deriving conclusions, tracking what depends on what, and automatically invalidating anything built on a fact that later turns out to be wrong.

Why this matters:

  • Retractions. When an observation changes mid-investigation, every conclusion that depended on it is removed automatically — no re-reading the whole history and hoping the model notices the contradiction.
  • An audit trail. You can ask “why does the agent believe this?” and get the exact observations and rules behind the answer. Unsupported claims can’t silently become part of the investigation.
  • Cheaper long sessions. One benchmark needed ~2,700 tokens per question with Lemmalog versus ~104,000 for full-context prompting. The conversation is read once and turned into facts, not re-read on every query — so long-running agents don’t hit context limits.
  • Honest results. Mid-pack overall on standard memory benchmarks, but best-in-class on “knowledge update” questions (learning A is false, then figuring out what that breaks), and far better than full-context prompting at refusing questions built on false premises.

The deeper argument: “LLM memory” is two different problems. Finding relevant past information is retrieval — vector search already handles that. Knowing what is currently true is state maintenance — a problem databases have been solving for decades. Zomer’s score improvements came almost entirely from concrete engineering fixes (entity matching, date handling, retrieval aliasing), not bigger models.

He’s not claiming victory — the leader on these benchmarks still beats him overall, and inference-heavy questions remain weak. But the direction is compelling: maybe we don’t need a bigger context window every time an agent forgets something. Sometimes we just need to maintain the state.