Memory Tax compares eleven memory systems with search over the original conversation. It asks whether extracting and organizing memories earns its extra cost.
The authors separate new answers that memory enables from answers lost relative to raw-history search. A memory can preserve a fact yet make it harder to find.
- On LongMemEval-s’s 500 questions with Qwen3.6-27B, raw-history search scores 77.6% with one round and 81.4% with two. OpenViking scores 58.8% in the tested configuration.
- The baseline combines word matching, meaning-based retrieval and reranking over dated chunks. It is more substantial than vector search alone.
- Retrieving original history alongside constructed memories raises SimpleMem from 73.6% to 82.8% on that test.
- Memory also wins some comparisons. On PersonaMem-v2 with Qwen3.6-27B, MemOS scores 50.2% against two-round retrieval’s 48.7%.
These are author-reported, model-graded results. Retrieval settings expose comparable store fractions, rather than equal input-token budgets. Some datasets and model columns use subsets.
The release provides results and shared utilities, but complete baseline reproduction code remains unfinished. OpenViking’s pinned setup still leaves run commands and retrieval settings as TODOs.
Caveat: These mostly static English tests cannot diagnose a differently configured, long-running memory installation.
A path back to original evidence can help recover lost facts. Actual deletion and forgetting requests still need to govern that fallback.