Natalia Connolly and Kevin Madura of AlixPartners explain a post-merger procurement analysis, using a synthetic replica to illustrate the case.
Plausible answers, wrong decisions
- The real engagement involved hundreds of vendors and more than 40,000 contract documents across separate systems.
- In the illustrated case, Claude Code’s contract value missed about $800,000 because an amendment was not linked to its parent.
- A renewal deadline was a month late: the agent used a termination date rather than the date derived from the renewal clause.
- Citations down to the contract page or invoice line let people check answers before negotiations.
Structure before analysis
- The team describes two weeks building a pipeline for preprocessing, classification, extraction, spend mapping and agent analysis; processing the contracts then ran overnight.
- Preserve context across page-spanning tables, link amendments to their agreements, and distinguish a restated total from an amount to add.
- For charts, they describe feeding page images through DSPy to produce structured data instead of relying on text-only OCR.
- Custom tools and database views encode deal-specific logic, such as retrieving a vendor’s spend or rate-card page.
Make failures visible
- Return explicit access-denied errors: an agent that treats an inaccessible invoice as zero can produce a convincing but false total.
- Reconcile parallel agents’ savings proposals to avoid counting the same opportunity twice.
- Include business context, including projects already underway, so the agent does not recommend work already in progress.
The speakers report their own engagement, not a controlled accuracy benchmark; the company names and example are synthetic.
“Invest some tokens and effort and money upfront to structure your data first.”