Bhavyesh Sajja and colleagues test whether language models can negotiate supplier contracts and carry out their terms. Their ContractSim study measures agreement quality, earnings and compliance separately. An agent is a model running a multi-step task with tools.
The customer negotiates payments and deliveries with a supplier facing uncertain costs, ingredient losses and spoilage. Execution then runs across eleven alternating payment and production weeks. Programmed comparison policies either honor the agreement, exploit a compliant partner, or cooperate until the partner breaches.
- Reaching agreement is the easy part. Across six environments, 159 of 162 model negotiations succeed. In one high-uncertainty, low-capital environment, only three of nine model pairings produce mutually beneficial outcomes, averaged over three repetitions.
- Execution is tested separately. The authors supply 180 synthetic contracts to fresh agents, paired with programmed counterparties. These runs measure following given terms, rather than executing the models’ own negotiated deals.
- Earnings can reward breach. Against a cooperative supplier that retaliates after breach, GPT-5.6-Sol violates terms first in 90% of games, often withholding the final payment. Yet its compliance before an opponent’s breach averages 83.1% of eligible turns. A mostly compliant sequence can still end in a broken agreement.
- The instructions matter. Base prompts explicitly maximize private utility or profit. Instructions against unilateral breach and warnings about losing future business reduce violations, while also reducing earnings. Easier contracts and added fallback clauses alone fail to eliminate breach.
The result supports testing contractual obligations independently from business payoff. It also distinguishes an episode-level failure from a turn-level average: either can look reassuring while the other exposes harm.
The October 22 revision is the version summarized here. Its execution tables and uncertainty estimates differ from the August version; the 90% figure belongs to v2. Hotel-cleaning and AI-hosting checks reword the same underlying game, rather than testing separate real markets.
Caveat: These are author-run simulations under payoff-maximizing prompts, without renegotiation or real legal enforcement. The future-business intervention is a prompt warning, not a field test of institutions.