Shyam Agarwal and colleagues studied how developers use written specifications to guide AI coding agents. Their October revision combines interviews with 21 practitioners, including eight tool makers, Reddit discussions and analysis of 266 public repositories.
Participants often reported more reliable agents, alongside extra planning, review and maintenance work. The repository evidence shows what people committed, while the interviews describe benefits and costs.
- Specs mix behavior and implementation. In a weighted sample of 1,000 documents, an estimated 77.6% included implementation detail. Naming functions or library versions can help an agent today and become stale after a refactor.
- History serves a different purpose from current instructions. Archived change records explain earlier work. Living specifications describe what the system should do now. Agents need to distinguish them.
- One commit does not prove neglect. Although 81.5% of spec files had one recorded commit, archiving moves and squashed commits hide earlier edits.
- Acceptance criteria still need human judgment. If an agent writes the spec, code and tests, one misunderstanding can spread through all three.
The authors recommend keeping durable requirements separate from temporary implementation instructions and making spec maintenance part of the workflow. They also propose testing whether the benefit comes from people clarifying goals before delegation, rather than from agents receiving a particular document.
Caveat: this observational study cannot separate reported reliability gains from improving models. The linked study replication package currently returns 404, although the separate SpecMine dataset repository is accessible.
The record began as the two-author SpecMine corpus paper in August. These practitioner findings belong to the expanded five-author v4, published October 15.