Harvey’s Joey Wang reports operational lessons from Spectre, an internal cloud platform used to run coding agents. His counts and performance numbers describe Harvey’s own deployment, not a reproduced cross-company benchmark.
Make the agent interchangeable; own the harness
- Engineers can bring preferred agents while Spectre supplies isolated workspaces, access to the real stack, review rules, evidence and the path from alert through CI to a pull request.
- Wang says more than 10,000 agent-written PRs and about 460 distinct internal users had gone through the platform. He describes a follow-up loop that reacts to CI failures before human review.
- Exposing platform actions through a CLI let other agents start tasks; Wang reports about 6,500 terminal-initiated tasks in October. Checkout of repositories became a latency bottleneck, so the team kept reusable local copies.
Bound concurrent work before it overwhelms the platform
- One alert per workspace caused roughly 900 duplicate investigations in four hours. Repeated alerts now join an existing investigation.
- A burst of roughly 950 terminal-started tasks filled the cluster; a fair launch queue and visible position replaced the unrestricted start path. Wang reports a load-test latency improvement for accepting tasks, not end-to-end task completion.
- Scheduled automations can fail repeatedly or spend unchecked; Harvey introduced a five-failure shutdown and optional budget or expiry.
The bottleneck moves downstream
- Wang says company PR volume rose about 60% in a month, while merges slowed from 15–30 minutes to over an hour. He has not solved the merge queue problem.
- His reported platform improvements changed several things at once; he does not attribute outcomes to one queue or one model.
- Review proofs, stable mainline CI, secrets and fair scheduling matter after code generation; agent-written code still needs human accountability.
“The work that’s left is the harness that runs it, checks it, and gets it merged.” — Joey Wang