Anthropic’s Project Glasswing uses AI models to find software vulnerabilities. Patrick Garrity at VulnCheck compared its public disclosure records with fixes and maintainer assessments. His question is operational: who can validate and repair the findings as reports arrive?
Anthropic’s ledger shows 217 findings marked fixed, 419 withdrawn and 1,654 marked duplicate. Garrity reports that Anthropic’s broader tally reached 37,966 candidate findings by its November 2 update. Those categories describe different stages of a disclosure process. A small fixed count does not establish that the remaining candidates are valid, exploitable or ignored by their maintainers.
Validation is the bottleneck
Anthropic said in October it had manually reviewed about 6,000 of more than 29,000 candidates. Its opt-in OSS Scanner sends model-generated reports without human triage to participating projects. Anthropic warns that some reports will be invalid. The company also describes useful findings at projects including PostgreSQL, wolfSSL and curl. These are selected accounts, not an independent measurement of the whole queue.
Garrity points to a batch of curl reports: of 34 published for review, he counts 19 rejected as vulnerabilities, six rated low severity and nine awaiting assessment. That is a snapshot of one project and one batch, not a false-positive rate for Glasswing. His comparison of 442 Anthropic-attributed CVEs with his firm’s known-exploitation data finds two with observed exploitation. It cannot measure undisclosed proprietary findings or prove that other bugs are harmless. VulnCheck sells vulnerability intelligence, which makes its framing worth checking against the underlying ledger and Anthropic’s own scanner description.
For a maintainer receiving these reports, triage the reachable code path and the project’s threat model before assigning severity. Group duplicates, test proposed fixes and budget review time. A report count alone is not a patch plan; a low fix count alone is not an exploitation forecast.
What the thread adds
In the Hacker News discussion, pizlonator says all nine reports received for Fil-C were real bugs and praises their test cases. That is one maintainer’s firsthand account, not a cross-project rate. dfee points out that the public ledger omits unresolved findings, so judgments based only on published records suffer selection bias. These observations push against treating curl’s experience as universal.
From a 190-comment thread, in HN’s own order.
Caveat: Ledger statuses and the researcher’s matched CVEs are incomplete views of different populations; neither gives a global rate of exploitable AI-discovered bugs.