The November 7 update adds Qwen3.8 Flash Next at its extra-high reasoning setting. MaxxBench changes the requirements or starting code of 89 DeepSWE programming tasks.
A benchmark is a fixed set of tasks for comparing models. Qwen passes 82 of 178 attempts, or 46.1%, across two runs per task.
The site’s 58.7% comparison comes from Qwen’s own DeepSWE results. Qwen reports its best result across two agent setups, rather than a matched rerun by MaxxBench.
The published task files and runner let developers inspect revised instructions, reference patches and separate grading tests. Passing requires every feature and regression check to succeed.
One public Qwen run correctly lowercases browser-report filenames, passing 68 feature checks and 469 regression checks. That example illustrates success rather than explaining the remaining failures.
Caveat: The cross-test score gap alone cannot separate changed task difficulty, execution settings and prior exposure to familiar tasks.
(HN 117 points · 30 comments · collected research account · dated update and complete per-task counts)
Financial documents need checks against what readers can actually see#
Tom Clancy’s November 7 account describes internal tests using white text, off-page text and a forged regulatory footnote that instructs software to replace financial figures.
His mechanical checks compare rendered pixels, extracted characters, character positions and accessibility descriptions. Separate checks look for invisible Unicode characters and overlapping digits.
An agent is a model running a multi-step task with tools. These checks inspect its input documents, rather than ask another model to decide whether an instruction is trustworthy.
Coverage has specific gaps: image-only scans lack extracted text, and PDF accessibility substitutions can fool multiple checks simultaneously. Comparing ordinary rendering with text-only rendering catches some image-based digit changes.
The work remains internal and unfinished. Clancy reports no attacks caught in real documents, gives no aggregate detection accuracy and identifies tinted backgrounds as a false-positive source.
Caveat: This is a practitioner’s test-design account, with documented blind spots rather than evidence that hidden-document attacks are reliably blocked.
(Lobsters · collected discussion · complete methods and stated gaps)
A research-group closure raises a separate maintenance question#
In his November 6 account, Langford says he left after the group’s September move into Microsoft AI. He describes AI Frontiers as roughly 50 people.
Its work included AutoGen, a framework for coordinating agents, and Fara, a model for operating computers. Langford describes a conflict between exploratory research and the receiving organization’s emphasis on scale.
Separately, AutoGen’s current repository says Microsoft provides no further bug fixes, security fixes or support. The README change dates to October 16, so that warning predates this account.
The repository directs users toward Microsoft Agent Framework and its migration guide. Existing AutoGen deployments need a maintenance decision, not an assumption that research-group closure disables their software.
Caveat: The organizational explanation is Langford’s first-person assessment, while the repository independently establishes AutoGen’s existing support status.
(HN 233 points · 132 comments · first-person account, repository notice and dated change)
HyperFlow’s compressed video-and-audio model → OFFSTACK: the checked release targets media generation, a lower-priority fit than coding tests and document safety. Its README also requires authorization for use in the United States.
Karpathy’s TristanTube demonstration → THIN_GUARD: a rapid-build showcase supplies less useful technical evidence than the selected developments. The claimed completion time is not a controlled productivity result.