Public math proofs need more than a download link
- AI-assisted math proofs differ sharply in what readers can check — New public proof files show why a readable argument, a formal proof and the evidence needed to reproduce a result are separate deliverables.
- Terence Tao lists a human-readable Euler fluid-equation proof by Levent Alpöge, Tristan Buckmaster and Matei Coiculescu, with heavy AI assistance. Its Lean version uses software that checks formal mathematical arguments.
- For the Navier–Stokes fluid-equation proof, Tao says the broad strategy is clear but a professionally written explanation still needs work. These new fluid results concern equations without an external stirring force. September’s coverage gives background on earlier lab work and its credit dispute.
- The Navier–Stokes repository discloses a separate reproduction gap: its numerical certificate and rigorous checking computation are absent. It also says the latest patched formal package was not rebuilt before release.
- Tao describes Alpöge’s Generalized Riemann Hypothesis claim, concerning prime-number patterns, as an enormous Lean-only argument generated with an internal model. He points to ongoing efforts to verify it and extract its strategy.
- In a separate human-written proposal, Ravi Vakil’s update reports problems identified with assistance from different AI models. That concerns Alessandro Masullo’s proposal, distinct from Alpöge’s formal files.
- Caveat: Public formal files and expert commentary do not establish completed independent verification of every claimed result. (HN 462 points · 191 comments · HN 2157 points · 1002 comments · Tao, Vakil and primary proof repositories)
An outside check on yesterday’s model release
- Artificial Analysis tests Inkling 2 Mini’s accuracy and running costs — An outside comparison gives developers evidence beyond the launch claims, while separating hosted running costs from performance on their own machines.
- Inkling scores 20 on the Intelligence Index, matching Qwen3.8 27B at its highest reasoning setting. This composite combines tests of work, coding, science and general knowledge, rather than measuring a percentage of tasks solved.
- It averages about 152,000 output tokens per task versus Qwen’s 110,000, roughly 38% more. Tokens are the pieces into which models divide text. Task costs therefore need to include the full answer length.
- Artificial Analysis reports $0.59 and 9.0 minutes per task, versus $3.53 and 38.2 minutes for Qwen. Inkling’s cost uses Thinking Machines’ first-party API pricing, including cached-input rates.
- This comparison leaves cheaper alternatives: GPT-6 Luna at maximum reasoning scores 28 at $0.15 per task. The results support comparing quality and total task cost, rather than assuming a smaller model always costs least.
- Yesterday Afternoon’s release coverage supplies download and runtime requirements. Today’s addition is the outside evaluation, without repeating those specifications.
- Caveat: These hosted test results do not establish laptop speed or the cost of running the model yourself. (Techmeme · Artificial Analysis evaluation and published methodology)
Make repeated conversations cheaper to serve
- Saving model state at conversation boundaries cuts repeated work paper notes → — FrontierReplay offers a concrete serving design for reusing earlier conversation processing without repeatedly interrupting the server to save it.
- Hybrid models combine attention with a fixed-size running state. FrontierReplay reconstructs exact saved states at useful conversation boundaries after prompt processing, using a short record of internal inputs.
- The implementation uses vLLM 0.30.0. Its main tests cover Qwen3.5-9B and Falcon-H1-7B across four workloads, on an A800 with 80 GB GPU memory and a 32 GiB cache budget.
- In Falcon coding-session replays, median first-output wait falls from 1.1 seconds to 125 milliseconds. The rate of calls meeting both response targets rises 4.26-fold. Those targets allow 500 milliseconds to start and average 50 milliseconds per output token.
- Qwen’s coding-session gains are smaller: first-output wait falls from 173 to 107 milliseconds. Its target-meeting call rate reaches 1.11 times the baseline’s, showing how benefits depend on the model and workload.
- A larger-model document test reaches only 0.76 times the baseline’s target-meeting call rate because separate memory pools restrict capacity. Exact saved state therefore offers no guarantee of higher serving capacity.
- Caveat: These controlled workload replays fix recorded replies, so the measurements exclude changes in what a live assistant chooses to do next. (arXiv 2611.26020 · full system design, experimental setup, results and adverse scaling case)
Test repairs and the graders behind them
-
Adding bugs exposes a gap between recovered scores and repairs paper notes → — BuggyMLE shows why checking a repaired application’s output score can miss faults that remain inside its code.
- Researchers create working repositories for 22 machine-learning competitions, then inject faults at four difficulty levels. A benchmark is a fixed set of tasks for comparing models. An agent is a model running a multi-step task with tools.
- Claude Opus 5.5 fully repairs 95.5% of the single-bug tasks and 9.1% of the hardest tier. That tier roughly doubles repository size and inserts 23–60 bugs, despite being labeled “50 Bugs.” Each task gets one attempt, with Opus allowed up to 80 turns.
- Full repair requires at least 99% metric recovery, continued execution through the affected modules, and a genuine fix at every verified bug site. A judge model, a model grading another model’s output, checks those sites against recorded defects.
- Training Qwen3.5-9B on an easy-to-hard repair curriculum raises its SWE-bench Lite success from 24.3% to 33.0%, across 300 repository issues. Generating fresh machine-learning solutions shows mixed transfer, keeping the result specific to repair.
- Caveat: The faults are deliberately injected and full-repair scores partly depend on model grading, so these rates differ from directly measured production incident resolution. (arXiv 2611.25088 · full generation method, repair criteria, difficulty tiers and transfer results)
-
Model graders can teach assistants to refuse answers they know paper notes → — Checking refusals by question category can reveal damage that overall accuracy and refusal rates conceal.
- Researchers conduct 814 runs using reinforcement learning, training from task rewards. They teach Qwen2.5-7B and Llama-3.1-8B to answer short factual questions or abstain. Each run uses 750 training questions.
- A strict Qwen grader rejects eleven of twelve correct training answers about former-state capitals. The trained Llama model then refuses 98% of the 187 unseen former-state capitals it previously knew.
- In a separate controlled intervention, changing only twelve grading decisions moves refusal from 5% to 90%. This uses a grading threshold set to match the answer key’s acceptance rate, separating category-specific judgments from blanket strictness.
- Blocking “I don’t know” recovers correct answers on 94% of the refused capitals in the headline experiment. The failure is largely withholding available knowledge, rather than erasing it.
- The released code and run outputs support inspecting category-level rejection patterns before training and refusal patterns afterward. Trained model adapters are absent from the release.
- Caveat: The study concerns short factual answers under constrained training, and whether this mechanism transfers to open-ended assistant work remains untested. (arXiv 2611.25191 · full study, controlled grading interventions, limitations and reproduction repository)
Shorter instructions can come with a training bill
- An image can replace some lengthy instructions for small models paper notes → — Skill.png offers an inspectable experiment in shrinking task instructions, with substantial limits on training cost and reuse across models.
- Researchers optimize an image’s pixels through a frozen vision-language model, which accepts pictures and text. Users then attach the resulting image to a request while leaving the model’s learned weights unchanged.
- Across four models and four task sets, image instructions significantly beat the SkillOpt text-instruction optimizer in nine of sixteen comparisons. They use 64–256 image tokens, compared with text instructions averaging 937–1,963 tokens.
- The public implementation includes training and evaluation instructions. Optimizing one image takes about 12.2–19.9 GPU hours per task set on the study’s H100/H200 hardware.
- Gains vary sharply: standalone images fail to improve exact tool selection significantly and lower one model’s score by 9.82 percentage points. Joint image-and-text training can help where simply combining separately optimized instructions fails.
- An image optimized jointly for Qwen3.5-4B and Gemma-4-E4B-it improves both, but images fail to improve models outside their training group. The images resemble colored noise and lack text instructions’ human readability.
- Caveat: Main results use reasoning disabled, and the authors warn that some gains may reflect task-format shortcuts rather than broader ability. (arXiv 2611.26296 · full methods, task-specific results, transfer tests and public implementation)
All gathered items - what was cut and why (10)
- Inkling 2 Mini’s original release → DEDUP: yesterday Afternoon already covers downloads, architecture and runtime requirements. This edition keeps the distinct outside evaluation instead.
- DreamCoder’s launch → DEDUP: yesterday Morning covers the same workspace, pricing and provider-data boundary. More HN discussion supplies no checked release change.
- Signal’s mobile screenshot restrictions → DEDUP: the actual standalone treatment and yesterday Morning already cover this development.
- The developers-to-architects essay → DEDUP: the current standalone capture already treats the logging example and the disputed promotion framing.
- Language Models Can Rewrite Their Own Weights → DEDUP: existing canonical paper notes cover the same revised study. A higher-scoring HN thread adds no new result.
- METR’s expanded task-horizon study → DEDUP: yesterday Afternoon already covers the new judgment tasks and uncertain human-time estimates.
- ClawDOS → LOW_UTILITY: the source and constrained DOS implementation are real, but vintage-machine support is less applicable to this edition’s engineering choices. Its model runs through a hosted API, rather than on the old processor.
- Ubisoft’s Motherbrain subsidiary → LOW_UTILITY: the announcement describes intended AI gameplay and a corporate reorganization, without a developer SDK, reproducible method or released runtime in the report.
- Etsy sellers’ response to AI-training terms → UNVERIFIABLE: extraction was blocked twice, including an alternate request for the same article. The collected headline alone cannot establish the detailed policy scope or sellers’ evidence.
- The AI-linked stock sell-off → LOW_UTILITY: the report documents market movements, without a corresponding checked change to model access, pricing or deployment in the selected stories.