• AI-assisted math proofs differ sharply in what readers can check — New public proof files show why a readable argument, a formal proof and the evidence needed to reproduce a result are separate deliverables.
    • Terence Tao lists a human-readable Euler fluid-equation proof by Levent Alpöge, Tristan Buckmaster and Matei Coiculescu, with heavy AI assistance. Its Lean version uses software that checks formal mathematical arguments.
    • For the Navier–Stokes fluid-equation proof, Tao says the broad strategy is clear but a professionally written explanation still needs work. These new fluid results concern equations without an external stirring force. September’s coverage gives background on earlier lab work and its credit dispute.
    • The Navier–Stokes repository discloses a separate reproduction gap: its numerical certificate and rigorous checking computation are absent. It also says the latest patched formal package was not rebuilt before release.
    • Tao describes Alpöge’s Generalized Riemann Hypothesis claim, concerning prime-number patterns, as an enormous Lean-only argument generated with an internal model. He points to ongoing efforts to verify it and extract its strategy.
    • In a separate human-written proposal, Ravi Vakil’s update reports problems identified with assistance from different AI models. That concerns Alessandro Masullo’s proposal, distinct from Alpöge’s formal files.
    • Caveat: Public formal files and expert commentary do not establish completed independent verification of every claimed result. (HN 462 points · 191 comments · HN 2157 points · 1002 comments · Tao, Vakil and primary proof repositories)

An outside check on yesterday’s model release

  • Artificial Analysis tests Inkling 2 Mini’s accuracy and running costs — An outside comparison gives developers evidence beyond the launch claims, while separating hosted running costs from performance on their own machines.
    • Inkling scores 20 on the Intelligence Index, matching Qwen3.8 27B at its highest reasoning setting. This composite combines tests of work, coding, science and general knowledge, rather than measuring a percentage of tasks solved.
    • It averages about 152,000 output tokens per task versus Qwen’s 110,000, roughly 38% more. Tokens are the pieces into which models divide text. Task costs therefore need to include the full answer length.
    • Artificial Analysis reports $0.59 and 9.0 minutes per task, versus $3.53 and 38.2 minutes for Qwen. Inkling’s cost uses Thinking Machines’ first-party API pricing, including cached-input rates.
    • This comparison leaves cheaper alternatives: GPT-6 Luna at maximum reasoning scores 28 at $0.15 per task. The results support comparing quality and total task cost, rather than assuming a smaller model always costs least.
    • Yesterday Afternoon’s release coverage supplies download and runtime requirements. Today’s addition is the outside evaluation, without repeating those specifications.
    • Caveat: These hosted test results do not establish laptop speed or the cost of running the model yourself. (Techmeme · Artificial Analysis evaluation and published methodology)

Make repeated conversations cheaper to serve

  • Saving model state at conversation boundaries cuts repeated work paper notes → — FrontierReplay offers a concrete serving design for reusing earlier conversation processing without repeatedly interrupting the server to save it.
    • Hybrid models combine attention with a fixed-size running state. FrontierReplay reconstructs exact saved states at useful conversation boundaries after prompt processing, using a short record of internal inputs.
    • The implementation uses vLLM 0.30.0. Its main tests cover Qwen3.5-9B and Falcon-H1-7B across four workloads, on an A800 with 80 GB GPU memory and a 32 GiB cache budget.
    • In Falcon coding-session replays, median first-output wait falls from 1.1 seconds to 125 milliseconds. The rate of calls meeting both response targets rises 4.26-fold. Those targets allow 500 milliseconds to start and average 50 milliseconds per output token.
    • Qwen’s coding-session gains are smaller: first-output wait falls from 173 to 107 milliseconds. Its target-meeting call rate reaches 1.11 times the baseline’s, showing how benefits depend on the model and workload.
    • A larger-model document test reaches only 0.76 times the baseline’s target-meeting call rate because separate memory pools restrict capacity. Exact saved state therefore offers no guarantee of higher serving capacity.
    • Caveat: These controlled workload replays fix recorded replies, so the measurements exclude changes in what a live assistant chooses to do next. (arXiv 2611.26020 · full system design, experimental setup, results and adverse scaling case)

Test repairs and the graders behind them

  • Adding bugs exposes a gap between recovered scores and repairs paper notes → — BuggyMLE shows why checking a repaired application’s output score can miss faults that remain inside its code.

    • Researchers create working repositories for 22 machine-learning competitions, then inject faults at four difficulty levels. A benchmark is a fixed set of tasks for comparing models. An agent is a model running a multi-step task with tools.
    • Claude Opus 5.5 fully repairs 95.5% of the single-bug tasks and 9.1% of the hardest tier. That tier roughly doubles repository size and inserts 23–60 bugs, despite being labeled “50 Bugs.” Each task gets one attempt, with Opus allowed up to 80 turns.
    • Full repair requires at least 99% metric recovery, continued execution through the affected modules, and a genuine fix at every verified bug site. A judge model, a model grading another model’s output, checks those sites against recorded defects.
    • Training Qwen3.5-9B on an easy-to-hard repair curriculum raises its SWE-bench Lite success from 24.3% to 33.0%, across 300 repository issues. Generating fresh machine-learning solutions shows mixed transfer, keeping the result specific to repair.
    • Caveat: The faults are deliberately injected and full-repair scores partly depend on model grading, so these rates differ from directly measured production incident resolution. (arXiv 2611.25088 · full generation method, repair criteria, difficulty tiers and transfer results)
  • Model graders can teach assistants to refuse answers they know paper notes → — Checking refusals by question category can reveal damage that overall accuracy and refusal rates conceal.

    • Researchers conduct 814 runs using reinforcement learning, training from task rewards. They teach Qwen2.5-7B and Llama-3.1-8B to answer short factual questions or abstain. Each run uses 750 training questions.
    • A strict Qwen grader rejects eleven of twelve correct training answers about former-state capitals. The trained Llama model then refuses 98% of the 187 unseen former-state capitals it previously knew.
    • In a separate controlled intervention, changing only twelve grading decisions moves refusal from 5% to 90%. This uses a grading threshold set to match the answer key’s acceptance rate, separating category-specific judgments from blanket strictness.
    • Blocking “I don’t know” recovers correct answers on 94% of the refused capitals in the headline experiment. The failure is largely withholding available knowledge, rather than erasing it.
    • The released code and run outputs support inspecting category-level rejection patterns before training and refusal patterns afterward. Trained model adapters are absent from the release.
    • Caveat: The study concerns short factual answers under constrained training, and whether this mechanism transfers to open-ended assistant work remains untested. (arXiv 2611.25191 · full study, controlled grading interventions, limitations and reproduction repository)

Shorter instructions can come with a training bill

  • An image can replace some lengthy instructions for small models paper notes → — Skill.png offers an inspectable experiment in shrinking task instructions, with substantial limits on training cost and reuse across models.
    • Researchers optimize an image’s pixels through a frozen vision-language model, which accepts pictures and text. Users then attach the resulting image to a request while leaving the model’s learned weights unchanged.
    • Across four models and four task sets, image instructions significantly beat the SkillOpt text-instruction optimizer in nine of sixteen comparisons. They use 64–256 image tokens, compared with text instructions averaging 937–1,963 tokens.
    • The public implementation includes training and evaluation instructions. Optimizing one image takes about 12.2–19.9 GPU hours per task set on the study’s H100/H200 hardware.
    • Gains vary sharply: standalone images fail to improve exact tool selection significantly and lower one model’s score by 9.82 percentage points. Joint image-and-text training can help where simply combining separately optimized instructions fails.
    • An image optimized jointly for Qwen3.5-4B and Gemma-4-E4B-it improves both, but images fail to improve models outside their training group. The images resemble colored noise and lack text instructions’ human readability.
    • Caveat: Main results use reasoning disabled, and the authors warn that some gains may reflect task-format shortcuts rather than broader ability. (arXiv 2611.26296 · full methods, task-specific results, transfer tests and public implementation)
All gathered items - what was cut and why (10)