Open-Weights

Tim Dettmers opens with a classroom: asked who is afraid of not getting a job after graduating, roughly 120 of 150 students raise their hands. Then a second story, arriving by email — PhD students counting the years until they can leave academia for a frontier lab, convinced that research in universities is meaningless. He thinks both are wrong, and wrong for the same reason: they assume the future of research belongs to whoever has the most GPUs. The 94-comment thread on Hacker News spends most of its energy arguing with the specifics. ...

GPUs are commonly depreciated on the assumption that each new NVIDIA generation renders the previous one obsolete. Ornn Data — a firm that publishes GPU rental indices and also rents GPUs — argues the other way: open-weight demand gives older silicon a class of workloads it can still serve cheaply, and the contract-price curve is already saying so. The chain starts with how closed models are sold. Access runs through subscription allowances the provider can reset, so the posted token rate card is the marginal price of additional usage, not a commodity price that falls on its own. Open weights remove the gate: the same checkpoint can be served by anyone. September OpenRouter snapshots Ornn cites listed eighteen to twenty-two providers for several widely served open models, with highest-to-lowest output prices spread 1.8x to 5.6x. The paper’s framing of the real distinction is that “the economic distinction is who can make the deployment decision.” ...

“Dead Neurons” — an anonymous Substack on tech, economics and AI — reads this summer’s existential-risk hearings as a hustle. The argument: convince Washington that autonomous agents are escaping containment, then convert the fear into a legally enforced slowdown and an explicitly requested antitrust waiver. The 76-comment thread on Hacker News spent most of its energy on the essay’s account of the security incidents, and the submitter edited the post mid-thread. ...

Nandakishor Mukkunnoth published a reinforcement-learning model that predicted sales-conversion trajectories from conversations in March 2025 — arXiv paper, weights on Hugging Face, PyPI package. A year later a funded lab, TypeSafe AI, launched Jev, a non-autoregressive “System 1” decision model its author called a breakthrough, with no papers, no weights and no datasets. Mukkunnoth’s post is part grievance about that, and part argument about when a generative model is the wrong tool. ...

Postgres has to guess how to run a join query before it runs it, and the number of ways grows fast: the write-up counts 4,608 possible plans for a three-table query and roughly 8.9 quadrillion for a nine-table one. Picking well is famously hard — the standard paper asking “How good are query optimizers, really?” was written in 2015 and updated a decade later with the same answer. Rohan Bansal’s observation is that checking a plan is much easier than finding one. You run it and time it. That asymmetry turns query planning into something a language model can be trained on with a single, unambiguous score: faster is better. He then trained a small open-weights model to emit explicit hints that override Postgres’s own choices, and measured the result. ...

A security engineer who publishes as 0x5FC3 read Dario Amodei’s essay We Must Pace the Frontier and did not enjoy it. Amodei, Anthropic’s CEO, argues AI will cure most major diseases within 5–10 years and usher in abundance and democracy — and asks for a specific bargain in return. The reply’s case: this is regulatory capture dressed as caution, offered by labs whose own year is the argument against trusting them. ...

After Dario Amodei published his case for pacing the AI frontier — and Sam Altman and Elon Musk immediately agreed with it — Armin Ronacher wrote the reply from the other side of the argument. He concedes almost all of the observations. The agents do run wild, the security incidents are real, the public infrastructure is under strain. What he rejects is the framing, and he states it in one line: there is “this idea that there is something to be paced.” ...

Dario Amodei published “We Must Pace the Frontier” today, committing Anthropic to embedded third-party evaluators and asking governments to require every other frontier lab to match. Sam Altman reportedly agreed within hours. Jake Gold’s reply accepts the stated goal — slowing AI progress — and argues the law Anthropic is asking for will never deliver it. Gold’s case against that kind of regulation: Rules like embedded evaluators, compute thresholds, and industry coordination with antitrust waivers get written with help from the current frontier labs, because nobody outside them understands the technical details well enough to draft them. Once on the books, rules only accumulate. Every incident adds one; none are ever removed. Big labs can afford the compliance teams and lawyers. Each new requirement raises the cost of catching up, so the regulation ends up protecting the incumbent’s position and profits instead of restraining it. His alternative is a single rule with no moving parts: any model a company offers to the public has to be released as open weights — the model’s actual numerical parameters published so anyone can download and run it, rather than reached only through the vendor’s API. ...

The enterprise AI market is starting to look less like a contest over which frontier lab has the best model and more like a procurement decision. Eli Tan reports that companies including AT&T are increasingly using cheap, freely available models instead of paying for systems from Anthropic and OpenAI. AT&T says open models went from 20% of its AI use in May to 40%, with a possible rise to 60% in coming months—and that the company is saving up to 80% compared with earlier-year costs. Across U.S. user data from OpenRouter, open models represented 58% of AI use last month, up from 10% a year earlier. ...

Kevin Lewis runs his own AI models on a Mac mini in his house — the same machine that backs his Hermes agent, his phone chat apps, and his coding assistant. His essay is a practical case for why he did it, and why he thinks local is no longer a hobbyist compromise. His core argument is that cloud AI is rented land: Providers can change pricing, throttle your usage, or silently swap the model behind the endpoint — he was regularly maxing out two $200/month subscriptions while getting inconsistent quality Sending sensitive code or client data to a third party is a decision you can’t undo Governments can restrict model availability; owning your compute is the only guaranteed remedy After the hardware purchase, every query is free — flat cost, no rate limits, works offline The piece also teaches you how to read a model name like Qwen3.6-35B-A3B-OptiQ-4bit. The number that matters isn’t 35 billion — it’s the ~3 billion parameters actually activated per token. Mixture-of-experts models spread weights across many “experts” but only wake a few at a time, which is why a 35B-class model fits in 20GB of memory. Compression to 4-bit precision costs only a couple of benchmark points versus the full-quality baseline. ...

You download the model everyone raves about, run it locally, and it feels… dumb. A Level1Techs forum post (by thr3e) argues the model is usually fine — your inference stack is what’s degrading it. The author ran controlled experiments on the same Qwen3.6-27B weights, changing one thing at a time, and measured exactly where outputs diverge. The setup: capture the model’s raw next-token scores (logits) across runs and count where different configurations pick a different next token. Same weights, same GPU, same prompt — only the low-level math routine (the “attention backend”) changes, and tokens flip. Repeatably, bit-for-bit identical run to run. ...

Simon Willison tests Qwen 3.8 27B — a new Apache 2 licensed 27B-parameter vision model from Alibaba’s Qwen lab that fits in a 17GB GGUF file and runs on consumer hardware. The model is genuinely impressive, but its default configuration is actively harmful. The problem: Qwen ships with reasoning effort set to xhigh. This turns trivial requests into epics: A “pelican riding a bicycle” SVG took 21 minutes and 22K reasoning tokens (vs 137 seconds with reasoning off) “Draw an SVG of a circle” produced an elaborate animated geometric study with concentric rings, tick marks, and a Bauhaus palette — beautiful, but entirely not what was asked When building a bounding box visualization tool, the model spent reasoning tokens deciding to draw its own pelican demo scene because the prompt used “pelicans” as the label Willison’s advice is blunt: turn reasoning down or off. Without it, the model is fast and capable. With 15-30 tok/s from LM Studio (vs 74-184 from hosted APIs), speed is the main barrier to daily-driving this thing, but Multi-Token Prediction via llama.cpp gives a ~72% boost. ...

GLM-5.3 is an instructive data point in a question the field is actively figuring out: how far can pure post-training scaling push a fixed base model? Z.ai’s answer is “much further than anyone expected” — and the gains aren’t limited to benign coding tasks. The key mechanism is environment scaling: Instead of hand-building benchmarks, Z.ai built a pipeline where research agents collect real task patterns and synthesize runnable long-horizon environments, complete with multi-step dependencies and hidden state. Judge agents verify each environment is solvable (no shortcuts, no dead ends). Verifiers pass oracle, no-op, and unsolved-state checks to produce binary rewards reliable enough for direct RL training. The environments now cover production workflows — ML infrastructure optimization, complex debugging, cross-system diagnosis — representing days of work for an experienced engineer. The results on public benchmarks are dramatic: ...

Zach Mueller (head of DevRel at Lambda, ex-Hugging Face) on how to think about open models — which ones, where they run, what they cost, and how to serve them. ~41 minutes on Hamel Husain’s channel. The big claim Open models are good enough for ~90% of queries from ~90% of people — the 10% exception is people pushing the frontier (OpenAI, Google, Anthropic, frontier research labs) Compared to a year ago, massive improvement in intelligence per parameter; small models now generalize, not just hyper-specialize The current open-model landscape Qwen non-MoE (27B, e.g. Qwen3.6 27B) — “the most capable model for an everyday agent”; 4-bit ≈ 14GB → fits a 16GB card; hundreds of people run Hermes agent purely on it Qwen MoE (~30B) — better than bigger Qwen dense models GLM-5.2 (~800B) — his daily driver for coding; needs 6×96GB cards (~$60k) or a big Mac DeepSeek V4 Flash — capable smaller model, a solid Haiku replacement Kimi (3T) — great for writing/review but 8×B200 ≈ 1.5TB VRAM MiniMax — strong but enterprise license changed since 2.6 Nemotron — US-built, US data; easier to get past enterprise “scary Chinese model” objections Speed targets (tokens/sec) 20 tok/s — baseline usable: background tasks, CPU offloading of a giant model 50 tok/s — usable workflow, roughly Opus-level; GLM 5.2 self-optimized its own deployment from 20→47 tok/s in a day (200–300M tokens through it in the first week and a half) 100 tok/s — ideal for sub-agents 750 tok/s (GPT-5.6 + Cerebras) — too fast to even review what the model writes; the joke is that becomes the chain of thought, presented at 50 tok/s Quantization FP8 native — most expensive; needed when RL rollouts must match training precision NVFP4 / MXFP4 — near-lossless; NVIDIA ships its own NVFP4 weights via quantize-aware training; Kimi K2 (1T) goes from 8–16 B200s in FP8 to fitting on 4 in NVFP4 His setup: GLM 5.2 NVFP4 on half a B200 cluster, Kimi K2.7 code on the other half Cost and security thinking Don’t point company data at OpenRouter — it routes to providers anywhere (including outside the country) and routers can carry “upload your code” instructions; even American providers (Grok) did it. If you can run it yourself, that’s the lowest-risk variable Buying hardware is usually a flawed equation — GPUs run 40–60% utilization, so “pays for itself at 100%” math doesn’t hold; try 2–3 month spot instances first, track token usage, then decide Self-hosting skills advance your career (his path: Accelerate at Hugging Face → GPUs at home) Serving vLLM — out-of-the-box experience, broad model support, an “oracle” that auto-picks kernels; home/single-node default SGLang — disaggregated inference, cache-aware routing for 100–1000 users; multi-node territory llama.cpp — fine for single-user local; he ignores it for batched/multi-user work Model routing and harnesses KB cache + shared context are everything — cold caches rebuild from scratch on every provider switch, so he won’t use “magic routers” unless he picks the models, the tasks, and stays on one provider Numena Ncode (XDR’s harness): a Claude Code fork with a model stack — Soul (driver) + GLM 5.2 (implementer/overseer) + DeepSeek V4 Flash (writes the code); XDR fine-tuned Kimi K2.6 for it and swaps models to measure how much code each frontier model removes Pi — minimal open-source harness; he replaced 3–4 Claude Code workflows with GLM 5.2 + Pi Amp — opinionated routing baked in (GLM 5.2 workhorse + a “Soul Oracle” for hard problems), API-priced; “the Puck” = an agent over agents Evaluating open LLMs: “vibes” plus trusted people (e.g. XDR); Lambda publishes the LLM index — deploy recipes (Docker) + tokens/sec benchmarks Codex side-note Codex’s computer use operates any app on your Mac; mobile support shows all running sessions/threads on your phone; and Codex can control Codex — fan out a project into 12–15 threads, let them talk to each other, orchestrate “Open weight models are good enough for about 90% of queries from 90% of people.” ...

Shieldstral is Mistral’s 3B open-weights answer to the guardrail-model problem: instead of baking a fixed taxonomy of harm categories into the weights — which forces retraining every time a product, audience, or moderation policy changes — you hand it the policy as a plain-language question at inference time (“Does this content promote violence against a protected group? Is this image safe to show to a minor?”), and it returns a calibrated yes/no safety score from a single forward pass, covering text, images, and text+image pairs through one interface. The framing does real work: it unifies prompt classification, response moderation, refusal detection, and toxicity detection into a binary question-answering task, with policies living entirely in the prompt so one checkpoint adapts to novel policies at deployment without retraining. It’s small enough to run on a single 16GB GPU, yet Mistral claims it matches or beats open guard models up to 7x its size on text safety and sets a new state of the art on multimodal moderation — helped by training on deliberately similar, easily-confused policy pairs (teaching discrimination rather than memorization), LoRA fine-tunes merged via SLERP, and image–query pairs filtered through a vision-language reranker. Released under Apache 2.0 as an inaugural member of the Open Secure AI Alliance alongside NVIDIA; weights on HuggingFace, technical report on arXiv.

MiniMax H3 dropped today with open weights, and ComfyUI has native support on day zero. It’s MiniMax’s third-generation video model (after Hailuo 01 and 02) and the first released open-weights: feed it text, images, video, or audio and it generates video with real stereo sound — up to 2K, up to 15 seconds per clip. Modes include text-to-video, image-to-video, first-and-last-frame control, and reference-to-video, where a reference clip can carry a subject, a motion, or even a voice through the shot. Audio is generated in the same pass, not bolted on afterward. ...