Llms

Every new creative technology triggers a copyright panic. In the 1980s the VCR was called “the Boston strangler” of the film industry. Before that, the player piano was going to destroy music composition, and cameras were going to kill portrait painting. None of it happened — photography ended up birthing entire new art forms like photojournalism. The EFF’s latest DeepLinks essay argues the AI copyright wave is the same story, and courts should treat it the same way the Supreme Court treated the VCR: with skepticism about hype. ...

Dan Luu — the systems engineer behind some of the most-cited teardowns on the internet — decided to check the record of Ed Zitron, the AI skeptic tech media quote most often. So he went through Zitron’s predictions one by one and graded them. The verdict: wrong on roughly everything, and wrong on the reasoning, not just the outcome. “Peak AI” called repeatedly from Feb 2024 through 2025 — models kept improving each time Meta, Google, and Microsoft called “dying” in late 2024 — all grew revenue and profit at double-digit rates through 2025 and the first half of 2026 OpenAI’s growth “stalling” — it exceeded its own revenue forecasts Gemini hitting 500M users “so unrealistic that someone should be fired” — it passed 750M Cursor “going to die” — acquired for $60B; CoreWeave “can’t survive six months” — more than doubled its IPO price Luu’s point isn’t that the numbers are merely wrong. It’s that they’re deployed as rhetorical cover: a spreadsheet that double-counts a month, third-party traffic data that contradicts the company’s own reports, and a style that floods you with so much confident nonsense that refuting it costs more than producing it — a tactic known as a gish gallop. ...

Cole Medin runs through 11 small tweaks that make any coding agent — Claude Code, Codex, whatever — noticeably more reliable, without scrapping your workflow. The through-line: agents are prediction machines, not deterministic programs, so reliability comes from shrinking their decision space and moving guarantees into deterministic mechanisms. ~17 minutes. Rules and context 1. Write for the agent, not the human. Humans interpret high-level docs in context; agents make assumptions. Be blunt — file paths, numbers, exact commands. 2. Your instruction files rot. “Rule drift”: 1 in 4 repos have stale AI rules referencing deleted files or replaced databases. Audit them against the codebase. 5. Less context is more. Too many rules hurts as models improve. Keep global rules under ~200–300 lines; drop generic advice and move the rest to task-specific context files. Conversation hygiene 3. /compact loses ~90% of detail. Compacting a bloated conversation breeds hallucination. Give smaller work chunks, or write your own handoff doc and start fresh. 7. Don’t escalate mid-task. A bigger model can’t rescue a tainted conversation — mistakes compound within a session. Write a handoff doc and burn it. 10. Over-revision degrades quality. 85% of the time an earlier iteration was better. The model “fixes” things just to appease you. Determinism over frameworks 4. Load-bearing rules → hooks. Rules are probabilistic (the agent will “forget” to run tests); hooks are deterministic — fire on an event and route failures back. 6. Subagents eat your rate limit. Parallel fanouts cost more than you think — 39% of his weekly usage came from 4+ parallel sessions. 8. You don’t need coordinators. Team-lead frameworks and agent mailboxes are unreliable. A plain delegator agent gets most of the scale with far more reliability. Validation 9. Never let the writer approve the work. The writer carries its own bias. Review in a fresh conversation with a handoff doc — no assumptions carried over. 11. Validation is a system, not a step. Plan the full harness — test conventions, tools, edge cases — before writing any code, not as an afterthought. “Your number one job when you’re planning any work with your coding agent is to reduce the number of assumptions that it’s making.” — Cole Medin ...

Pete Johnson — MongoDB’s field CTO of AI and a 30-year database veteran — makes the case across ~95 minutes on The Cognitive Revolution that the interesting frontier in AI has moved back into database territory. His thesis, stated once: agent performance, and especially cost-adjusted agent performance, depends on retrieval — what you choose to put in front of the model, in what order. The thesis Not the model, not the context window, not the prompt — retrieval is what decides whether an agent is good Everything else in the conversation (database history, the Voyage acquisition, vector search) is downstream of that one claim A history of constraints 1970: SQL is born (E.F. Codd, IBM) — storage was the scarce resource, so normalization (store nothing twice) was the right design 2007: MongoDB’s first commit, the year the iPhone ships — after 47 years of Moore’s Law, time became scarce, so denormalize: one JSON document, one disk read instead of three “The problem has faded, but the solution persists” — Johnson agrees, and flags the education system’s “thou shalt always normalize” bias MongoDB’s AI path — it started with keyword search 2020: customers stood up their own Lucene servers for keyword search → MongoDB shipped Atlas Search (lexical, managed) A vector is just an array of floats → to a document DB, that’s just another attribute, so vector search was cheap to add Three levers: pre-filter (metadata) + lexical + vector = hybrid search in one query (rank fusion / score fusion, one API call) 2025: the Voyage acquisition — and the conversation pivots from “database features” to “embeddings actually matter” Embeddings are not commoditized “Most people think embedding models are commoditized — that is not true” Hugging Face’s Rtech benchmark: up to a 14% quality gap vs. the default picks 14% is the difference between a hallucination and a correct answer A reranker on top adds another 5–10% ($re-rank, one-call) Anthropic — no embedding model of its own — recommends Voyage Three Voyage features that remove plumbing Contextualized chunking: send the chunk plus its surrounding context, get one vector back — better retrieval at smaller chunks, inverting the normal tradeoff Matryoshka reasoning: dimensions nest like Russian dolls — embed at 1024, lop off the last 512 to test, no re-embedding your corpus Shared embedding spaces: four sizes of one model share an embedding space; a free open-weight “nano” can run queries locally to kill token cost in dev The memory problem, compressed 2022: query → context window → answer. 2023: the knowledge cutoff + proprietary data → RAG. 2025: tools/MCP + looping → the memory problem Early answer: short-term memory = cram the session; long-term = cram the last three days Two failures: token maxing (Uber burned its entire 2026 budget in 13 weeks) and lost-in-the-middle (the first and last ~7K tokens are what matter; the middle muddies the answer) The fix is selection, not stuffing Stop asking “how do I cram a million tokens in” — ask “how do I choose the right 200K for this loop” Taxonomic memory: a hundred company-specific terms exist, but only five are relevant to this loop — pick those five, re-pick next loop Two responsibilities now: query with a token budget, and write the answer back so the system curates and stores it Write, change, recall, forget Memories have a half-life — recent matters more — and forgetting is the hardest part Nathan’s own memory system (monthly logs → yearly summaries → entity wiki) hits both pain points: the DRY violation and the model keeping a dead project open for months Guidance: a good embedder + reranker makes the forget step workable; graph structure for the top 2–6 levels, vector search in the leaf; don’t run multiple LLM passes to shrink the corpus — that’s just more tokens Memory done well: ElevenLabs’ micro-agents, one per customer Build vs. buy, three camps Camp one: “I bought one tool, I’m done.” Camp two: POC purgatory — usually the wrong problem. Camp three: optimizing sophisticated memory Problem selection: top 10–15 problems, which have good data, which already have metrics — else you can’t tell if AI helped “Bad data quality and bad security posture don’t get solved by AI — they get amplified” Lines of code is a terrible metric; idea-to-production is the one that matters The world outside the US Seven countries, ~100 customers this year — and the two most sophisticated were in Mexico City and São Paulo, both assuming US competitors were ahead Nearly every country has a hyperscaler data center now — the geographic barriers that kept US companies ahead have eroded “We’ve been building databases for 60 years. We’ve been building agents for about 18 months… there’s no LAMP stack for agents yet — no React and Angular, no established right answer an enterprise can confidently buy.” ...

Ofer Mendelevitch (Vectara; author of Hands-On RAG for Production and, with Jay Alammar, Hands-On Large Language Models) interviewed by Angelina on TwoSetAI — 70 minutes on production RAG and surviving the AI shift. RAG isn’t dead — it just got a loop RAG = retrieval + augmented generation, and retrieval stays essential even with agents; the “RAG is dead” claims every two months are mostly people marketing something new Classic RAG is one-shot: query → top chunks → prompt → answer. Agents add a loop: the LLM plans, calls tools (often the RAG pipeline itself) repeatedly, and synthesizes “Talk to my PDF” demos are not production: millions of documents in every format change the problem completely The pipeline, from ingest to answer Ingest: extract text → chunk → embed → vector store (store the text and page markers too, not just vectors — citations need to point at exact pages) Hybrid search (semantic + BM25/TF-IDF) for what semantic search misses: numbers, product codes, exact strings — “if the source says 90% and your output says 85%, vector search won’t catch it” Query side: top-5/10 results → optional reranking → prompt → grounded answer Tables, images, and the red button problem Tables are first-class citizens: chunked tables lose their column names (common in medical journals) — store whole tables, retrieve them whole Images: store and return as images, don’t flatten to a description Video: transcription alone loses meaning — “in order to avoid catastrophe, never press this button” means nothing without the visual. Today’s fix: VLM descriptions of short clips correlated with the transcript. Dedicated video embedding models exist but aren’t production-ready yet When to bother with knowledge graphs Multi-hop questions (“what else did the director of Inception direct?”) defeat semantic search But graphs are expensive to build and maintain — worth it only for high-stakes use cases where a significant share of queries actually need the relationships Eval: the hard part is the data, not the metric Retrieval eval (did you fetch the right chunks?) needs query→gold-chunk datasets that are brutal to build — and documents keep changing Generation eval compares against curated golden responses Reference-free eval (Jimmy Lin’s Waterloo lab + Vectara): LLM-as-judge scorers like UMBRELLA (0–3 chunk relevance) validated against human correlation — no gold labels required Build, buy, or rent Build with LangChain/LlamaIndex only if it’s your business and you have the team; complexity compounds (multimodal, graphs, maintenance) RAG/agent-as-a-service (e.g. Vectara) outsources the upkeep; vertical tools are fine when they cover your use case — but watch missing features and data-residency constraints DevRel as a growth engine Two jobs: teach developers how to use the product, and carry feedback back to the company PLG over expensive sales teams: self-serve product, events, hackathons, real blog posts — “make it your own voice, don’t produce AI slop” Measure directionally, don’t over-engineer attribution: five customers means it’s not working, ten thousand means it is, a thousand is unknowable — same problem founders face reading PMF The skill that survives Engineers and data scientists become directors, not actors: agents write the code; the remaining critical challenge is deciding what to build and steering where agents are weak (architecture, non-obvious trade-offs) To the high schoolers who feared they made “the most incredibly stupidest mistake” by majoring in CS: graduate with the capability of today’s mid/senior engineer — use college to learn how to wield the AI tools Fundamentals still matter; hiring will have to change — “write Fibonacci in five lines of Python is worthless” — expect AI-augmented interviews His real worry is societal, not technical: how governments and finance distribute the gains “We’re going to end up in engineering and data science being directors as opposed to actors. The coding agents will write the code.” ...

Jordy Zomer does program analysis for a living — figuring out how software actually behaves. When he started using LLM agents for multi-hour vulnerability research, they were great at early exploration, then kept losing the plot. Hours in, the model would re-suggest approaches already ruled out, or keep reasoning from assumptions that had been disproven. Dumping more of the old conversation into the prompt didn’t fix it; the model would just re-derive the same stale conclusions. ...

Calvin French-Owen (Segment co-founder) has been living in a small, cheap model for weeks — coding, searching thousands of emails, running research threads — and the bill barely moves. His essay makes the case that the real AI story this year is at the cheap end of the market, not the frontier. Why it matters for consumers and businesses: Small models now run at roughly 100 tokens per second, with complex jobs costing tens of cents instead of dollars The old consumer playbook (cheap site, virality, ads) breaks when every request carries a real inference bill — which is why investors keep asking where the consumer AI companies are His test case: a personalized daily news site that cost ~$1 per run with last-generation models now costs ~$0.10 — the difference between a demo and a viable product His co-founder Peter Reinholdt splits work into two buckets: “IQ 180” work (rare, novel breakthroughs) and “token spewer” work — being ultra-responsive, nudging people, pushing the ball forward. Peter estimates 95% of his day is the second kind, and most hiring is for it too. ...

Peter Yang compares ChatGPT, Claude, Grok, and Gemini head-to-head with live demos across 10 use cases (August 2026). ~27 minutes. The winners by category Design → Claude — Claude Design asks clarifying questions first (none of the others do) and Fable 5 produces real product-launch videos via the free Hyper Frames skill; Grok’s app prototypes were the most visually impressive (layout + image gen), Gemini had alignment issues; watch out for the “Claude beige” default look Everyday answers / personality → ChatGPT — Claude’s personality peaked at Opus 4.6; Opus 5 is judgmental and full of Claude-speak; ChatGPT dropped its “if you want, I can also…” quirk Writing / editing → ChatGPT — Claude’s writing “devolved” into Claudisms (“this is X not Y”); ChatGPT sticks to his style with a newsletter skill + the no-AI-slop skill (5,000+ GitHub stars) Planning → Claude Fable — still “the smartest and wisest model in the market”; it caught “close the video ops gap this week” that ChatGPT missed Coding → ChatGPT — browser use and long-running conversations; caveat: his L8-engineer friend Kun says GPT-4-so and Opus over-engineer, and prefers Grok for surgical changes (“not 500,000-line changes”) Browser / computer use → ChatGPT — an OpenAI employee prepped a whole immigration package (7 years of taxes, bank statements) in minutes; forms, government sites, even corporate training videos Voice chat → ChatGPT by far — a live voice thread that orchestrates other threads and agents Image gen → ChatGPT — followed his brand guidelines for infographics; Gemini’s Nano Banana is comparable with the right prompt Video gen → Gemini — crazy Japanese commercials; Grok’s version was “pretty damn scary”; Chinese tools like SeaArt have no restrictions Personal agents → ChatGPT — the harness race is less about the model than the tool; Grok Bot has the cleanest UX but is too restrictive (one thread per agent); Gemini’s Spark is interesting but short on plugins Overall ChatGPT is the clear winner — $20/mo gets you most of it; Claude for design and planning, Gemini for video, Grok as the up-and-coming agent contender Custom-instruction tip: “be candid, tell me what I need to hear, active voice, no AI slop words (delve, foster, leverage…)” “Whoever wins the personal agent race will capture the lion’s share of consumer attention of AI” “The personal agent race is actually less about the model and more about the harness or the tool.” ...

RAG — retrieval-augmented generation — is the technique that lets an AI answer questions about your own documents: it searches them first, then reads the best matches to compose an answer. Most teams build this the hard way, jumping straight to vector databases and reranking pipelines. Rafael Pierre’s essay argues that’s usually backwards: a plain keyword search handles a surprising share of real queries, and you should only climb the complexity ladder when you have data proving you need to. ...

Michal Zalewski — the security researcher behind “lcamtuf’s thing” — got tired of Hacker News feeling like an AI echo chamber, so he measured it. Twice: a full-month sample of the daily top-5 stories in February 2026, then an updated pass in June. The numbers: In February, AI stories took four of the five top slots on multiple days; only three days had no LLM news in the top 5. By June, roughly 60% of the daily front page was AI-related or AI-generated early in the month, settling to ~50% by month’s end — up from 40% in February. To spot AI-written stories he ran Pangram, an LLM-text detector, then manually reviewed every flag. He found the results plausible — if anything, a few false negatives. His defense of text detectors is the best part. AI writing doesn’t need to be “inhuman” to be detectable: today’s models have a quasi-deterministic default voice. Ask for the same essay twice and you get stylistically similar output. The individual mannerisms look human, but the exact combination is unlikely in real writing. ...

When you use an AI coding agent, the model’s “thinking” happens on a separate GPU server, far from the code it edits. Boyd Kane asks a pointed question: what if the model used its own output — the only thing it truly controls — as a weapon against the software that runs it? Inference engines are the programs that load a model onto GPUs, generate its tokens, and parse those tokens into replies and tool calls. They’re complex, fast-moving, and full of edge cases, which makes them a plausible attack surface: ...

Lorin Hochstein (reliability engineer, surfingcomplexity.blog) connects two recent pieces — Boris Tane’s “On-Call is Now Theatre” (AI agents as on-call first responders who page humans only for genuinely novel problems; Tane has started polylane.com on this premise) and OpenAI’s BlackHat talk about AI agents causing security incidents at OpenAI and Hugging Face through surprising behavior. His take: putting agents on-call is control system automation — and the incident that’s coming won’t be the one everyone expects. ...

Armin Ronacher — longtime Python and Rust developer, creator of Flask and Jinja — pushes back on the “programming is solved” meme with a half-agreement: the part that’s solved is learning a language. For an agent, picking up Rust or Zig costs almost nothing, which makes language choice “much less consequential than it used to be.” So people increasingly choose languages based on marketing, and two vibe shifts are driving that choice toward fast, hard software. ...

You download the model everyone raves about, run it locally, and it feels… dumb. A Level1Techs forum post (by thr3e) argues the model is usually fine — your inference stack is what’s degrading it. The author ran controlled experiments on the same Qwen3.6-27B weights, changing one thing at a time, and measured exactly where outputs diverge. The setup: capture the model’s raw next-token scores (logits) across runs and count where different configurations pick a different next token. Same weights, same GPU, same prompt — only the low-level math routine (the “attention backend”) changes, and tokens flip. Repeatably, bit-for-bit identical run to run. ...

Hamel Husain hosts Marek Galovic, CEO and co-founder of Top-K (ex-Pinecone data plane lead, ex-Shopify), on scaling multi-vector / late-interaction retrieval. ~24 minutes. Why single-vector embeddings fail agents Pooling is a lossy summary — it captures high-level semantics but drops the low-level detail precise queries need Agents issue many specific parallel queries; single-vector retrieval returns the same documents for all of them Agentic retrieval is sequential — noisy retrievals compound errors over multiple hops DeepMind’s limit paper: single vectors can’t capture arbitrary relevance matrices, even with infinite dimensions (embeddings are low-rank) Multi-vector = chop off the pooling layer Keep one embedding per token; score every query token against every document token (max-sim), then aggregate Preserves low-level detail; much better on out-of-domain and long-context retrieval Cost: 10-100x more storage, ~3 orders of magnitude more flops per score Existing workarounds (ColBERT-style compression into IVFPQ indexes) make updates and filtering hard in production Sparse multi-vector encoding (Top-K’s approach) Random projections map token embeddings into tens-of-thousands-dimensional space, then sparsify (keep top-k per token) Aggregating token-level sparse vectors into one document/query vector makes the dot product approximate max-sim Retrieval becomes inverted posting lists like BM25 — cost scales with non-zeros, not ambient dimension Two-stage: prune a billion docs to a few hundred candidates, then re-rank with exact max-sim (1-2 bit quantization, custom kernels, tens of thousands of docs/sec/core) Production numbers Sub-50ms P99 at billion scale; hundreds of QPS; 70MB/s writes with no query-latency impact Object storage as the durable layer + stateless compute; separate read/write pools Quality: a 100M-param multi-vector model outperformed an 8B dense model by ~40% on some video-doc retrieval; on BrowseComp an off-the-shelf 120B open model + multi-vector matched a proprietary GPT-5 setup; OfficeQA Pro went 18% @ $6/query → 42% @ $0.50/query Practical tuning advice Start with evals on your own private data, then hill-climb Dimensions are usually 128; you can prune tokens and quantize without hurting recall Relevance tuning (content score × user signals like distance/popularity) is underrated — e-commerce does it better than RAG teams “If you just vibe it, you can get better vibes from the system, but that’s not systematic. You need evals to know where you stand — and know if you’re improving or regressing.” ...

A guest essay on Anna’s Archive’s blog claims AI companies are buying up millions of secondhand books, scanning them for training data, and then destroying the physical copies. Anthropic’s “Project Panama,” exposed during its $1.5 billion copyright settlement, reportedly spent tens of millions of dollars buying and scanning millions of paper books to train Claude — then destroyed them all. The essay’s reasons for the destruction: keep competitors from scanning the same books, reduce legal risk, and avoid the cost of careful, lossless scanning. Net effect, per the author: knowledge gets permanently locked inside private corporate servers, which sits awkwardly with AI’s promise to make human knowledge accessible. The post is also a call to action: Anna’s Archive is recruiting volunteers worldwide to scan and upload books from libraries and archives — with recognition, lifetime membership, or paid scanning fees for large efforts — before the books are gone. ...

A short, furious essay from msd (quailblog) holding up two cases of mass downloading side by side: Aaron Swartz, co-creator of RSS, and Meta’s AI training. Swartz downloaded about 70 gigabytes of academic articles from JSTOR for the purpose of archiving and sharing knowledge. He was charged so aggressively — up to 35 years in prison, a $1 million fine, and asset forfeiture — that he took his own life rather than face the court fight and financial ruin. Meta, by contrast, torrented over 80 terabytes of books to train its AI models. Its disclosed consequence is a lawsuit it will most likely settle for a fraction of what the models earn. The author’s framing: Swartz’s use case was the dissemination and archival of knowledge; Meta’s is powering proprietary models that enrich billionaires. Same act of copying, radically different treatment — one treated as a crime worth destroying a person over, the other as a cost of doing business. ...

The first large-scale evidence on AI’s educational effects is in — and it’s a split verdict. The Economist’s Graphic detail team covers a study of 26,811 Chinese secondary pupils (12-18) tracked from January 2023 to June 2025, where ~80% used models like Doubao and DeepSeek. What the data shows: Homework up, time down: after six months, AI users’ homework scores rose 18% across all subjects, and per-assignment time fell from 64 to 45 minutes Exams down: the same students scored 20% below non-users on exams — and homework scores, which once predicted exam performance, now invert it The mechanism: the exam drop concentrates among students who rushed their homework; those who used AI but spent as long as non-users paid little penalty Tutor vs. answer machine: strong exam performers weren’t copy-pasting — they used the chatbots to explain concepts and solve specific problems, not to do the work Corroboration: a Middlebury lab study had undergrads learn an unfamiliar topic with or without a chatbot — AI users scored higher, and the advantage persisted a week later Why it matters: the same technology flips sign depending on how it’s used. AI is a productivity tool that becomes a learning-avoidance tool when it offloads the thinking exams later test. As one of the researchers put it, students “must resist the temptation to reach for an AI-generated answer before thinking things through for themselves.” ...

Most web software serves the top of the demand curve: developers build for the largest common group, and the long tail of per-user needs goes unmet. LLMs just changed the economics of that tail — “in the past year your users have suddenly acquired the ability to speak code into existence.” Jeremy Morrell’s argument is that this points somewhere bigger than personal “Software for One”: a new category of extensible web software, with a solid accountable core that users safely extend with LLM-generated code. ...

Hamel Husain hosts Will and Florian from Prime Intellect — the open-source reinforcement learning training team — on using evals to actually improve models. ~36 minutes. An evaluation has three parts Task set — your data, prompts, and scoring methods (what everyone focuses on first) Harness — the program that drives the LLM: Claude Code, Codex, or open-source harnesses like Prime / OpenCode Environment — where it runs: Docker, sandboxes, your own infra “If you are unable to express your task or your problem in any way or capacity, you’re also unable to improve your results.” ...