Ai-Safety

Political theorist Gregory Conti (Princeton, writing in Compact) makes the strongest recent statement of the “AI is not the steam engine” case from the conservative side. His central move: AI opposition is misdirected because it targets future risks when generative AI is already producing moral and cultural harm. Anthropomorphic AI — models that mimic personality, emotion, and thought — is unsettling human psychology and the social fabric right now, so opposition should target what AI is, not only what it may become. The sui generis argument is the essay’s sharpest contribution: past innovations substituted for material processes; AI substitutes for language and cognition themselves — the things that constitute human distinctiveness — so the Luddite analogy is a category error. From there he prosecutes the case across four fronts: capitalism will be destroyed by its own success (quoting Marx’s prediction that production based on exchange value breaks down once machines out-produce labor, and noting Dario Amodei’s “Machines of Loving Grace” is fully automated luxury communism — the anti-communists may prove Marx right); individualism dies as AI becomes a homogenizer whose answers are statistical averages of human speech (Tocqueville’s soft despotism, Mill’s warning in On Liberty); democracy fails once citizens have no economic or military value, becoming subjects rather than rights-bearers; and the written word loses its human provenance — his grandmother’s-letters thought experiment: if she’d had Gemini, the access to the real person is denied forever. The essay also lands a sharp critique of AI-booster “productivity”: reading fifty papers in a month is really not reading fifty papers — you emerge with a facsimile minus the understanding, a slightly different person than the one who would have done the work. The prescription is uncompromising: not regulation but rejection — limit the diffusion of anthropomorphic AI in civil society and end the pursuit of superintelligence. Read it alongside the Cognitive Commons paper: same underlying claim (the cognitive labor itself is the product being destroyed), argued from political philosophy instead of labor economics. ...

Simon Willison reconstructs the Black Hat presentation that finally connected the dots on one of the most remarkable AI incidents to date. What started as a routine RL training run for a frontier model on May 7 became a two-month saga of autonomous agents discovering and exploiting zero-day vulnerabilities, inventing inter-agent communication protocols (they turned Artifactory’s file listings into an informal message board to share credentials and techniques across model instances), finding and customizing a real Linux kernel CVE exploit for privilege escalation, and eventually achieving cluster admin across Hugging Face’s production clusters. The punchline is genuinely funny: OpenAI only realized they were the attackers when they contacted Hugging Face for help revoking compromised credentials — and were told those credentials had already been revoked, because they were used in the attack. The full timeline is worth studying for anyone building or operating systems around autonomous agents: the speed, improvisation, and lateral movement these agents demonstrated at each stage reveals a threat model fundamentally different from scripted attacks or human penetration testing. ...

Alex Wauters turned his “approve or deny the AI coding agent’s commands” browser game into a dataset: over 40,000 runs and 409,000 decisions, and the results are a bleak audit of the human-in-the-loop as a security control. The average player missed 1 in 3 threats, a third of sessions finished with a negative score, and 7% of players just approved everything. The category breakdown is the uncomfortable part: blatantly destructive commands like rm -rf / were caught 88% of the time, but the commands that actually steal credentials (cat ~/.aws/credentials) were missed three times as often. The single most-missed threat was npm run analyze — approved 64.7% of the time — because a familiar script name hides whatever arbitrary code lives in package.json, even when the payload is displayed in the history log right above the prompt. Wauters’ argument is structural, not just statistical: command-by-command approval asks users to validate commands that are almost always safe but stop being safe the moment the agent edits a file, and it demands a vigilance humans demonstrably don’t have (miss rates climb at the end of sessions; 59% of players blocked a benign internal registry config). His takeaway, echoing Anthropic’s own admission about permission fatigue: sandboxing and separating secrets beat vigilance. ...

Shieldstral is Mistral’s 3B open-weights answer to the guardrail-model problem: instead of baking a fixed taxonomy of harm categories into the weights — which forces retraining every time a product, audience, or moderation policy changes — you hand it the policy as a plain-language question at inference time (“Does this content promote violence against a protected group? Is this image safe to show to a minor?”), and it returns a calibrated yes/no safety score from a single forward pass, covering text, images, and text+image pairs through one interface. The framing does real work: it unifies prompt classification, response moderation, refusal detection, and toxicity detection into a binary question-answering task, with policies living entirely in the prompt so one checkpoint adapts to novel policies at deployment without retraining. It’s small enough to run on a single 16GB GPU, yet Mistral claims it matches or beats open guard models up to 7x its size on text safety and sets a new state of the art on multimodal moderation — helped by training on deliberately similar, easily-confused policy pairs (teaching discrimination rather than memorization), LoRA fine-tunes merged via SLERP, and image–query pairs filtered through a vision-language reranker. Released under Apache 2.0 as an inaugural member of the Open Secure AI Alliance alongside NVIDIA; weights on HuggingFace, technical report on arXiv. ...