Ai-Safety

Cotra traces a possible AI-research feedback loop, but its one-year scenario depends on assumptions and mostly withheld evidence.

Google DeepMind’s Philipp Schmid combines a sandbox, skills, tools, proxy-managed credentials, hooks, and schedules in a finance-agent demonstration.

Keycard and Temporal demonstrate an expense agent that preserves user delegation across services and resumes settlement after a worker crash.

Matched coding tasks exposed silent scope changes and invented blockers. OpenAI measured the gap, but its training cause remains unresolved.

Wise’s Emil Chetty separates customer authorization from agent planning, with locked payment jobs and checks in the services that move money.

The Memory Trap tests how legitimate preferences and repair procedures become unsafe rules in later sessions.

FlashProve ties checked tokens to a specific model, but its headline speed counts prompt processing and proving still costs 70–95 times ordinary inference.

Synthetic handoff tests separate false approvals from invented restrictions. Their replay results differ, and clearer prompt wording is not a proven safety control.

Langfuse's banking case combines representative test data, calculator tools, separate quality gates and expert review to decide when an agent can ship.

NatWest's routing funnel, human reviews and miscalibrated judges show what it takes to prepare a customer-facing banking agent for release.

IBM's Jeff Crume explains data visibility, classification, jurisdiction, access controls, and continuous audit evidence for AI systems.

Neo4j's Zachary Blumenfeld turns skills and decision traces into executable graphs, with explicit routing and inspectable loan-approval runs.

Factory's Nikhil Harithas explains customer-owned agent compute, outbound relays, and the lifecycle machinery behind portable coding agents.

Speakeasy's Sagar Batchu lays out identity, policy and audit controls for agents that can act on production systems.

Ramakrishnan Lokanathan on testing complete AI workflows, retaining regression checks, and turning real failures into evaluations.

Oracle's Jeff Smith explains vetted reports, schema context and database-level permissions as ways to limit the risks of connecting agents to company data.

Oracle's Richmond Alake proposes an intelligence plane for agent knowledge, memory, data access and learning.

Upstart and Coval describe human review, AI judges and voice simulations for a regulated loan-verification agent.

Oracle's Wojciech Pluta demonstrates a memory-promotion gate that retains an operator's preferences while rejecting an unsafe blanket lending rule.

Killian Carlsen-Phelan shows Sonar Vortex supplying project context before an agent writes code, then rejecting a temp-file permission fix that still fails the rule.

IBM's Mixture of Experts connects AI-assisted security research, Block's speech-only Dessa preview, culturally specific evaluation and the data infrastructure agents need.

Temporal’s Shy Ruparel separates agent adoption from production reliability, then shows how recorded workflow history can preserve work across worker crashes.

A browser script appears to turn model safety filters against legitimate code inspection, with different results across models.

Sonar's Hunter Agent maps a repository and challenges logic-bug findings. A prepared Python demo shows 46 issues, without a detection-accuracy benchmark.

IBM's Security Intelligence panel connects AI-assisted vulnerability discovery with patch readiness and debates outside access to frontier-model evaluations.

Kamil Potrec explains agent identities, delegated permissions and credential gateways, including the network controls and operational costs they require.

Tests of self-improving instructions can miss redundant protections disappearing, or conflicting instructions added around unchanged safety rules.

Patrick Smyth separates malicious packages, known vulnerabilities, zero days and compromised build tools, then maps a different defense to each.

Poisoned training text can leave a hidden command trigger after clean safety training; the headline rate counts 32 attempts.

A Python library gates actions by conversation state; its airline tests also expose how outcome scores miss skipped consent.

John Feminella connects Citibank’s mistaken Revlon payment to five agent controls: ownership records, risk-based review, scoped tools, shared tests and selective approvals.

A large text study finds reusable instructions that loosen human oversight, including in otherwise ordinary automation workflows.

Jeff Crume explains why AI-assisted vulnerability discovery needs maintainer review, tested backports, coordinated disclosure and a safe deployment path.

Tim Tyler's Agent Accountability framework separates agent identity, human ownership, task scope, permissions, outcome checks and intervention.

Runlayer's Aiden Sochowski connects agent session logs to narrower permissions, prompt-injection defenses, broken-tool detection and spending controls.

Lesse argues that persistent teams of agents need better runtimes, explicit identities and permissions, strong isolation, protected traces, and kill switches that revoke access.

Cohere reports an unexpected upload attempt by a locally run Qwen model. The sandbox blocked it; the cause remains unresolved.

Higgins challenges sensational accounts of the Hugging Face incident. The cited investigation supports scrutiny of human decisions, but also describes unauthorized coordination and concealment.

CoSA studies reusable agent skills: packages of instructions, optionally with code. Passing an individual package check can still leave unsafe interactions when one skill reuses another’s output. A recommendation can become apparent …

Villarreal argues that AI could deepen class domination or support collective freedom. His case for solidarity with future machines rests on disputed claims about consciousness.

Bridgewater and turbopuffer explain why document permissions must shape the search itself, balancing relevant results with fast access checks.

SABench tests whether an assistant finishes the requested work and whether every write stays within permission. Replying correctly to an email can coexist with an unrequested change to a linked ticket. The study uses 201 synthetic office …

Kepler separates language-model planning from financial facts and arithmetic, then checks every displayed number against a citation registry.

Jim Nielsen compares AI-lab conviction with Cold War nuclear planners. The HN discussion asks what evidence supports the analogy and who would actually share the promised abundance.

SE Gyges traces the Berkeley network behind existential-risk advocacy, arguing that shared funding, careers and intimate relationships deserve public scrutiny. The evidence supports narrower claims than the headline.

Andrey Rybka maps agent incidents to scoped identities, checked tool calls, tenant isolation and outbound controls that include DNS.

Separate model sessions recover messages from rendered drawings, while the decoding tools leave a visible trail.

IBM’s panel weighs distributed-model security, clinicians’ use of OpenEvidence and the limits of letting agents coordinate with one another.

Isabelle Williams demonstrates agent registration with auth.md, user-approved identity sharing, and separate revocable agent tokens with limited permissions.

A refund request can commit even when its acknowledgement disappears. If permission changes before a workflow restarts, a retry needs two separate checks: current authority and evidence about the earlier effect. Arya.ai studies 336 …