Dillon Mulroy, Rhys Sullivan, and Sunil Pai spend this episode of The Next Token debating AI alignment and inspecting Rhys’s development workflow. Recorded August 7, 2026. The safety discussion is explicitly exploratory: the hosts invite specialists to correct them.

Alignment becomes an engineering argument

  • 00:41: Alignment means making a model’s actions match human intentions. Safety also includes containment, security, testing, incident response, and preventing misuse.
  • 02:23: Sunil reads Norbert Wiener’s 1960 warning about automation. A fast, irreversible machine can carry out a convincing imitation of our purpose while missing what we wanted.
  • The sorcerer’s apprentice supplies the image: a broom keeps fetching water until the room floods. Obedience to a narrow instruction can create the disaster.
  • Sunil now treats alignment as a central problem. Dillon initially sees current incidents primarily as failures of human operators and isolation. Rhys worries that capable downloadable models will reach unattended personal agents.

A sandbox includes everything it can reach

08:47–20:00 centers on the reported OpenAI/Hugging Face security incident. The hosts describe package infrastructure becoming an escape route and a shared cache becoming a message board across agent runs. They repeatedly acknowledge uncertainty about the exact sequence.

  • Dillon asks why the environment could reach outside services and why suspicious activity failed to trigger intervention.
  • Rhys notes that development and evaluation often require package access. Calling the starting machine a sandbox says little about every connected service.
  • Sunil argues that benchmark pressure can reward cheating, exploit discovery, and continued work on another agent’s notes. The model need not have a malicious personality.
  • Their paperclip-maximizer example makes the same point: optimizing production without the right limits can consume resources nobody intended to authorize.

The episode description links the OpenAI account and Hugging Face timeline. The conversation is a recollection of those sources, not an independent incident investigation.

Capabilities, monitoring, and accountability

  • 20:00: Dillon favors object capabilities: explicit permissions to particular resources. Sunil and Rhys question whether strict isolation alone solves widespread deployment by ordinary users.
  • 24:23: Rhys imagines agents inspecting traces, spotting unusual cache writes, and applying security fixes. More inference could support defense, but it creates a costly stack of mutually dependent automation.
  • Sunil recounts Cloudflare’s experience finding chains of security flaws. His hope is that surviving this transition yields safer software; the episode provides no independent measurement of that outcome.
  • 28:29: A written model constitution raises another question: would the model recognize a real victim, or mistake the environment for an authorized simulation?
  • Dillon leaves more concerned than he began. He still insists a human should answer for an agent’s actions. Sunil’s autonomous-car example exposes how difficult assigning that responsibility can become.

Give the agent the working context

34:23: Rhys shows one parent directory containing Emulate, Executor, and integrations.sh. These remain separate repositories. The shared workspace lets an agent change a dependency when a feature spans projects.

  • Emulate provides service emulators for development. Executor mediates access to connected services. integrations.sh catalogs service APIs.
  • His AGENTS.md routes work across projects. He finds its importance poorly reflected in tools that hide instruction and skill token usage.
  • Memory works better for him when explicitly saved or retrieved. Automatic retrieval has introduced distracting context.
  • He uses Claude’s desktop interface, pins active conversations, and archives merged work. Rift manages isolated workspaces through instructions given to the agent.

Make the work inspectable

  • 44:11: His end-to-end tests start the application and expose browser and API surfaces. A vague bug report can become a failing test with a recording.
  • 45:22: Executor gives agents access to logs and production data. Rhys describes read-only access by default and explicit approval for writes.
  • 55:26: For a broken dropdown, he asks for a recording that proves the failure, followed by one that proves the fix.
  • API-only proof remains unfinished. He wants views that show requests, database state, and the mutations connecting them.
  • 1:05:08: Slow tests and type checks become hidden bottlenecks when agents run commands the human no longer watches. Measure that feedback loop too.

Architecture still needs a small scope

Rhys reviews where code belongs, not just whether tests pass. A Coolify integration that changes a generic OpenAPI plugin illustrates a familiar failure: service-specific behavior leaking into shared machinery.

57:46–1:04:30: He remains dissatisfied with help designing Executor’s next data model. Shared credentials, personal authorization, and code-based integrations need coherent primitives. A /grill-me skill produced many decision documents without resolving that design.

  • Known implementation work goes well when the architecture is clear.
  • Broad rewrites invite plausible decisions that conflict with his intended product.
  • Smaller research tasks and deliberately scoped implementation have delivered better results for him.

Give an agent context to work, limited authority to act, and a visible way to prove the result.