Akshat Bubna, Modal’s CTO, outlines the infrastructure behind agents that run code, pause for people, and share workspaces in this 18-minute AI Engineer talk.

Separate the agent loop from its tools

  • His opening example gives every spreadsheet cell an agent, with shared storage scoped to collaborating rows.
  • Keep the harness, the software running the agent loop, separate from the container executing its tool calls.
  • A sandbox may contain multiple containers or microVMs, including a sidecar for supporting services.
  • Launch compute when work arrives, then preserve its state when the agent pauses.

Persist the workspace, release idle compute

  • A coding agent needs its repository, dependencies, and development services together.
  • An accounting agent may wait days for a person’s answer before resuming its documents and scripts.
  • Choose whole-filesystem snapshots, selected-directory snapshots, or shared volumes according to what must survive.
  • Separate a dependency directory’s lifetime from upgrades to the base image.

Size resources for the current step

  • Bubna shows Ramp Inspect workloads whose CPU and memory needs vary across agents and within one run.
  • Provisioning every sandbox for the observed peaks wastes capacity; he argues for resources that expand during expensive steps.
  • Let agents request GPUs for scientific models, rendering, video work, and training experiments.
  • Shared files need clear consistency rules when humans, parent agents, and subagents edit together.

Enforce access outside the agent

  • Control incoming connections, outgoing destinations, authentication, and how credentials reach external services.
  • Inject credentials through controlled infrastructure so the agent cannot read their secret bytes.
  • Domain allowlists alone are too coarse when only particular API paths or operations should be allowed.
  • Bubna describes customer-written sidecar proxies that enforce those narrower rules.

Make failures visible to agents

  • Give agents training logs, live metrics, GPU utilization, and spending information they can inspect.
  • A failed experiment should feed a diagnosis and a revised run, rather than leave the agent waiting blindly.
  • Reliable infrastructure still requires engineering for rare failures, security boundaries, and large bursts of work.

From identical servers to changing workspaces

  • Bubna’s “brains and hands” metaphor separates the agent loop from the environments it creates and uses.
  • He extends “pets versus cattle” into an “ocean of microbes”: many changing environments collaborating and retaining useful artifacts.
  • Modal’s customer and scale figures are vendor-reported; this talk supplies an architecture argument, not a comparative benchmark.

“Great primitives let your agents do more.” — Akshat Bubna, 17:20