Peter Yang hosts Nan Yu and Jacob Shumway from Linear for a deep dive on the Linear Agent — an end-to-end production agent, live inside Linear’s own Slack. ~38 minutes.

What an agent actually is

  • An agent is “just calling an LLM in a loop”: a goal, milestones, tools to build its own context, then question-answer until the goal is done
  • Colloquially it’s a product — a facade (chatbot) over many interacting subsystems

Origin: the “Robotic Program Manager”

  • Nan’s 2025 memo: “Computers can do a lot of work for us. Let’s get rid of all the work we don’t want to do and give it to computers.”
  • First version was hacky — the LLM called from the front end with the command-menu actions as tools; stealth-launched in Slack
  • Users immediately got lazy: “@Linear do the right thing”, even a pointing-finger emoji — the agent reasons through what happened and acts

The five rules

  1. Give it tools to load context, don’t give it context — “give it as little instruction as possible… they’re just smart enough to get what they need if you give it a really good defined goal”
  2. Encode your opinions in loadable skills — they tried everything-actions and raw GraphQL (hallucinated); skills with opinions (how to set priority, write a description) made the agent behave like a power user of the app
  3. Let feedback become evals — evals grow from usage: unexpected user behavior gets added to the dataset; thumbs-down moments become evals (“we’ve had a ton of these”)
  4. Biggest model first, then optimize down — throw the frontier model at it until it works, build evals, then shrink to the smallest model that still passes
  5. Tie every agent to a human, and let it report its own gaps — work lands in someone’s backlog (assigned to Jacob, 6-minute issue-to-PR); when the agent lacks a capability it files an issue automatically, keeping a streaming list of what it can’t do

Evals at Linear

  • Mix of objective (deterministic: “if the user says in progress, always set the status”) and subjective (LLM-as-judge: did you extract the right info?)
  • Use judges sparingly — evals work best for consistency where consistency matters; agents have variance and too many evals produce false signals
  • The evals focus on ergonomics: “did you understand what the user actually wanted, or were you too eager?”

The bigger picture

  • “The biggest problem in applied AI is not that agents aren’t smart enough — it’s a capability overhang: models are really smart but we’re just not using them enough”
  • The middle 80% gets bigger until it reaches the edges: humans hold the first/last touch
  • Advice for builders: break down the actual workflow users want; the chat surface is the follow-up, the real entry points are Slack threads, meeting debriefs, project updates

“Give it as little instruction as possible. Give it the tools to load context. Don’t give it context.”