Nicholas Arcolano, head of AI and research at Jellyfish, examines AI coding output in this 17-minute AI Engineer talk.

Measure the whole working environment

  • His selected dataset covers about 50,000 engineers at 447 companies using Cursor or Claude Code.
  • The outcome is merged pull requests per engineer per week: proposed code changes accepted into a project.
  • His statistical model attributes under 1% of variation to the tool, about 3% to skills, 9% to usage and 12% to company/team.
  • Individual differences account for roughly three quarters; that category bundles factors the talk does not fully unpack.

Skills can matter even when adoption is sparse

  • Almost half of observed engineer-weeks had essentially no skill use as of September; only about 12% fell into his high-use group.
  • At 30-plus skill runs per engineer per week, he reports roughly 30% higher throughput after accounting for measured factors.
  • A small population-level share and a large association among heavy users can coexist when adoption is limited.

More tokens have diminishing returns

  • An earlier analysis covers about 500,000 developer-weeks and associates higher model usage with more merged changes.
  • Moving from median consumption to the top tenth corresponds to roughly twice the output at ten times the tokens.
  • Company environments show roughly a twofold top-to-bottom-decile spread at fixed consumption; individuals show about 3.5-fold variation.

Improve what leaders can change

  • Review processes, codebase friction, shared skills and working culture are levers managers can act on directly.
  • In a separate 10,000-engineer comparison, top-quartile turnover changed little before and after AI adoption.
  • His golf analogy ends with an organizational advantage: leaders can change the course itself, including its hazards.

Caveat: these are vendor-reported observational associations using merged-change counts. The talk explicitly excludes code quality from this outcome and does not establish that buying tokens causes productivity gains.

“Unlike golf, you can change the course.” — Arcolano, 15:47