Steven Latré, who leads AI at imec, presents the research center’s coding-agent infrastructure study at AI Engineer.

What the team tested

  • imec reports thousands of runs on SWE-bench Pro, a software-engineering task benchmark, across models, coding harnesses and hardware.
  • The comparison considers task resolution, latency, concurrency and infrastructure cost rather than model accuracy alone.
  • These results describe one coding-assistant workload, not every enterprise application.

Hardware and concurrency change the calculation

  • Model hardware requirements rise in steps: a small change in model choice can require a much larger GPU configuration.
  • Each simultaneous user adds memory pressure from the model’s working cache; serving more users can also increase latency.
  • In one large-model setup, Latré reports about four comfortable concurrent users and latency roughly three times the API baseline.
  • The team’s modeled break-even utilization ranged from about 89% for a smaller system to 15% for an eight-B200 system; these are workload-specific estimates, not universal thresholds.

The harness matters too

  • A coding harness coordinates the model, tools and task execution; similar task success can come with very different token use.
  • For one GLM configuration, Latré says Codex used about half the generated tokens and GPU time of the other tested harnesses.
  • imec plans to publish a broader benchmark dataset; its results and cost assumptions should be checked against your own workload.

“Moving this from API to infrastructure … is just not a very simple or very linear choice.” — Steven Latré