Oliver Kennon, a SemiAnalysis research analyst, discusses AI infrastructure economics in this 17-minute AI Engineer talk.

Compare useful output

  • GPU hours measure the compute you buy; completed tasks measure what customers get.
  • Kennon proposes full cost per successful task at the required quality as the eventual comparison.
  • Token prices offer a practical proxy when workloads and quality requirements are comparable.

The operator changes what the hardware delivers

  • His InferenceX example compares B200 deployments serving the same model with roughly 8,000 input and 1,000 output tokens.
  • Serving software, settings and concurrent requests change how much output the system produces while meeting its response-speed target.
  • Kennon reports about 50% more output from one configuration on the same chip type.
  • With equal hourly bills and acceptable responses, the higher-output setup spreads rental spending over more usable tokens.

Reliability and security count too

  • Useful responses must meet quality, speed, availability and agreed access boundaries.
  • Kennon describes SemiAnalysis finding monitoring access that exposed another customer’s logs and metrics.
  • Operators need workload tests, load tests, safe rollouts, rollback options and ongoing maintenance.
  • SemiAnalysis’s ClusterMAX rates managed clusters; its ratings do not automatically cover a provider’s bare-metal products.

Count the whole bill

  • A GPU rental payment is revenue to the provider and an input cost to the inference service selling responses.
  • That inference service also pays for software, staff and supporting services.
  • Owners must spread equipment investment and running costs over useful output.
  • Better operations cost money themselves; additional output must justify that spending.
  • A durable advantage requires repeated improvements as workloads, software and competitors change.

Understand what a compute price index measures

  • Service grades should describe what buyers receive and which operating responsibilities remain theirs.
  • A GPU rental index tracks prices in a defined market; its sampling and contract terms matter.
  • Compute futures may help manage rental-price changes when the reference tracks the actual rent paid or earned.
  • Operator ratings alone cannot create a reliable price benchmark, liquid market or effective hedge.

Caveat: the 50% figure is a workload-specific comparison presented by SemiAnalysis. It is not a general gain for every model or a complete service-cost saving.

“A GPU falls to the level of the operator.” — Oliver Kennon, 0:30