Morgan Suo of MiniMax uses a financial-research assignment to explain model-speed tradeoffs. A 17-minute talk on AI Engineer.

Start with an acceptable finished report

  • The task spans 14 company reports: are circuit-board suppliers turning growing AI-server demand into cash?
  • An analyst needs accessible sources, matching reporting periods, checkable calculations and evidence that could change the conclusion.
  • Rising profit can coexist with weak cash flow when customers owe money or inventory absorbs cash.

Faster internals still need task-level checks

  • Sparse attention selects parts of a long input for closer processing; a short cash-flow footnote can matter more than pages of outlook.
  • A separate retrieval test reportedly passed 39 requests, with inputs reaching about 830,000 tokens; real-report evidence needs its own tests.
  • Lower precision stores internal numbers in fewer bits, reducing data movement; average scores can hide regressions on particular tasks.
  • Speculative decoding drafts several tokens for the main model to check; drafting, rejected work and verification all consume time.

More output can mean a longer wait

  • A separate serving test doubled simultaneous requests from 32 to 64 and produced about 1.9 times the total output per second.
  • The slow-end wait for the first token rose sharply; measure delivery of the usable report as well as initial output.
  • A separate coding test fell from 80% accuracy at maximum reasoning to 68% at high reasoning, despite using fewer output tokens.
  • Set and record the reasoning budget explicitly; the documented default discussed here is maximum effort.

Compare the whole workflow

  1. Fix the question, source reports, information cutoff and acceptance criteria before comparing models.
  2. Count retrieval, tool calls, retries, failed attempts and human corrections in the cost of an acceptable report.
  3. Test expected user load against a delivery deadline, preserving model versions, runtime settings, prompts and failure examples.
  4. Roll out gradually and keep a rollback route.

Caveat: these are MiniMax’s internal examples and separate component tests. The talk gives a method for evaluating the research workflow, without a complete measured before-and-after result for that job.

“If a person must fix the report before use, count that work too.” — Morgan Suo