Written by the maintainer of dial9 (a flight recorder for Tokio) after a RustConf Unconf session, this is a living document of async Rust performance principles. The honest framing up front: there are few hard-and-fast rules, because performance depends on what else is running on the runtime at that moment — which is why so many of these problems only show up in production.

The principles that carry the most weight:

  • Measure first. Most applications have long polls that are entirely benign; work backwards from a real metric, and treat schedule latency (ready-to-polled delay) as the most useful single signal.
  • Split for latency, batch for throughput. Explicit yields make pipelined connections fair — one mini-Redis example cut p50 from 0.967ms to 0.105ms — while batching amortizes the runtime events themselves.
  • Beware global resources. The blocking pool is global and showed problems around 50,000 spawn_blocking calls per second on a 32-core host; tokio::fs is called “considered harmful” for exactly this reason.
  • Keep mutex critical sections tiny. A contended blocking mutex can stall every worker at once, so stealing is impossible and the whole runtime wedges — the classic culprit being a metrics registry whose flush holds the lock.
  • Constrain parallelism with a semaphore, and isolate Tokio workers with cgroups or pinning; under OS load, waking a worker can take 10–20ms, which is fatal at single-digit-millisecond p99.
  • Know when to ignore all of it: long polls are fine at light load, and multiple pinned runtimes are the strongest isolation when latency-sensitive and background work share a process.

What makes this more than a tips list is the diagnostics. Every section ends with a “How do I know if I have this problem?” checklist — p99 far above p50, spawn_blocking hot in flamegraphs, a consistently deep global queue, latency spikes at predictable intervals. That turns advice into something you can actually measure.

Worth noting, given the source: the author says dial9 as often proves Tokio is not the problem as it finds one, which is an unusually credible stance for someone selling Tokio diagnostics.