Swagath Venkataramani compares CPU, GPU, and dataflow scheduling in this 11-minute IBM Technology explainer.
Three ways to decide what runs next
- CPUs spend hardware on fast instruction handling, branches, and irregular work.
- GPUs pursue throughput by applying instructions across many parallel data elements.
- Dataflow-style accelerators explicitly map computation and data movement onto specialized hardware.
- The running example stays the same: compute
x = a + b, theny = x × c.
Scheduling and memory
- An out-of-order CPU executes ready operations early while retiring instructions in program order.
- A GPU scheduler selects ready thread groups, keeping arithmetic busy while other groups wait for memory.
- Branches that split a group across different paths reduce GPU utilization.
- A dataflow mapping sends the addition’s output directly to the engine performing multiplication.
- Systolic arrays pass weights and activations between neighboring multiply-accumulate units, reusing data for matrix multiplication.
Match the hardware to the workload
- CPUs suit mixed, branch-heavy software; GPUs reward sufficiently large, regular parallel work.
- Dataflow efficiency depends on supported operations, suitable tensor shapes, and a good mapping to the hardware.
- Dynamic shapes and control flow can weaken that mapping; compiler support matters alongside the chip.
- Caveat: this is an architectural explanation, with simplified memory models and no measured speed or energy comparison.
“The math did not change.” — Swagath Venkataramani, 10:56