MacPaw researchers test four families of model-serving optimizations on Apple Silicon. Their measurements show why a server-GPU speedup needs checking against the model, software and request pattern you actually use.
- Pruning the KV cache, a model’s stored prompt working memory, reaches 1.75 times baseline speed in one Apple configuration. That experiment leaves answer-quality changes unmeasured.
- Six speculative methods, which draft tokens for the main model to verify together, all lose end-to-end speed in the principal M2 Pro/M3 Ultra chat comparison.
- Positive exceptions matter: DDTree gains 15.5% end-to-end on M3 Pro with the tested higher-precision target. Speculation can help particular Mac setups.
- Reusing unchanged conversation history saves 57% of ninth-turn wall time by avoiding repeated prompt processing. It does not make each generated token faster.
Kernel ports and lower-precision weights also have device-dependent results. Measure prompt processing, generation, memory and output quality separately before adopting a headline speed claim.
Caveat: The primary workload serves one request at a time. Multi-user serving and the quality cost of cache pruning remain open.