Eileen Yoon shelved her reverse-engineered Apple Neural Engine driver three years ago, concluding the block was too opinionated to build a general-purpose accelerator platform around. She restarts the project now that the M5 has folded ANE cores into the GPU cores, reading that as the beginning of the end for the standalone NPU.

The goal is no longer to run workloads on the ANE but to map the architecture, on the theory that its internal decisions reveal what Apple was willing to commit to silicon in 2017 — and what the shift from CNNs to transformers did to those commitments.

Some of what the probes turn up:

  • 16 cores × 128 FP16 MAC lanes (256 INT8) — 2048 parallel MACs, with time as the only reduction axis. A dot product is a dot product; what specialized the ANE for CNNs was the dataflow around the MACs, and autoregressive decode is what broke it.
  • The accumulator saturates at 2^15. Feeding an all-ones dot product, a multiplier of 128 comes back as 0x7c00 (+∞) while the CPU sees a valid FP16 0x7800 — so the clamp is inside the 32-bit Q16.16 accumulator, not the FP16 readout.
  • tanh is a 33-entry lookup table, not a function unit: 33 FP16 words at 0x4288 matching round16(tanh(i/8)). A single-spike impulse LUT produces a triangle across neighbouring cells, proving piecewise-linear interpolation between entries.
  • The driver is boring on purpose. It never receives a CONV or RELU; operations are pre-compiled into fixed-size task descriptors, staged via TM_ADDR, and committed by ringing TM_PUSH — a GPU pushbuffer design.
  • There is no L2 to KMem path, so kernel fetches fall back to DRAM. Fine when kernels are static; fatal when decode makes the weights the streamed operand.

The measurements are the argument. Sustained DRAM read bandwidth on M3: ANE kernel DMA 37.99 GB/s, ANE tile DMA 59.08 GB/s, GPU 77.70 GB/s. And the kernel and tile DMA times are additive rather than overlapping, fitted as T_AB = 0.001 + 0.939·T_A + 0.981·T_B — the ANE is under the GPU roofline twice over. Since single-token decode is read-bandwidth bound, peak MACs do not enter into it.

The method generalizes past silicon: construct a probe that isolates one variable, hexdiff the compiled register file, and fit a slope instead of trusting a spec sheet. The conclusion is that a dataflow optimized hard for one era’s workload assumptions, plus one design decision that was free in 2017 and expensive later, is how specialized hardware becomes a block that only Finder’s preview upsampling regularly uses.