Reverse-Engineering Apple's Neural Engine as the Standalone NPU Fades

A developer who abandoned an open-source Linux driver for Apple’s Neural Engine (ANE) three years ago has returned to fully map the accelerator’s M1 internals — not to make it useful, but to document its architecture before the standalone NPU disappears. The trigger is Apple’s M5, whose headline LLM performance came alongside folding the ANE cores into the GPU. To the author that confirms the end of the dedicated neural block: the design assumptions Apple committed to silicon starting with the 2017 A11 Bionic were tuned for the CNN era, and the industry has since moved to GPUs running transformer workloads.

The teardown shows the ANE’s compute is unremarkable on its own — 16 cores of 128 FP16 (or 256 INT8) multiply-accumulate lanes, 2048 in total, each doing a time-domain scalar reduction into a 32-bit Q16.16 accumulator that clamps at 2^15 and reads out as FP16. What actually specialized the chip was the dataflow around those MACs: it exploited the predictable weight-and-activation reuse of dense image convolutions to stay efficient enough for a phone. Autoregressive transformer decode broke exactly that predictable-reuse assumption, which is why the raw MAC array survives inside the GPU but its bespoke datapath does not.

Digging into compiled CoreML register files, the author also recovered how the ANE handles nonlinear activations: functions like tanh and ReLU are implemented as 33-entry piecewise-linear lookup tables applied inline to each completed reduction, avoiding a memory round-trip, with an impulse-response test confirming adjacent entries are linearly interpolated. The work is a rare, concrete look at the microarchitectural bets a major vendor made on ML hardware — and a case study in how quickly workload shifts can render fixed-function silicon obsolete.