Apple MLX Runs 816B-Param LLM at 36 tok/s on Four Mac Studios

Apple’s ML Research team demonstrated that the open-source MLX framework, updated to v0.30.1, can run Muse Spark Open—an 816-billion-parameter mixture-of-experts model in NVFP4 precision—across a cluster of four Mac Studios, sustaining over 36 tokens per second. The setup relies on macOS 26.2’s new RDMA-over-Thunderbolt 5 driver and MLX’s “jaccl” communication collectives to pipeline and tensor-parallelize model layers across machines linked by dual Thunderbolt cables. The M5 Ultra’s GPU Neural Accelerators deliver up to 4.6× faster time-to-first-token and 1.6× faster decode throughput compared to the prior M3 Ultra, while peak power stays around 300W—well below typical datacenter GPU rigs.