Metal capability shim gives macOS VMs near-bare-metal LLM speed on Apple Silicon

Cua, the team behind the Lume macOS virtualization stack, has found that macOS guests running under Apple’s Virtualization.framework were leaving most of their GPU performance on the table. The paravirtualized GPU that Apple exposes to a guest reports a conservative Metal profile — roughly an Apple family 5 device with 32 KB of threadgroup memory and no SIMD-group matrix support — so Metal apps like llama.cpp dutifully pick slower kernels even though the underlying hardware can run faster ones. The fix is a small, process-scoped shim that intercepts a guest process’s Metal capability queries and returns higher values: it answers supportsFamily up to Apple family 9 and raises the reported threadgroup memory to 64 KB. That is enough to steer llama.cpp onto its newer SIMD-group reduction, SIMD-group matrix, and bfloat16 code paths, all while the work still runs on Apple’s existing virtual GPU path rather than any real PCI passthrough.

The measured gains are large. On an M1 Ultra, TinyLlama 1.1B ran prompts 11.08× faster and generated tokens 16.36× faster than the same stock VM, with prompt processing reaching 98% of bare metal (token generation lagged at about 72%). Google’s Gemma 4 12B QAT model improved 7.20× on prompt processing and 14.54× on generation, hitting 99.59% and 94.82% of bare-metal speed respectively. A 30B Muse Glimmer model in a 64 GiB guest saw 7.55× and 8.87× gains. Cua is shipping the source, build scripts, capability probe, and raw benchmark logs under the same permissive license as Lume so others can reproduce and map which chips, macOS versions, and workloads benefit.

The significance is less about a single benchmark and more about the mechanism. This is not GPU passthrough in the VFIO sense — no physical device is assigned to the guest, and no kernel changes are involved. It is a demonstration that Apple’s virtual GPU can already execute modern Metal kernels, and that the only real barrier is a conservative capability report that applications trust by design. The same limitation has surfaced elsewhere — Tart, another Virtualization.framework CLI, has an open issue about weak GPU and LLM performance in macOS guests — which suggests the shim could benefit anyone running local inference or GPU-heavy workloads inside macOS VMs. The trade-off is that each additional Metal API needs separate validation, since a spoofed family number only vouches for the paths that were actually tested.