Why Emulating x86's Strict Memory Model on ARM Is So Painful

The FEX project, which runs x86 binaries on ARM hardware, breaks down the central bottleneck in x86 emulation: reconciling two opposite memory models. x86 uses Total Store Ordering (TSO), a strict model where a store becomes coherently visible to all cores and loads see prior stores in order—matching intuitive programmer expectations. ARM instead uses a weak, relaxed model that lets cores defer cache-line visibility to save power and gain efficiency, offering load-acquire and store-release instructions (ARM’s RCsc model) to enforce ordering only where needed.

On baseline ARMv8.0-a hardware, FEX had no middle ground, so it turned every x86 load into a load-acquire and every store into a store-release. That is actually stricter than TSO requires and carries a steep cost, because these ordering instructions were never meant to dominate the instruction stream. Microbenchmarks show acquire-loads dragging performance on three of five tested CPUs, with AmpereOne’s release-stores and the M1’s acquire-loads falling well below their normal load/store baselines—since acquire semantics prevent the CPU from reordering loads at all.

The fix arrived with the LRCPC load instructions, mandatory since ARMv8.3, which add the Release Consistency processor-consistent (RCpc) model built essentially for this x86-emulation use case. On chips that implement it, LRCPC loads run at nearly the speed of ordinary loads, largely closing the memory-performance gap. The article notes several harder, still-unsolved problems remain—atomic instructions, mandatory split-lock handling, and uncached memory—signaling that TSO emulation is far from fully solved even as newer ARM extensions help.