FEX-Emu runs x86 Linux binaries on ARM. Almost every hard performance problem it has traces back to one mismatch: x86’s Total Store Ordering memory model versus ARM’s relaxed one — with benchmarks to show what each shortcut costs.

What makes x86 different:

  • Under x86-TSO, stores become visible to other cores in program order, and unaligned loads and stores inside a cacheline are atomic — they never tear, even in a race.
  • ARM’s weak model guarantees none of that by default. Unaligned accesses can tear, and acquire/release-style ordering costs real bandwidth when it is the common case rather than the exception.
  • The baseline mapping (every x86 load becomes a load-acquire, every store a store-release, with memory barriers patched in for unaligned accesses) is correct but slow. On some CPUs the store path collapses entirely.

What hardware has actually delivered:

  • FEAT_LRCPC (ARMv8.3+) adds TSO-shaped loads. FEX switches to them on detection and the microbenchmark penalty largely disappears.
  • FEAT_LSE2 loosens alignment requirements to a 16-byte granule — but x86 software does not respect 16-byte granules, so the help is marginal.
  • Apple Silicon adds a real x86-TSO mode: regular ARM loads and stores change behaviour, and unaligned accesses stop needing patched-in barriers.
  • Qualcomm’s Oryon-3 ships coherent cachelines, so atomics anywhere inside a cacheline cost the same as aligned ones.

What is still broken:

  • Split-locks — atomic operations that cross a cacheline — must never tear on x86, and ARM has no instruction that reproduces that. FEX handles them by taking an alignment fault on every execution: a kernel → signal handler → back-to-code round trip, thousands of times per second.
  • Even a global mutex is not correct, because aligned atomics that do not participate can still tear the data underneath.
  • Uncached, write-combine memory — what games use to hand data to a PCIe GPU — is the worst case, with store bandwidth up to 816x worse than native x86. Two shipping games dropped below 1 FPS because of it.
  • Their proposal: let ARM’s 128-bit CASP straddle the atomic granule boundary, so a failing compare-exchange can safely retry. It needs hardware cooperation nobody has committed to.

The takeaway is that a memory model is not an abstraction detail you can shim away. It is an ecosystem-wide design constraint, and every ARM extension in this story closes exactly one edge case.

The cost of the remaining holes is measured in games that run at a fraction of a frame per second, which is a useful reminder of what “we can just emulate it” actually prices in.