Packaging normaliz for a community Debian LoongArch port kept timing out: the math library’s built-in test was stuck in a loop it could not exit. The loop terminated when nr_points_matched == nr_to_match, and nr_points_matched — incremented under #pragma omp atomic alongside a second counter — simply stopped reaching the total, even though every point in the list had been processed.

The value of a counter that four threads increment atomically was diverging from reality. Six months later, Wang Miao and Jiajie Chen established why: on the LA664 core, Loongson’s atomic add occasionally loses updates.

The first round of investigation, in February 2026:

  • Memory ordering was ruled out — the code only touches the atomic variables themselves, so it is logically correct under any ordering
  • Disassembly confirmed the compiler emitted amadd.d, the expected LoongArch64 atomic add
  • Two extra std::atomic control counters were added; all four diverged, by different amounts, run to run
  • A standalone atomic-add benchmark would not reproduce it, and stripping computation out of normaliz would not either, because the program was too large to reduce by hand

They shelved it. In August they came back with a different method: point a model at the loop, tell it the bug is LoongArch-only, then tell it the problem is already localized to the atomic add and ask for a minimal reproducer. It landed on the memcpy call in the processing function — glibc selects a vectorized implementation via multi-arch dispatch, so it uses LASX (xvld) on this hardware. Two days later there was a reliable reproducer.

Reproducing a lost update required all three conditions at once:

  • Threads running on different physical cores, not two SMT siblings
  • Atomic operations on the same address with no data barrier
  • At least one thread interleaving vectorized memory reads between atomic operations

Failure rates over 30 trials on a 3C6000/S ranged from 43% (ammax.d, both sides doing LASX copies) to 100% (amadd.d, ammax.d, amswap.d with both sides doing LASX reads). The barrier-carrying form amcas_db.d never failed. amcas needs a new ISA extension to compile, and ammax/ammin have no C interface, so the one that matters in practice is amadd — reference counting.

The aside is the best part of the story. AOSC OS could reproduce the hang in February but not in August, while Debian could do both. AOSC’s glibc had been built with --enable-multi-arch disabled after February, so its memcpy no longer took the LASX path. On Debian, GLIBC_TUNABLES=glibc.cpu.hwcaps=-LASX makes the bug vanish the same way. The trigger had been silently removed from an entire distribution by a build flag.

The stakes are memory safety, not vulns. A lost increment on a refcount frees an object early — use-after-free or double free. A safe Rust program using std::sync::Arc or cloning an mpsc::Sender crashed with SIGABRT or glibc heap corruption, because both use barrier-less amadd. Exploiting it deliberately is impractical: attacker and victim must be in the same process.

Application developers have no lever here — the compiler picks the instruction — so software-level avoidance means emitting am<OP>_db or LL/SC loops, which requires recompiling every affected binary. Loongson’s actual fix came fast: reported August 26, test firmware September 9, set bit 13 of an undocumented internal CSR (MCSR24), near-zero performance cost, releasable from the Linux kernel if a firmware update is not available.

The lesson the authors draw is about errata generally: they are common across vendors, most are harmless, and the expensive part is not the fix but the months spent without a minimal reproducer. Better to localize the cause and report it while it is still a flaky test than to have it surface later as an unexplained crash in production.