membarrier(2): flip the memory-ordering tax from readers to writers

On x86, smp_mb() costs a LOCK-prefixed instruction or MFENCE. On ARM64 it is DMB ISH. Either way, if you have a hundred threads reading a shared pointer millions of times per second, that barrier on every read adds up fast. The question worth asking: does the barrier have to sit on the read side?

membarrier(2) says no. You register your intent once at startup, then call it from the writer after the store. The kernel issues an IPI to every thread in your process, each of which executes a full memory barrier on the kernel-to-userspace return path. Readers pay nothing. Writers pay once per pointer update instead of every CPU paying on every read.

The syscall

#include <sys/syscall.h>
#include <linux/membarrier.h>
#include <unistd.h>

int ret = syscall(SYS_membarrier, cmd, flags, cpu_id);

No glibc wrapper before glibc 2.35. Use syscall(2). The cpu_id argument was added in Linux 5.10 for the MEMBARRIER_CMD_PRIVATE_EXPEDITED_RSEQ command; pass 0 for everything else.

Query what the running kernel supports:

int supported = syscall(SYS_membarrier, MEMBARRIER_CMD_QUERY, 0, 0);
if (supported < 0)
    err(1, "membarrier not available");
/* supported is a bitmask of MEMBARRIER_CMD_* the kernel accepts */

Commands

MEMBARRIER_CMD_GLOBAL (Linux 4.3): wait for every CPU on the system to pass through a memory barrier. No registration required. This is the slow path: it includes kernel threads and other processes, so it can block for a scheduler tick. Useful for one-shot teardown paths, not hot loops.

MEMBARRIER_CMD_PRIVATE_EXPEDITED (Linux 4.14): IPI only the threads belonging to your process. Much faster. Requires that you register first:

/* once, at startup, before any reader threads exist */
ret = syscall(SYS_membarrier, MEMBARRIER_CMD_REGISTER_PRIVATE_EXPEDITED, 0, 0);
if (ret)
    err(1, "membarrier register");

Registration tells the kernel to set a per-thread flag on every new thread your process spawns, so the expedited path knows who to IPI. Miss the registration and you get EPERM.

The RCU pattern

The classic QSBR/RCU reader-writer split without membarrier:

/* writer */
atomic_store_explicit(&ptr, new_value, memory_order_release);
smp_mb();   /* full barrier: ensure all CPUs see the store */

/* reader (every single call, on every CPU) */
smp_mb();   /* acquire barrier before the load */
val = atomic_load_explicit(&ptr, memory_order_relaxed);

With membarrier, readers drop their barrier:

/* writer */
atomic_store_explicit(&ptr, new_value, memory_order_relaxed);
syscall(SYS_membarrier, MEMBARRIER_CMD_PRIVATE_EXPEDITED, 0, 0);
/* kernel IPIs every thread; each executes a barrier on return to userspace */

/* reader: no barrier needed */
val = atomic_load_explicit(&ptr, memory_order_acquire);

The guarantee: by the time membarrier returns, every reader thread that was running before the store has passed through a barrier. Any reader that starts after membarrier returns sees the new value by definition because the store completed before the IPI.

liburcu has shipped a sys_membarrier scheme since before the syscall was upstreamed (it had a fallback using /dev/cpu_dma_latency and sched_setaffinity tricks). The kernel variant is cleaner and available on any machine running Linux 4.14+.

SYNC_CORE variant

MEMBARRIER_CMD_PRIVATE_EXPEDITED_SYNC_CORE (Linux 4.16) goes further: it issues an icache-serializing instruction on every thread’s return from the IPI. This is the command JIT compilers need when patching code in place: after writing new machine instructions into a mapped region, call this variant to ensure no CPU is executing stale icache lines from that region. Requires MEMBARRIER_CMD_REGISTER_PRIVATE_EXPEDITED_SYNC_CORE at startup.

Without it, __builtin___clear_cache() or cacheflush(2) on the thread that patched the code only flushes that thread’s icache. Other CPUs see the old instructions until they take an exception or a context switch forces a pipeline flush. On x86 this is usually fine because x86 hardware self-snoop icache for other CPUs in the same socket. On ARM64 and RISC-V it is not.

Errors worth knowing

EPERM: you called MEMBARRIER_CMD_PRIVATE_EXPEDITED without registering first.

EINVAL: unknown command, or flags nonzero for a command that doesn’t accept flags.

ENOSYS: kernel older than 4.3, or the CONFIG_MEMBARRIER Kconfig option is not set (rare on any modern distro).

FreeBSD

No membarrier(2). FreeBSD’s atomic_thread_fence(9) with ATOMIC_ACQ_REL or memory_order_seq_cst on the read side is the equivalent, but it puts the cost back on readers. For RCU-style patterns on FreeBSD, epoch(9) in the kernel and liburcu in userspace both fall back to the POSIX per-thread signal approach (send a signal to each reader thread, the signal handler issues the barrier). Correct, but slower than kernel-assisted IPIs by a factor of two to ten depending on thread count.