pkey_mprotect: in-process memory isolation without a syscall

mprotect(2) costs a syscall. That’s fine for one-shot setup but not for a JIT compiler that needs to toggle a code page between writable and executable on every emit pass. In 2016 Intel shipped a feature called Protection Keys for User pages (PKU) in Skylake, and Linux 4.8 wired it up: three new syscalls (pkey_alloc, pkey_mprotect, pkey_free) and a userspace register you can write without ever touching the kernel.

How it works

The CPU maintains a 32-bit per-thread register called PKRU. Two bits per protection key: bit 2n is the Access Disable (AD) flag, bit 2n+1 is the Write Disable (WD) flag. Set AD for key 3 and any read or write to memory tagged with key 3 faults. Set only WD and reads still work, writes fault. Clear both and the domain is fully open. The WRPKRU instruction changes PKRU in userspace, no privilege needed, no syscall.

You have 16 keys total (pkeys 0 through 15). Pkey 0 is implicit: every page of memory that never gets a pkey_mprotect call belongs to it. You get 15 usable slots for your own domains. That is enough to isolate a handful of secrets or JIT regions; it is not enough to sandbox a hundred plugins.

The syscalls

pkey_alloc(flags, access_rights) returns an unused pkey number or -1. Flags must be 0. The access_rights argument sets the initial PKRU bits for this key in the calling thread: pass PKEY_DISABLE_ACCESS to lock the domain immediately on allocation.

pkey_mprotect(addr, len, prot, pkey) is mprotect(2) with a fourth argument. It sets the VMA protection flags as usual and stamps the pkey onto every page in the range. After this call, whether a thread can actually access those pages also depends on its PKRU state for that key.

pkey_free(pkey) releases the slot. Do this after you have untagged the memory.

#include <sys/mman.h>
#include <sys/pkey.h>    /* glibc 2.27+: pkey_alloc, pkey_set, etc. */
#include <err.h>
#include <string.h>

/* Allocate a key; lock it immediately in this thread. */
int pk = pkey_alloc(0, PKEY_DISABLE_ACCESS);
if (pk == -1)
    err(1, "pkey_alloc");  /* ENOSYS if kernel < 4.8 or not x86 */

/* Map a page for a secret. */
void *buf = mmap(NULL, 4096, PROT_READ | PROT_WRITE,
                 MAP_PRIVATE | MAP_ANONYMOUS, -1, 0);
if (buf == MAP_FAILED)
    err(1, "mmap");

/* Tag the page with our key. mprotect flags unchanged. */
if (pkey_mprotect(buf, 4096, PROT_READ | PROT_WRITE, pk) == -1)
    err(1, "pkey_mprotect");

/* Any access here faults. SIGFAULT, not a polite error. */

/* Unlock: one WRPKRU instruction, no syscall. */
pkey_set(pk, 0);
memcpy(buf, key_material, key_len);
pkey_set(pk, PKEY_DISABLE_ACCESS);

/* Locked again. Other threads were never affected. */
pkey_free(pk);

pkey_set() is a glibc wrapper around WRPKRU. If you are on a bare toolchain, the intrinsic is _wrpkru_u32() from <immintrin.h> or just inline assembly.

Where it bites you

Signal handlers. PKRU is per-thread and it does not change on signal delivery. If your signal handler touches a domain that was disabled at the interrupted point, it faults. Fix: read and save PKRU at handler entry with pkey_get(), set it fully open for the keys you need, restore before return. Most people forget this until they ship a crash to production.

exec(2). PKRU resets to all-enabled on exec. Fine. The problem is fork followed by exec: the child inherits the parent’s locked PKRU state before exec resets it. If the child runs library code between fork and exec that touches a locked domain, it dies. The real fix is to unlock everything in the child before exec, or use close_range(2) discipline and do the fork/exec atomically enough that no library call runs between them.

x86-only. AMD added PKU support in Zen 1. ARM has a related mechanism in MTE (Memory Tagging Extension, ARMv8.5) but it does different things and is not compatible. If your binary ships on both architectures, you need a runtime check: pkey_alloc(0, 0) returns -1 with ENOSYS on kernels or CPUs that lack PKU. FreeBSD has no equivalent in base as of 14.2, so if you are serious about portable isolation, look at landlock (Linux 5.13, no x86 dependency) or seccomp-bpf for syscall-level restriction.

16 keys. You cannot partition 50 shared libraries into separate domains. Design around this early. The common pattern is coarse-grained: one key for JIT code pages, one for secret-key material, one for attacker-controlled input buffers. Three keys is workable; fifteen is a red flag that the architecture needs rethinking.

Who actually uses this

Firefox’s SpiderMonkey JIT uses PKU to enforce W^X on code pages without a syscall on every JIT transition. OpenSSL 3.x has a prototype that wraps private key buffers in a pkey domain and gates access to them through the EVP layer. If you want kernel-level secret memory with no userspace access at all (not even your own code), memfd_secret is the other tool in that drawer: it unmaps the pages from the kernel’s own linear map, so only your process sees them, and even a kernel exploit cannot read the contents without physical memory access.

MPK is genuinely good engineering: hardware enforcement, per-thread granularity, sub-nanosecond transitions. The x86-only limitation is annoying but not unusual for security features that started as Intel proposals. Check for ENOSYS, isolate your initialization code, and handle the signal handler footgun on day one.