io_uring: why Jens Axboe's ring buffer is annoying to disagree with

Jens Axboe has been doing Linux block layer work since before most of you had a Linux box. He wrote the elevator I/O scheduler, blk-mq, fio, and a handful of other things you use without knowing you use them. In 2019 he merged io_uring into Linux 5.1 and the story he told was: every existing async I/O interface in Linux is broken in some way, and here’s the one that isn’t.

He was right about the first part. aio(7) only works for O_DIRECT files in practice. Passing work to a thread pool via libaio is async in the same way that mailing a letter is async. POSIX AIO is implemented in glibc via threads and is a lie. epoll is synchronous: you’re still blocking in epoll_wait and issuing the actual I/O in a callback.

What Axboe built instead: two ring buffers in shared memory between userspace and the kernel, an SQ for submissions and a CQ for completions, with head and tail pointers that each side advances independently. Userspace writes SQEs, the kernel reads them, executes the I/O, and writes CQEs. No lock contention between the two sides if you’re single-producer single-consumer. With SQPOLL mode, no syscall at all after the initial setup.

I watched three years of CVEs come out of this thing and prepared my “I told you so” speech. I still can’t deliver it. The design is right.

The ring buffer model

io_uring_setup(entries, params) is the only mandatory syscall. It returns a file descriptor. You mmap three regions off that fd:

• IORING_OFF_SQ_RING: the submission queue ring (contains head, tail, and the index array)
• IORING_OFF_CQ_RING: the completion queue ring (contains head, tail, and CQE array)
• IORING_OFF_SQES: the actual SQE array (sized separately from the ring)

The SQ ring contains an index array that maps into the SQE array. This indirection means you can fill SQEs in any order and reorder them in the index array before advancing the tail, useful for batch-building. The CQ ring is simpler: the kernel writes CQEs directly into a flat array and advances the tail; you read from the head and advance it.

The invariant: whoever owns the tail writes it atomically and releases. The other side reads the tail with an acquire barrier before looking at the entries behind it. No mutex, no futex, no wakeup unless you explicitly ask. Two cache lines bouncing between two cores is your only coordination cost, and with SQPOLL, even that goes away.

liburing: don’t do the mmap math yourself

The raw interface is about 200 lines of memory-barrier arithmetic you will get wrong on ARM. Use liburing. Install liburing-dev or liburing-devel.

#include <liburing.h>
#include <fcntl.h>
#include <stdio.h>
#include <string.h>

int main(void) {
    struct io_uring ring;
    struct io_uring_sqe *sqe;
    struct io_uring_cqe *cqe;
    char buf[4096];
    int fd, ret;

    /* Initialize ring with 32 SQE slots */
    ret = io_uring_queue_init(32, &ring, 0);
    if (ret < 0) {
        fprintf(stderr, "io_uring_queue_init: %s\n", strerror(-ret));
        return 1;
    }

    fd = open("/etc/os-release", O_RDONLY);

    /* Grab a submission slot */
    sqe = io_uring_get_sqe(&ring);
    io_uring_prep_read(sqe, fd, buf, sizeof(buf) - 1, 0);

    /* Tag this SQE with a user pointer for matching in the CQE */
    io_uring_sqe_set_data(sqe, (void *)(uintptr_t)fd);

    /* Submit: calls io_uring_enter() under the hood */
    io_uring_submit(&ring);

    /* Block until one CQE appears */
    ret = io_uring_wait_cqe(&ring, &cqe);
    if (ret == 0 && cqe->res > 0) {
        buf[cqe->res] = '\0';
        printf("read %d bytes\n", cqe->res);
    }

    /* Advance the CQ head: kernel can reuse this slot */
    io_uring_cqe_seen(&ring, cqe);

    io_uring_queue_exit(&ring);
    return 0;
}

Compile: gcc -o uring_read uring_read.c -luring. The cqe->res field is the return value of the underlying operation: bytes read, or a negative errno. Same sign convention as the syscall, no surprises.

The call to io_uring_sqe_set_data(sqe, ptr) sets the user_data field in the SQE, which the kernel copies verbatim into the matching CQE. You use this to identify which request completed when you’re running many in-flight. A common pattern is to store a pointer to a per-request context struct there.

SQPOLL: removing the submit syscall

The real performance story is IORING_SETUP_SQPOLL. Pass it to io_uring_queue_init_params():

struct io_uring_params params = {};
params.flags = IORING_SETUP_SQPOLL;
params.sq_thread_idle = 2000;  /* park the polling thread after 2s idle */

ret = io_uring_queue_init_params(128, &ring, &params);

With SQPOLL, the kernel spawns a thread that spins on the SQ tail pointer. To submit, userspace writes SQEs and advances the tail: no syscall, no io_uring_enter(). The kernel polling thread sees the new tail value and picks up the work.

After sq_thread_idle milliseconds of idle time, the thread parks. Submitting to a parked ring requires a wakeup: io_uring_sqe_set_flags(sqe, IOSQE_ASYNC) doesn’t do it, you need to call io_uring_enter() with IORING_ENTER_SQ_WAKEUP. liburing’s io_uring_submit() handles this transparently via the IORING_SQ_NEED_WAKEUP flag in the SQ ring head.

SQPOLL historically required CAP_SYS_NICE. Since Linux 5.13 (with the unprivileged SQPOLL feature flag IORING_FEAT_SQPOLL_NONFIXED), it’s available to unprivileged users. Check params.features after init to confirm the kernel you’re on supports it.

The benchmark numbers on NVMe with io_uring SQPOLL are real. Axboe’s own fio numbers show the syscall overhead is measurable at high IOPS, and eliminating it is not a theoretical win: it’s 5–15% wall time on fast storage at scale. If you’re running a storage-intensive service and you’re still using blocking reads in a thread pool, you’re leaving performance on the floor because the API was scary.

Fixed file registration

Every read() or write() syscall starts with a table walk to convert the file descriptor integer to a struct file *. In a tight I/O loop, this walk happens on every operation. io_uring lets you register a table of file descriptors once and reference entries by index instead:

/* Register an array of fds */
int fds[4] = { fd0, fd1, fd2, fd3 };
io_uring_register_files(&ring, fds, 4);

/* Reference by index instead of fd number */
sqe = io_uring_get_sqe(&ring);
io_uring_prep_read_fixed(sqe, 0, buf, sizeof(buf), 0, 0);
/*                            ^-- index into registered table, not fd */

Same idea applies to buffers with io_uring_register_buffers(): pre-pin the userspace buffers with the kernel so the copy path skips the per-operation get_user_pages(). For a service doing sustained sequential I/O on a small set of files and buffers, fixed registration is the mechanical difference between “this is fast” and “this is as fast as it can be.”

The CVE situation, accurately described

Docker disabled io_uring in its default seccomp profile in 2022. By 2023, it had accumulated enough kernel escapes (CVE-2022-0185, CVE-2022-29582, CVE-2023-2598, CVE-2023-21400, and more) that the seccomp-BPF and container communities had a simple answer: block io_uring in untrusted workloads.

This is the correct response and it doesn’t mean the design is wrong. The attack surface is real: io_uring exposes a new code path into the kernel for async operations, and more code means more bugs. The proofs-of-concept have been impressive: full ring0 from userspace, repeated, across multiple stable kernels. That’s not “a few bugs.” It’s a pattern.

The working answer is: use io_uring in trusted server workloads where you control the software and the kernel version. Do not expose io_uring to untrusted code: sandbox it with seccomp and block IORING_OP_* ops you don’t need, or block the io_uring_setup syscall entirely for processes that don’t need it. The distinction between “trusted high-performance service” and “untrusted container workload” is the actual split, and treating both the same in either direction is the mistake.

Run grep -r io_uring /etc/docker/ if you want to see whether your container runtime has made this call for you already.

Checking kernel support and feature flags

The feature flags in params.features tell you what your kernel actually supports:

# Quick check: does this kernel have io_uring at all?
ls /proc/sys/kernel/io_uring_disabled 2>/dev/null && cat /proc/sys/kernel/io_uring_disabled

# 0 = enabled, 1 = disabled for unprivileged, 2 = disabled entirely
# io_uring_disabled was added in 5.15 as an operator escape hatch after the CVE pile

In C, after io_uring_queue_init_params():

if (params.features & IORING_FEAT_FAST_POLL)
    printf("kernel supports fast poll (5.7+)\n");
if (params.features & IORING_FEAT_SQPOLL_NONFIXED)
    printf("SQPOLL available without CAP_SYS_NICE (5.13+)\n");
if (params.features & IORING_FEAT_LINKED_FILE)
    printf("linked operations share file table (5.6+)\n");

IORING_FEAT_FAST_POLL (5.7+) is the one that makes accept, connect, recv, and send worth using in io_uring; before it, those operations would sometimes block the submission, defeating the point.

Benchmarking: is your workload actually I/O-bound?

Before reaching for io_uring, confirm your bottleneck is I/O submission overhead, not seek latency or throughput. perf stat tells you fast:

perf stat -e syscalls:sys_enter_read,syscalls:sys_enter_write,syscalls:sys_enter_epoll_wait ./myservice

If your read/write event counts are in the millions per second, io_uring’s overhead elimination is meaningful. If they’re in the thousands, your problem is elsewhere and io_uring is complexity for its own sake. See perf stat basics for the full picture, and bpftrace I/O latency for per-operation breakdown when you need to know where the latency is, not just how many syscalls you’re making.

---

The kernel source under io_uring/ is readable if you have the stomach for it; Axboe keeps it better commented than most subsystems. The io_uring manpage is incomplete but improving; the most current reference is Documentation/block/io-uring.rst in the kernel tree. Gregg’s BPF Performance Tools doesn’t cover io_uring directly (predates its maturity) but gives you the tracing vocabulary to observe it. Kerrisk’s The Linux Programming Interface is still the reference for the syscall model io_uring is replacing, and it’s worth having both.