sync_file_range(2): writeback without the fsync stall

The thing nobody tells you about fsync is that it stalls the calling thread until the kernel finishes pushing every dirty page for that file to stable storage and updates the inode metadata. On a busy WAL writer or a log-structured store, that stall budget is how you kill throughput. sync_file_range(2) is the syscall that lets you get out of that hole without giving up safety guarantees you were never actually relying on.

What it does

#define _GNU_SOURCE
#include <fcntl.h>

int sync_file_range(int fd, off64_t offset, off64_t nbytes, unsigned int flags);

It operates on a byte range of a file, not the whole file. nbytes = 0 means “from offset to end of file.” The flags field is where the control lives:

• SYNC_FILE_RANGE_WRITE (0x2): submit writeback I/O for dirty pages in the range. Returns immediately. The kernel queues the I/O; you don’t wait.
• SYNC_FILE_RANGE_WAIT_AFTER (0x4): wait for in-flight I/O covering the range to complete. Pair with a prior SYNC_FILE_RANGE_WRITE call.
• SYNC_FILE_RANGE_WAIT_BEFORE (0x1): wait for any in-flight writeback that was already running before you called SYNC_FILE_RANGE_WRITE. Mostly relevant when multiple threads write to overlapping ranges.

You can combine them. The common two-phase pattern:

// Phase 1: kick off async writeback for the range you just wrote.
// Returns immediately.
sync_file_range(fd, prev_offset, prev_len, SYNC_FILE_RANGE_WRITE);

// ... keep writing the next chunk ...

// Phase 2: wait for the previous range to land before you recycle the buffer
// or advance your checkpoint.
sync_file_range(fd, prev_offset, prev_len,
    SYNC_FILE_RANGE_WAIT_BEFORE | SYNC_FILE_RANGE_WRITE | SYNC_FILE_RANGE_WAIT_AFTER);

Phase 1 and phase 2 together: you overlap I/O and compute. The kernel is flushing chunk N while you write chunk N+1.

What it does not do

This is the part that bites people. sync_file_range does not:

• Flush inode metadata (mtime, file size). A crash after sync_file_range can still lose the file’s existence in its parent directory.
• Call into the journal. On ext4 or XFS, the journal is not involved. Journaled metadata can be lost.
• Give you any durability guarantee. The POSIX definition of “synchronized I/O” is not what this syscall promises.

If you need crash-safe data you must still call fdatasync at your checkpoint boundary. sync_file_range is not a cheaper fdatasync. It is a cheaper nothing: you were going to write and then stall on fsync at checkpoint time anyway, and this trades that one big stall for a series of smaller async pushes plus one final fdatasync.

Glibc will not give you this as a wrapper: _GNU_SOURCE and a direct syscall or the sync_file_range glibc extension is how you get it. It has been in the kernel since 2.6.17 (2006). It is Linux-only: FreeBSD, OpenBSD, and macOS have no equivalent.

The PostgreSQL pattern

PostgreSQL’s pg_flush_data() in src/backend/storage/file/fd.c does exactly this. When the WAL writer finishes a segment it calls sync_file_range with SYNC_FILE_RANGE_WRITE to kick off async writeback, continues to the next segment, then before it needs the buffer again it issues the wait form. The fdatasync (or fsync, depending on wal_sync_method) comes only at the transaction commit barrier, not for every write.

The result: on a fast sequential workload the I/O and the CPU pipeline largely overlap and you pay the blocking fdatasync once per commit instead of once per write. For workloads with large sequential writes, like WAL, this matters. For random small writes it mostly does not. For random small writes you need a different approach, like io_uring with IOSQE_IO_DRAIN if you want fine-grained ordering without stalls.

Pitfalls

The biggest one: forgetting that writeback errors are deferred. If SYNC_FILE_RANGE_WRITE queues I/O that fails, the error surfaces on a later call, either the wait form or a subsequent write. Check return values on the wait form carefully, and check write return values too. This is not a theoretical concern: a disk that starts returning errors will look fine through the async path until the wait form comes back with EIO.

The second: nbytes = 0 is a footgun. It means “to end of file at the time of the call,” but file size can change between the write and the sync call. Use explicit lengths.

Availability

Linux 2.6.17+. _GNU_SOURCE required. Header: <fcntl.h>. Not in POSIX, not on BSDs. For existing tips on related fd tricks see the file descriptor series: preadv2 with RWF_NOWAIT and copy_file_range both interact with the same page cache machinery this syscall touches. The fallocate tip covers the other half of the pre-write story: reserve space before you write, then let sync_file_range manage writeback.