$RodHat_/search
Console Tips

fallocate(2): preallocate extents and punch holes without writing zeros

Published by · Updated

This article may contain Amazon affiliate links. I may earn a commission if you buy through them. Disclosure.

Editorial card: fallocate(2): preallocate extents and punch holes without writing zeros
Editorial card: RodHat
On this page · 4 sections
  1. Preallocate without lying about size
  2. Punch holes
  3. Collapse and insert
  4. The ZFS caveat

Someone once asked me why their database migration was burning an hour preallocating a 500GB tablespace. They were calling posix_fallocate(3). On ext4. The call returned zero, success, and wrote 500 gigabytes of zeros to get there. glibc’s POSIX wrapper falls back to write(2) in a loop when the underlying filesystem reports it doesn’t support native preallocation. Some filesystems claim support and some don’t, and glibc doesn’t tell you which path it took. The application measured success and sat there writing zeros to disk for an hour.

Use the syscall.

#include <fcntl.h>
#include <linux/falloc.h>
#include <sys/syscall.h>
#include <unistd.h>

/* allocate len bytes at offset, extending st_size if needed */
int r = fallocate(fd, 0, offset, len);

With mode set to zero, fallocate(2) tells the filesystem to commit the blocks now. The file’s reported size includes them. The blocks are not zeroed until first write, but they are allocated: a subsequent write at any offset within that range will not hit ENOSPC. On ext4 with delayed allocation enabled, this is the actual mechanism that prevents ENOSPC surprises on heavy write workloads. If the filesystem doesn’t support it (tmpfs, old kernels, some network filesystems), you get EOPNOTSUPP and you know.

Preallocate without lying about size

FALLOC_FL_KEEP_SIZE preallocates extents but leaves st_size unchanged. Reads past the original EOF still return 0 bytes. The application can call write(2) sequentially and get contiguous extents, no fragmentation, no reallocation cost:

/* grab 1 GiB of contiguous extents, don't touch st_size */
fallocate(fd, FALLOC_FL_KEEP_SIZE, 0, 1 << 30);

Postgres uses this. MySQL uses this. When you can afford to know the final size up front, reserving space this way sidesteps the extent tree reshuffling that happens with append-only growth on a loaded filesystem.

Punch holes

This is the one most people don’t know. FALLOC_FL_PUNCH_HOLE | FALLOC_FL_KEEP_SIZE zeros a byte range and frees the underlying blocks. The file still exists, st_size is unchanged, reads return zeros. The blocks are gone:

/* free 128 MiB at offset 0, range reads as zeros */
fallocate(fd, FALLOC_FL_PUNCH_HOLE | FALLOC_FL_KEEP_SIZE, 0, 128 << 20);

The filesystem rounds the range to block boundaries, so byte-level alignment is not required, but punching a range smaller than a block frees nothing. Check st_blocks before and after via statx(2) if you need to verify. Useful for WAL recycling, log compaction, removing obsolete journal entries from a file you cannot truncate in the middle.

From the shell, util-linux ships fallocate(1):

fallocate --punch-hole --offset 0 --length 128M logfile.dat

The offset and length flags accept human-readable sizes. Requires a writable fd; open the file with O_RDWR.

Collapse and insert

FALLOC_FL_COLLAPSE_RANGE physically removes a byte range, shifting everything after it. The file gets shorter. Useful for journal-style structures where old records at the head are done and you want the space back without copy-plus-truncate:

/* remove 64 MiB from offset 0, shift remaining data down */
fallocate(fd, FALLOC_FL_COLLAPSE_RANGE, 0, 64 << 20);

The inverse, FALLOC_FL_INSERT_RANGE (Linux 4.1+), inserts a hole at an offset, shifting data after it forward. I have never needed this one in production, which may say something about the designs I work with.

The ZFS caveat

ZFS on Linux ignores fallocate(2) preallocation. The call returns success but commits nothing. CoW doesn’t have a meaningful notion of “claim these blocks now.” Punch holes have historically had limited support. If you’re on ZFS and relying on fallocate for preallocation guarantees, you’re being lied to politely. Check FALLOC_FL_PUNCH_HOLE support against your specific OpenZFS version before depending on it in production code.

For the zero-copy data movement side of Linux I/O, copy_file_range(2) handles copying between file descriptors in kernel space. For reading and writing with per-call flags, preadv2(2) covers RWF_NOWAIT and RWF_HIPRI.

Sources

  1. fallocate(2) Linux man page (man7.org)
  2. fallocate(1) util-linux man page (man7.org)
  3. Punching holes in files (LWN.net, 2011) (LWN.net)