Every blog post about vmsplice opens with “zero-copy.” Most of them are telling you about the best case and skipping what it takes to get there. The best case is real, but you have to earn it.
The splice family
splice(2) moves data between two file descriptors through a kernel pipe buffer without it passing through userspace. tee(2) forks a pipe into two. Both only work kernel-side: the data has to be in a pipe already.
vmsplice(2) is the on-ramp. It takes your user buffers and puts them into a pipe, at which point splice() can take over and route them wherever.
#include <fcntl.h>
#include <sys/uio.h>
ssize_t vmsplice(int fd, const struct iovec *iov,
size_t nr_segs, unsigned int flags);fd must be the write end of a pipe. iov is an array of { iov_base, iov_len } pairs describing your buffers. flags is where the interesting part lives.
SPLICE_F_GIFT: the thing everyone forgets to mention
Without SPLICE_F_GIFT, the kernel copies your data into the pipe buffer anyway. You wrote vmsplice instead of write() and got identical throughput. Congrats.
With SPLICE_F_GIFT, the kernel takes ownership of those pages. It maps them directly into the pipe buffer without copying. Your buffers become kernel pages and you do not get them back.
The man page says “the caller must not modify this memory ever again after the call.” Rod translation: you just mailed away your only copy. Writing to those pages after a GIFT call is undefined behavior. On a good day you get silent data corruption. The kernel is now the owner of that memory.
Second catch: the buffers must be page-aligned and their lengths must be multiples of the page size. If they are not, the kernel silently ignores GIFT and does a copy anyway. It succeeds, returns the byte count, says nothing. You believe you are zero-copy and you are not.
Page-aligned allocation means mmap(NULL, len, PROT_READ|PROT_WRITE, MAP_ANON|MAP_PRIVATE, -1, 0) or posix_memalign(&buf, getpagesize(), len). Stack buffers, malloc() returns, and char[4096] declarations are all wrong unless you get lucky with alignment.
The actual zero-copy pattern
#include <fcntl.h>
#include <sys/mman.h>
#include <sys/uio.h>
#include <unistd.h>
/*
* Write len bytes from buf to out_fd without a copy.
* buf must be page-aligned and len a multiple of PAGE_SIZE.
* Caller must not touch buf after this returns successfully.
*/
ssize_t splice_write(int out_fd, void *buf, size_t len)
{
int pfd[2];
struct iovec iov = { .iov_base = buf, .iov_len = len };
ssize_t n;
if (pipe(pfd) < 0)
return -1;
n = vmsplice(pfd[1], &iov, 1, SPLICE_F_GIFT);
if (n > 0)
n = splice(pfd[0], NULL, out_fd, NULL, (size_t)n, SPLICE_F_MOVE);
close(pfd[0]);
close(pfd[1]);
return n;
}The SPLICE_F_MOVE on the splice() call is a hint to move rather than copy kernel pages to out_fd. It has been a no-op since Linux 2.6.21 for most cases (the kernel just ignores it now), but there is no cost to passing it and the intent is clear.
For sustained throughput, create the pipe once outside the loop and resize it with fcntl(pfd[1], F_SETPIPE_SZ, desired_size). A 4 MB pipe (F_SETPIPE_SZ, 4194304) can hold more data in flight before you need to drain it with splice().
Where this actually helps
The gain is real on high-throughput log pipelines or any write path where you build up a large, page-aligned buffer, hand it to the kernel, and stream it to a socket or file. At sustained 10 Gb/s+ with multi-megabyte writes, skipping a kernel memcpy matters. At 1 Gb/s with 4 KB records, you will not measure it.
The crossover is roughly 32 KB per call. Below that, the pipe machinery costs more than the copy saves. Use writev(2) for small, scatter/gather writes. See the tip on splice and tee for the kernel-to-kernel side of this, and the file descriptor tricks hub for the broader family.
FreeBSD and the alternatives
FreeBSD has no vmsplice. For the user-memory-to-socket path, sendfile(2) covers file-to-socket zero-copy and has since FreeBSD 3.1 in 1998. For user-memory-to-anything, you are writing with write() or writev() and the kernel is copying. The splice pipeline is a Linux-specific optimization that FreeBSD never needed to add because sendfile() covers the common case and jails kept people from building the kind of single-box write-multiplexers where vmsplice shines.
Version and availability
Linux 2.6.17 (2006). glibc wrapper since glibc 2.5; #include <fcntl.h> for the SPLICE_F_* constants and #include <sys/uio.h> for struct iovec. No equivalent in POSIX, no equivalent in BSD.