$RodHat_
Console Tips

fanotify(7): watch and block file opens at the VFS layer

Published by

fanotify(7): watch and block file opens at the VFS layer
Photo: AI-generated — no human photographer / RodHat AI Cover

Most people reach for inotify(7) when they want to watch files. inotify tells you what happened after it already happened. You get a notification that a file was opened; you had no say in it. That is fine if you are syncing a directory. It is not fine if you are doing access control.

The interface for that is fanotify(7), and it has been in the kernel since 2.6.36. It is not obscure because it is new. It is obscure because the useful modes require CAP_SYS_ADMIN and the API has more surface than inotify. The tradeoff is reasonable: you get an open file descriptor to the accessed file in every event, and with the right class you can block the open before the calling process’s syscall returns.

That is what file integrity monitors, endpoint detection agents, and antivirus daemons actually use. Not inotify, which gives you a name and a wave. fanotify, which puts you in the kernel’s path.

Two things inotify can’t do

Events carry a live fd, not a path. When fanotify reports that /home/rod/secrets was opened, it passes you an open file descriptor to that file. You can fstat(2) it, read it, or resolve it via /proc/self/fd/N. inotify gives you the filename in the watch directory. If the file was moved or hard-linked somewhere else, inotify gives you a wrong or missing name; fanotify gives you the actual file regardless of path.

Permission events let you block. FAN_OPEN_PERM and FAN_ACCESS_PERM freeze the calling process in the kernel until your daemon writes a decision back on the fanotify fd. FAN_ALLOW lets it through; FAN_DENY returns EPERM to the opener. The blocked process just waits. You have no time limit; take as long as you need to check a hash, consult a policy engine, or log to disk.

The four-step API

1. fanotify_init() creates the fanotify group and returns an fd.

2. fanotify_mark() attaches the group to a filesystem, mount, or inode.

3. A loop read()s events from the group fd; each event is a struct fanotify_event_metadata.

4. For permission events, write() a struct fanotify_response before closing the event fd.

fanotify_init: pick your class

int fan = fanotify_init(FAN_CLASS_NOTIF | FAN_REPORT_PID,
                        O_RDONLY | O_LARGEFILE);

Three classes exist and you pick one at init time:

  • FAN_CLASS_NOTIF: notification after the fact. This is the inotify-alike. Events arrive after the operation completed. No blocking.
  • FAN_CLASS_CONTENT: intercepts after the file has been written but before the fd is released. Used for AV scanning: file is fully written, you can read it, then release.
  • FAN_CLASS_PRE_CONTENT: intercepts before access completes. The class for permission events. If you need FAN_OPEN_PERM, this is what you initialize with.

The second argument sets flags on the per-event file descriptors the kernel opens for you. O_LARGEFILE is not optional if you run on a 32-bit userspace or have large files; skip it and you get EOVERFLOW on anything over 2 GB.

fanotify_mark: pick your scope

fanotify_mark(fan,
              FAN_MARK_ADD | FAN_MARK_FILESYSTEM,
              FAN_OPEN | FAN_CLOSE_WRITE,
              AT_FDCWD, "/home");

FAN_MARK_FILESYSTEM, added in Linux 5.1, marks the filesystem object itself rather than a specific mount point. Before 5.1, FAN_MARK_MOUNT was the widest scope and you had to re-mark every bind mount separately. With FAN_MARK_FILESYSTEM, new bind mounts inside the filesystem, snapshots, anything that resolves to the same underlying device all fall under the watch automatically. For whole-system monitoring, this is the feature that makes fanotify practical without a separate mount-tracking daemon.

The event mask is a bitmask of what you care about:

  • FAN_OPEN: any open
  • FAN_ACCESS: any read
  • FAN_MODIFY: any write
  • FAN_CLOSE_WRITE: writable fd closed
  • FAN_CLOSE_NOWRITE: read-only fd closed
  • FAN_OPEN_PERM: open you can block (requires FAN_CLASS_PRE_CONTENT)
  • FAN_ACCESS_PERM: read you can block

The event loop

#define _GNU_SOURCE
#include <sys/fanotify.h>
#include <fcntl.h>
#include <stdio.h>
#include <unistd.h>
#include <errno.h>

static void print_path(int fd)
{
    char link[64], path[4096];
    snprintf(link, sizeof(link), "/proc/self/fd/%d", fd);
    ssize_t n = readlink(link, path, sizeof(path) - 1);
    if (n < 0) return;
    path[n] = '\0';
    fprintf(stderr, "  path: %s\n", path);
}

int main(void)
{
    int fan = fanotify_init(FAN_CLASS_NOTIF | FAN_REPORT_PID,
                            O_RDONLY | O_LARGEFILE);
    if (fan < 0) { perror("fanotify_init"); return 1; }

    if (fanotify_mark(fan,
                      FAN_MARK_ADD | FAN_MARK_FILESYSTEM,
                      FAN_OPEN | FAN_CLOSE_WRITE,
                      AT_FDCWD, "/home") < 0) {
        perror("fanotify_mark"); return 1;
    }

    char buf[4096];
    for (;;) {
        ssize_t n = read(fan, buf, sizeof(buf));
        if (n < 0 && errno == EINTR) continue;
        if (n < 0) { perror("read"); break; }

        const struct fanotify_event_metadata *ev =
            (const struct fanotify_event_metadata *)buf;

        while (FAN_EVENT_OK(ev, n)) {
            if (ev->vers != FANOTIFY_METADATA_VERSION) {
                fprintf(stderr, "metadata version mismatch\n");
                break;
            }
            fprintf(stderr, "pid=%d mask=%llx\n",
                    ev->pid,
                    (unsigned long long)ev->mask);

            if (ev->fd != FAN_NOFD) {
                print_path(ev->fd);
                close(ev->fd);  /* mandatory: see below */
            }
            ev = FAN_EVENT_NEXT(ev, n);
        }
    }
    return 0;
}

Three things that eat people here:

One. read() can return multiple events in a single call. FAN_EVENT_OK(ev, n) checks that a full event fits in the remaining buffer; FAN_EVENT_NEXT(ev, n) steps past ev->event_len bytes to the next one. Not using these macros and treating the buffer as a single event is wrong.

Two. Every event fd must be close()d. The kernel holds the file open on your behalf in the event. If you do not close it, you bleed open fds at whatever rate files are accessed on the watched filesystem. On a busy box this is fast.

Three. ev->fd is FAN_NOFD for queue-overflow events (FAN_Q_OVERFLOW). Check before using it. Overflow means your event loop is slower than the event rate; the kernel dropped events rather than block. You will need to handle that case.

Permission events

For blocking, initialize with FAN_CLASS_PRE_CONTENT and add FAN_OPEN_PERM to the mark mask:

int fan = fanotify_init(FAN_CLASS_PRE_CONTENT | FAN_REPORT_PID,
                        O_RDONLY | O_LARGEFILE);

fanotify_mark(fan,
              FAN_MARK_ADD | FAN_MARK_FILESYSTEM,
              FAN_OPEN_PERM,
              AT_FDCWD, "/sensitive");

In the event loop, after inspecting the file, write back your decision before closing the event fd:

struct fanotify_response resp = {
    .fd       = ev->fd,
    .response = FAN_ALLOW,  /* or FAN_DENY */
};
write(fan, &resp, sizeof(resp));
close(ev->fd);

The process that triggered the open is sitting in the kernel waiting for that write. FAN_DENY hands it EPERM. There is no timeout; the blocked thread waits indefinitely.

One deadlock to know about: if your handler opens files that are on a marked filesystem, your own opens queue behind the event you have not yet responded to, and you are stuck. The cleanest fix is to check ev->pid == getpid() and immediately allow your own events without inspecting them. Alternatively, watch inodes or specific directories rather than the whole filesystem if you only care about a narrow path.

What 5.9 added

Linux 5.9 added FAN_REPORT_NAME and FAN_REPORT_DFID_NAME. With these flags, events include a directory fd and the filename, not just the accessed file fd. The metadata structure gains a variable-length tail with the name. More overhead, but you get the full path directly without chasing /proc/self/fd/N symlinks and dealing with deleted-file edge cases. If you are logging for audit trails rather than blocking, this is worth the cost.

Privilege

FAN_CLASS_NOTIF with FAN_MARK_INODE is reachable without CAP_SYS_ADMIN on 5.0+ under certain configurations. In practice, anything useful: FAN_MARK_MOUNT, FAN_MARK_FILESYSTEM, FAN_CLASS_CONTENT, FAN_CLASS_PRE_CONTENT — all require CAP_SYS_ADMIN. Permission-blocking events require it without question.

This is not an accident. The permission-blocking path sits in the VFS between the syscall and the file. An unprivileged process that can block any other process’s file opens indefinitely is a denial-of-service with no cap. The capability requirement is the correct gate.

If you want to sandbox what a process can open in the first place rather than monitor after the fact, landlock(2) does that without root. If you are watching for policy violations at the syscall level rather than the file level, seccomp BPF is the right tool. The openat2(2) post covers the resolver flags that control how paths are traversed, which matters if you are building path-based policy on top of fanotify marks.