zfs snapshot is per dataset. bectl is per boot environment. Neither helps when the thing you are about to break is the pool itself: a zpool upgrade, a pile of dataset property changes, a destroy you are only 90 percent sure about. For that there is zpool checkpoint, which most people on OpenZFS have never typed.
What it does
zpool checkpoint tank
That records the entire pool state at that instant: every dataset, every property, the pool features. One checkpoint per pool. If the next hour goes badly, you export the pool and import it back to the checkpoint:
zpool export tank
zpool import --rewind-to-checkpoint tank
Everything written after the checkpoint is gone. Not the dataset you were working on, all of it. Datasets created since vanish, destroyed ones come back, and feature flags enabled by zpool upgrade are un-enabled. That last one is the reason I use it: an upgrade that turns on a feature flag is otherwise a one-way door, and the checkpoint is a way back through it.
If you decide the change was fine, throw the checkpoint away:
zpool checkpoint -d tank
Look before you rewind
You can import the checkpointed state read-only to see what you would be going back to, without committing:
zpool export tank
zpool import -o readonly=on --rewind-to-checkpoint tank
Poke around, then export and import normally to carry on with the current state, or rewind for real. Do this when the question is “did I really lose anything after that point?” rather than guessing.
The trap: it holds space
A checkpoint pins every block that existed when you took it. Blocks you free afterwards cannot be reused, because the rewind needs them. So the pool fills up faster than the data suggests, and zpool status reports the checkpoint’s space so you can see the cost:
zpool status tank
A checkpoint is meant to be short-lived. Take it, do the work, verify, discard. The ones that hurt are left sitting for weeks on a busy pool, quietly eating free space until writes start failing, which is a nasty way to rediscover a disk that is full while du swears it is not. Put a calendar reminder on it if you must.
What it blocks
While a checkpoint exists, some pool-shape operations are refused: removing top-level vdevs, attaching or detaching mirror members, splitting a pool, and changing the pool GUID. Read the zpool-checkpoint(8) page on your release for the exact list, since it has shifted between versions and I am not going to tell you something from memory that the man page says differently. Practically: do not plan to swap a disk layout while holding a checkpoint. Replacing a failed disk in place is a different operation, but test that on a scratch pool first. A file-backed one from mdconfig is perfect for it.
How it differs from the things you already use
- Snapshot: one dataset (or a recursive tree), cheap, keeps as long as you like, selectively restorable. A checkpoint is the opposite on every axis.
- Boot environment: rolls back the OS dataset, leaves your data alone, which is usually what you want for a bad
pkg upgrade. A checkpoint also rewinds your data, which is usually what you do not want, unless the damage is in the pool structure. - Replicated copy: survives the pool dying. A checkpoint dies with it. This is not a backup. It is a seatbelt.
When I reach for it
Before zpool upgrade on a box I cannot easily reinstall. Before a scripted reshuffle of many datasets at once. Before an experiment I will regret by lunch. Then the routine is checkpoint, change, verify, discard, in that order, and the discard step is the one people forget.
I thought it was a gimmick until I enabled a feature flag on a pool, found the boot loader on the other machine could not read it, and went back through the door in about two minutes. More ZFS in the ZFS and storage hub, and the rest of the FreeBSD material in the FreeBSD and OpenBSD hub.
