> For the complete documentation index, see [llms.txt](https://book.bsdcn.org/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://book.bsdcn.org/ask/flat/chapter-28-the-zfs-file-system/di-28.5-jie-zfs-guan-li.md). # 28.5 ZFS Administration ## Creating and Destroying Datasets Unlike traditional disk and volume managers, ZFS does not pre-allocate space. Traditional file systems cannot create new file systems after partitioning and allocating space without adding new disks. ZFS allows creating new file systems at any time. Each **dataset** possesses features such as compression, deduplication, caching, and quotas, along with practical attributes like read-only, case sensitivity, network file sharing, and mount points. Datasets can be nested, and child datasets inherit properties from their parent datasets. Each dataset can be delegated for management, replicated, snapshotted, included in a jail, or directly destroyed. Creating a separate dataset for each file type or file collection is a recommended practice. However, having too many datasets also has drawbacks: commands like `zfs list` will slow down, and mounting hundreds or even thousands of datasets will also slow down FreeBSD boot time. View the current datasets: ```sh # zfs list NAME USED AVAIL REFER MOUNTPOINT zroot 916M 49.0G 96K /zroot zroot/ROOT 886M 49.0G 96K none zroot/ROOT/default 886M 49.0G 886M / zroot/home 27.8M 49.0G 96K /home zroot/home/ykla 27.7M 49.0G 27.7M /home/ykla zroot/tmp 128K 49.0G 128K /tmp zroot/usr 288K 49.0G 96K /usr zroot/usr/ports 96K 49.0G 96K /usr/ports zroot/usr/src 96K 49.0G 96K /usr/src zroot/var 636K 49.0G 96K /var zroot/var/audit 96K 49.0G 96K /var/audit zroot/var/crash 96K 49.0G 96K /var/crash zroot/var/log 156K 49.0G 156K /var/log zroot/var/mail 96K 49.0G 96K /var/mail zroot/var/tmp 96K 49.0G 96K /var/tmp ``` > **Tip** > > The username `ykla` and path `/home/ykla` appearing in the examples in this section are for demonstration purposes. Please replace them with your actual username and home directory. Create a new dataset and enable LZ4 compression: ```sh # zfs create -o compression=lz4 zroot/usr/mydataset # zfs list NAME USED AVAIL REFER MOUNTPOINT zroot 916M 49.0G 96K /zroot zroot/ROOT 886M 49.0G 96K none zroot/ROOT/default 886M 49.0G 886M / zroot/home 27.8M 49.0G 96K /home zroot/home/ykla 27.7M 49.0G 27.7M /home/ykla zroot/tmp 128K 49.0G 128K /tmp zroot/usr 384K 49.0G 96K /usr zroot/usr/mydataset 96K 49.0G 96K /usr/mydataset # Note this line zroot/usr/ports 96K 49.0G 96K /usr/ports zroot/usr/src 96K 49.0G 96K /usr/src zroot/var 636K 49.0G 96K /var zroot/var/audit 96K 49.0G 96K /var/audit zroot/var/crash 96K 49.0G 96K /var/crash zroot/var/log 156K 49.0G 156K /var/log zroot/var/mail 96K 49.0G 96K /var/mail zroot/var/tmp 96K 49.0G 96K /var/tmp ``` Since this operation does not involve scanning files and updating corresponding metadata, destroying a dataset is much faster than deleting files within the dataset. > **Warning** > > `zfs destroy` will permanently delete the dataset and all its data, and this action cannot be undone. Please make sure the target is correct. If necessary, use `zfs destroy -n -v` first to preview the datasets and snapshots that will be destroyed. Destroy the created dataset: ```sh # zfs destroy zroot/usr/mydataset # zfs list NAME USED AVAIL REFER MOUNTPOINT zroot 916M 49.0G 96K /zroot zroot/ROOT 886M 49.0G 96K none zroot/ROOT/default 886M 49.0G 886M / zroot/home 27.8M 49.0G 96K /home zroot/home/ykla 27.7M 49.0G 27.7M /home/ykla zroot/tmp 128K 49.0G 128K /tmp zroot/usr 288K 49.0G 96K /usr zroot/usr/ports 96K 49.0G 96K /usr/ports zroot/usr/src 96K 49.0G 96K /usr/src zroot/var 636K 49.0G 96K /var zroot/var/audit 96K 49.0G 96K /var/audit zroot/var/crash 96K 49.0G 96K /var/crash zroot/var/log 156K 49.0G 156K /var/log zroot/var/mail 96K 49.0G 96K /var/mail zroot/var/tmp 96K 49.0G 96K /var/tmp ``` In modern versions of ZFS, `zfs destroy` is asynchronous, and the freed space may take a few minutes to be reflected in the pool. Use `zpool get freeing poolname` to check the `freeing` property: ```sh NAME PROPERTY VALUE SOURCE zroot freeing 0 - ``` This property shows which datasets are currently releasing their blocks in the background. If there are child datasets (such as snapshots or other datasets), the parent dataset cannot be directly destroyed. To destroy a dataset and all its child datasets, use `-r` for recursive destruction. Using `-n -v` will list the datasets and snapshots that would be destroyed without actually destroying any data. When destroying snapshots, the reclaimed space will also be displayed. ## Creating and Destroying Volumes A volume is a special type of dataset. It is not mounted as a file system, but is exposed as a block device at the path **/dev/zvol/poolname/dataset**. Therefore, volumes can be used for other file systems, serve as virtual machine disks, or be provided to other hosts on the network through protocols such as iSCSI and HAST. Volumes can be formatted and used with any file system, or they can store raw data without formatting. To the user, a volume is no different from a regular disk. Placing a regular file system on these *zvols* provides capabilities that regular disks or file systems do not have. For example, enabling the compression property on a 250 MB volume allows creating a compressed FAT file system. Create a 3 GB ZFS volume and enable compression: ```sh # zfs create -V 3G -o compression=on zroot/fat32 ``` > **Tip** > > If the FAT32 capacity is set too low, it will not be possible to format properly due to insufficient file clusters. Confirm the volume has been created and check its space usage: ```sh # zfs list zroot/fat32 NAME USED AVAIL REFER MOUNTPOINT zroot/fat32 3.05G 49.0G 56K - ``` The volume appears as a block device under the **/dev/zvol/** path: ```sh # ls -al /dev/zvol/zroot/fat32 crw-r----- 1 root operator 0x75 Apr 16 12:21 /dev/zvol/zroot/fat32 ``` > **Warning** > > `newfs_msdos` will format the specified ZFS volume, and existing data on the volume will be permanently lost. Please confirm the device path `/dev/zvol/` is correct to avoid accidentally formatting other devices. Format the volume as a FAT32 file system: ```sh # newfs_msdos -F32 /dev/zvol/zroot/fat32 newfs_msdos: cannot get number of sectors per track: Operation not supported newfs_msdos: cannot get number of heads: Operation not supported /dev/zvol/zroot/fat32: 6288320 sectors in 98255 FAT32 clusters (32768 bytes/cluster) BytesPerSec=512 SecPerClust=64 ResSectors=64 FATs=2 Media=0xf0 SecPerTrack=63 Heads=255 HiddenSecs=0 HugeSectors=6291456 FATsecs=768 RootCluster=2 FSInfo=1 Backup=2 ``` Mount the volume and confirm the mount succeeded: ```sh # mount -t msdosfs /dev/zvol/zroot/fat32 /mnt # mount | grep fat32 /dev/zvol/zroot/fat32 on /mnt (msdosfs, local) ``` Copy some files for testing: ```sh # cp /home/ykla/Philosophische-untersuchungen.pdf /mnt/ # df -h /mnt | grep fat32 /dev/zvol/zroot/fat32 3.0G 36M 3.0G 1% /mnt ``` Destroying a volume is similar to destroying a regular file system dataset: ```sh # zfs destroy zroot/fat32 ``` This operation completes almost instantly, but free space may take a few minutes to be reclaimed in the background. ## Renaming Datasets To change the name of a dataset, use the `zfs rename` command. To change the parent of a dataset, use the same command. Renaming a dataset to a different parent will change the property values it inherits from its parent. When renaming a dataset, it is unmounted and mounted at the new location (which is inherited from the new parent). To prevent this behavior, use the `-u` option. Rename a dataset and move it to a different parent: First, create a dataset for demonstration and confirm its location: ```sh # zfs create zroot/usr/mydataset # zfs list NAME USED AVAIL REFER MOUNTPOINT zroot 3.94G 46.0G 96K /zroot zroot/ROOT 886M 46.0G 96K none zroot/ROOT/default 886M 46.0G 886M / zroot/fat32 3.05G 49.0G 31.0M - zroot/home 27.8M 46.0G 96K /home zroot/home/ykla 27.7M 46.0G 27.7M /home/ykla zroot/tmp 128K 46.0G 128K /tmp zroot/usr 384K 46.0G 96K /usr zroot/usr/mydataset 96K 46.0G 96K /usr/mydataset # Note this line zroot/usr/ports 96K 46.0G 96K /usr/ports zroot/usr/src 96K 46.0G 96K /usr/src zroot/var 636K 46.0G 96K /var zroot/var/audit 96K 46.0G 96K /var/audit zroot/var/crash 96K 46.0G 96K /var/crash zroot/var/log 156K 46.0G 156K /var/log zroot/var/mail 96K 46.0G 96K /var/mail zroot/var/tmp 96K 46.0G 96K /var/tmp ``` Create a snapshot for the dataset to demonstrate that snapshots are preserved after renaming: ```sh # zfs snapshot zroot/usr/mydataset@01 # zfs list -t snapshot NAME USED AVAIL REFER MOUNTPOINT zroot/usr/mydataset@01 0B - 96K - ``` Execute the rename, moving the dataset from zroot/usr to zroot/var: ```sh # zfs rename zroot/usr/mydataset zroot/var/newname # zfs list NAME USED AVAIL REFER MOUNTPOINT zroot 3.94G 46.0G 96K /zroot zroot/ROOT 886M 46.0G 96K none zroot/ROOT/default 886M 46.0G 886M / zroot/fat32 3.05G 49.0G 31.0M - zroot/home 27.8M 46.0G 96K /home zroot/home/ykla 27.7M 46.0G 27.7M /home/ykla zroot/tmp 128K 46.0G 128K /tmp zroot/usr 288K 46.0G 96K /usr zroot/usr/ports 96K 46.0G 96K /usr/ports zroot/usr/src 96K 46.0G 96K /usr/src zroot/var 732K 46.0G 96K /var zroot/var/audit 96K 46.0G 96K /var/audit zroot/var/crash 96K 46.0G 96K /var/crash zroot/var/log 156K 46.0G 156K /var/log zroot/var/mail 96K 46.0G 96K /var/mail zroot/var/newname 96K 46.0G 96K /var/newname # Note this line zroot/var/tmp 96K 46.0G 96K /var/tmp ``` Snapshots cannot change their parent. Create another snapshot to verify: ```sh # zfs snapshot zroot/var/newname@02 # zfs list -t snapshot NAME USED AVAIL REFER MOUNTPOINT zroot/var/newname@01 0B - 96K - zroot/var/newname@02 0B - 96K - ``` Renaming snapshots also uses the same command. ```sh # zfs rename zroot/var/newname@02 zroot/var/newname@03 # zfs list -t snapshot NAME USED AVAIL REFER MOUNTPOINT zroot/var/newname@01 0B - 96K - zroot/var/newname@03 0B - 96K - ``` To recursively rename snapshots, use the `-r` option, which renames snapshots with the same name in all child datasets. `-r` also applies to recursively renaming a dataset and all its child datasets. ## Setting Dataset Properties Each ZFS dataset has a set of properties that control its behavior. Most properties are inherited from the parent dataset by default but can be individually overridden. Use the `zfs set` command to set dataset properties, with the syntax ``. Most properties have a limited range of valid values, and `zfs get` lists each property and its valid values. Use `zfs get all ` to view all current properties of a dataset (including inherited and locally set values), which is useful for troubleshooting and status confirmation. Use `zfs inherit` to restore most properties to their inherited values. Additionally, user-defined custom properties can be defined, which become part of the dataset configuration and can provide additional information about the dataset or its contents. To distinguish custom properties from ZFS built-in properties, use a colon (`:`) to create a custom namespace. Set a custom property and view it: ```sh # zfs set custom:costcenter=1234 zroot/var/newname # zfs get custom:costcenter zroot/var/newname NAME PROPERTY VALUE SOURCE zroot/var/newname custom:costcenter 1234 local ``` To delete a custom property, use `zfs inherit` with the `-r` option. If the custom property is not defined in any parent dataset, this option will remove it (but the pool history will still record the change). Delete the custom property and verify: ```sh # zfs inherit -r custom:costcenter zroot/var/newname # zfs get custom:costcenter zroot/var/newname NAME PROPERTY VALUE SOURCE zroot/var/newname custom:costcenter - - # zfs get all zroot/var/newname | grep custom:costcenter # ``` ### Getting and Setting Share Properties Two commonly used and practical dataset properties are NFS and SMB share options. Setting these properties defines whether and how ZFS shares the dataset over the network. FreeBSD supports both NFS (`sharenfs`) and SMB (`sharesmb`) share properties, where SMB sharing requires Samba to be installed. Using NFS as an example, to get the current state of sharing, enter: ```sh # zfs get sharenfs zroot/home NAME PROPERTY VALUE SOURCE zroot/home sharenfs off default ``` To enable sharing for the dataset, enter: ```sh # zfs set sharenfs=on zroot/home # zfs get sharenfs zroot/home NAME PROPERTY VALUE SOURCE zroot/home sharenfs on local ``` Additional options can be set for datasets shared via NFS, such as `-alldirs`, `-maproot`, and `-network`. To set share options, enter: ```sh # zfs set sharenfs="-alldirs,-maproot=root,-network=192.168.1.0/24" zroot/home # zfs get sharenfs zroot/home NAME PROPERTY VALUE SOURCE zroot/home sharenfs -alldirs,-maproot=root,-network=192.168.1.0/24 local ``` > **Warning** > > `-maproot=root` maps the NFS client root user to the server root, giving the remote root user full read-write access to the shared dataset. NFS enables `root_squash` by default (mapping client root to `nobody`), and using `-maproot=root` effectively disables this security mechanism. It is recommended to use this only in trusted internal networks, or switch to `-maproot=nobody` to restrict permissions, or map to a specified non-privileged user (e.g., `-maproot=1000`). ## Managing Snapshots ZFS snapshots use the copy-on-write (COW) mechanism, making creation instantaneous without consuming additional disk space. After a snapshot is created, when data blocks in the original file system are modified, ZFS writes the new data to a new location while the old data blocks remain in place, thus capturing the point-in-time state of the file system at the moment the snapshot was created. Snapshots provide a read-only, point-in-time copy of a dataset. Snapshots operate on the entire dataset rather than individual files, consuming no additional space at creation time, and consuming space as the blocks they reference change. A typical use of snapshots is to quickly back up the current file system state before performing high-risk operations such as software installation or system upgrades. If the operation fails, you can roll back to the snapshot to restore the system state; if the upgrade succeeds, delete the snapshot to free space. Snapshot rollback is fast and causes almost no downtime. Snapshots cannot replace full pool backups, but they are extremely efficient for saving dataset copies at specific points in time. The Chinese FreeBSD Community (CFC) provides ZFS scripts that can be used to view, create, delete, and restore ZFS snapshots. [ZFS Script Project Repository](https://github.com/FreeBSD-Ask/zfs-snap). The script has been deployed at and can be downloaded directly on a FreeBSD system using the `fetch` command. ### Creating Snapshots By default, the partition structure created using the Auto ZFS layout is as follows: ```sh # zfs list # List all ZFS file systems and their properties NAME USED AVAIL REFER MOUNTPOINT zroot 3.94G 46.0G 96K /zroot zroot/ROOT 886M 46.0G 96K none zroot/ROOT/default 886M 46.0G 886M / zroot/fat32 3.05G 49.0G 31.0M - zroot/home 27.8M 46.0G 96K /home zroot/home/ykla 27.7M 46.0G 27.7M /home/ykla zroot/tmp 128K 46.0G 128K /tmp zroot/usr 288K 46.0G 96K /usr zroot/usr/ports 96K 46.0G 96K /usr/ports zroot/usr/src 96K 46.0G 96K /usr/src zroot/var 732K 46.0G 96K /var zroot/var/audit 96K 46.0G 96K /var/audit zroot/var/crash 96K 46.0G 96K /var/crash zroot/var/log 156K 46.0G 156K /var/log zroot/var/mail 96K 46.0G 96K /var/mail zroot/var/newname 96K 46.0G 96K /var/newname zroot/var/tmp 96K 46.0G 96K /var/tmp ``` To create a snapshot, use the `zfs snapshot @` command. Adding the `-r` option creates snapshots recursively with the same name on all child datasets. Create a recursive snapshot of the entire pool: ```sh # zfs snapshot -r zroot@test # Recursively create snapshot test for zroot pool and all its child file systems ``` Snapshots are not displayed in normal `zfs list` output. To list snapshots, the `-t snapshot` option must be added to the `zfs list` command. `-t all` displays both file systems and snapshots. ```sh # zfs list -t snap # List all ZFS snapshots NAME USED AVAIL REFER MOUNTPOINT zroot@test 0B - 96K - zroot/ROOT@test 0B - 96K - zroot/ROOT/default@test 0B - 886M - zroot/fat32@test 0B - 31.0M - zroot/home@test 0B - 96K - zroot/home/ykla@test 0B - 27.7M - zroot/tmp@test 0B - 128K - zroot/usr@test 0B - 96K - zroot/usr/ports@test 0B - 96K - zroot/usr/src@test 0B - 96K - zroot/var@test 0B - 96K - zroot/var/audit@test 0B - 96K - zroot/var/crash@test 0B - 96K - zroot/var/log@test 0B - 156K - zroot/var/mail@test 0B - 96K - zroot/var/newname@01 0B - 96K - zroot/var/newname@03 0B - 96K - zroot/var/newname@test 0B - 96K - zroot/var/tmp@test 0B - 96K - # Note this line ``` > **Tip** > > In commands, `snapshot` can be abbreviated as `snap`. Other commands also have corresponding abbreviations, which you can look up in the documentation. Snapshots are not directly mounted, so no path is displayed in the `MOUNTPOINT` column. Since snapshots are read-only after creation, ZFS does not display available space in the `AVAIL` column. You can compare snapshots with the original dataset using the following command: ```sh # zfs list -rt all zroot/home NAME USED AVAIL REFER MOUNTPOINT zroot/home 27.8M 45.9G 96K /home zroot/home@test 0B - 96K - zroot/home/ykla 27.7M 45.9G 27.7M /home/ykla zroot/home/ykla@test 0B - 27.7M - ``` Displaying both datasets and snapshots simultaneously reveals how snapshots work with copy-on-write (COW). They only save the changed portions (*delta*), rather than re-saving the entire file system contents. This means snapshots consume minimal space when changes occur. By copying files to the dataset and then creating a second snapshot, you can more clearly observe the space usage: ```sh # cp /COPYRIGHT /var/tmp # zfs snapshot zroot/var/tmp@test2 ``` View the space usage changes of each snapshot after creating the second snapshot: ```sh # zfs list -rt all zroot/var/tmp NAME USED AVAIL REFER MOUNTPOINT zroot/var/tmp 164K 45.9G 100K /var/tmp zroot/var/tmp@test 64K - 96K - zroot/var/tmp@test2 0B - 100K - ``` The second snapshot only contains the changes to the dataset after the copy operation, thus greatly saving space. > **Note** > > The size of snapshot `zroot/var/tmp@test` has also changed in the `USED` column, which reflects the difference between it and the snapshots created after it. ### Destroying Snapshots When destroying snapshots, the `-r` parameter can be used for recursive deletion: ```sh # zfs destroy -r zroot@test # Recursively delete the test snapshot of zroot pool and its child file systems # zfs list -t snap # List all ZFS snapshots no datasets available ``` ## Snapshot Holds Sometimes it is necessary to prevent critical snapshots from being accidentally deleted — for example, before a backup process is complete, or when specific point-in-time copies must be retained due to regulatory requirements. ZFS hold mechanism applies named lightweight references to snapshots, preventing them from being deleted by `zfs destroy` until the hold is released. Add a hold tag to a snapshot: ```sh # zfs hold keeptest zroot/var/tmp@test ``` > **Tip** > > `keeptest` is just a label for easy identification and can be changed to other names. View all holds on a snapshot: ```sh # zfs holds zroot/var/tmp@test NAME TAG TIMESTAMP zroot/var/tmp@test keeptest Wed May 13 13:18 2026 ``` Attempting to destroy a held snapshot returns an error: ```sh # zfs destroy zroot/var/tmp@test cannot destroy snapshot zroot/var/tmp@test: it's being held. Run 'zfs holds -r zroot/var/tmp@test' to see holders. ``` Use the `-r` option to recursively apply holds to snapshots with the same name across all child datasets: ```sh # zfs hold -r keepall zroot@test # zfs holds -r zroot@test NAME TAG TIMESTAMP zroot@test keepall Wed May 13 13:18 2026 zroot/ROOT@test keepall Wed May 13 13:18 2026 zroot/ROOT/default@test keepall Wed May 13 13:18 2026 zroot/home@test keepall Wed May 13 13:18 2026 zroot/home/ykla@test keepall Wed May 13 13:18 2026 zroot/tmp@test keepall Wed May 13 13:18 2026 zroot/usr@test keepall Wed May 13 13:18 2026 zroot/usr/ports@test keepall Wed May 13 13:18 2026 zroot/usr/src@test keepall Wed May 13 13:18 2026 zroot/var@test keepall Wed May 13 13:18 2026 zroot/var/audit@test keepall Wed May 13 13:18 2026 zroot/var/crash@test keepall Wed May 13 13:18 2026 zroot/var/log@test keepall Wed May 13 13:18 2026 zroot/var/mail@test keepall Wed May 13 13:18 2026 zroot/var/tmp@test keepall Wed May 13 13:18 2026 zroot/var/tmp@test keeptest Wed May 13 13:18 2026 ``` After releasing the hold tag, the snapshot can be destroyed normally: ```sh # zfs release keeptest zroot/var/tmp@test # zfs release keepall zroot/var/tmp@test # zfs destroy zroot/var/tmp@test ``` The `-h` option of `zfs send` can include hold tags in the send stream. This is particularly useful for maintaining backup integrity in disaster recovery environments. ```sh # zfs send -h zroot/var/tmp@test | zfs receive backup/var/tmp ``` When the receiving side uses `zfs receive` to accept the stream, the snapshot holds will be automatically reconstructed on the receiver. ### Comparing Snapshots ZFS provides built-in commands for comparing differences between two snapshots. For scenarios involving long-term storage of numerous snapshots, this feature is extremely practical, allowing users to see how the file system has changed over time. For example, `zfs diff` can help users find the most recent snapshot and check whether it still contains a mistakenly deleted file. Comparing the two snapshots created in the previous sections yields the following output: ```sh # zfs diff zroot/var/tmp@test + /var/tmp/COPYRIGHT M /var/tmp/ ``` This command lists the changes between the specified snapshot (here `zroot/var/tmp@test`) and the current file system. The first column shows the type of change: | Symbol | Function | | ------ | --------------------- | | + | Path or file added | | - | Path or file deleted | | M | Path or file modified | | R | Path or file renamed | Comparing the output with the symbols above, ZFS added the **COPYRIGHT** file after creating the `zroot/var/tmp@test` snapshot, which also caused the parent directory mounted at `/var/tmp` to be modified. Comparing two snapshots is very useful when using ZFS replication to transfer datasets to different hosts for backup. Compare two snapshots by specifying the full dataset name and two snapshot names: ```sh # cp /var/tmp/COPYRIGHT /var/tmp/COPYRIGHT.copy # zfs snapshot zroot/var/tmp@diff # zfs diff zroot/var/tmp@test zroot/var/tmp@diff + /var/tmp/COPYRIGHT M /var/tmp/ + /var/tmp/COPYRIGHT.copy # zfs diff zroot/var/tmp@test zroot/var/tmp@test2 + /var/tmp/COPYRIGHT M /var/tmp/ ``` Backup administrators can compare two snapshots received from the sending host and determine the actual changes in the dataset. ### Snapshot Rollback As long as at least one snapshot exists, you can roll back to it at any time. The most common scenarios for rollback are: the current dataset state has become invalid, or an older version is more suitable. For example, local development testing errors, system update failures causing functionality damage, or the need to recover deleted files or directories — these situations are all common. To roll back to a snapshot, use the `zfs rollback ` command. If the amount of changes is large, the operation may take longer. During this period, the dataset remains in a consistent state, similar to a database performing a rollback following ACID principles. The entire process completes while the dataset is online and requires no downtime. After rollback, the dataset state is restored to the state at the time the snapshot was created. Rolling back to a snapshot discards all data in the dataset that does not belong to that snapshot. If you create a snapshot of the current state before rolling back to an earlier snapshot, you can easily roll back again if you later need some data. This way, users can switch between snapshots without losing data that still has value. ### Snapshot Restore Testing You can verify the effectiveness of snapshots by adding and deleting files. In a test environment, if snapshots have been created beforehand, even executing `rm -rf /*` can be successfully recovered; if the system uses UEFI, you need to restore the EFI boot according to the instructions in other chapters. > **Note** > > This operation should only be performed in a test environment. Suppose in the previous example, an accidental `rm` command deleted more data than expected, and you need to roll back to a snapshot: First, view the currently available snapshots: ```sh # zfs list -rt all zroot/var/tmp NAME USED AVAIL REFER MOUNTPOINT zroot/var/tmp 232K 45.9G 112K /var/tmp zroot/var/tmp@test 64K - 96K - zroot/var/tmp@test2 56K - 100K - zroot/var/tmp@diff 0B - 112K - ``` View the files in the current directory: ```sh # ls /var/tmp COPYRIGHT COPYRIGHT.copy vi.recover ``` Simulate an accidental deletion by removing COPYRIGHT-related files: ```sh # rm /var/tmp/COPYRIGHT* # ls /var/tmp vi.recover ``` At this point, the user realizes that extra files were accidentally deleted and wants to recover them. ZFS provides a simple recovery method: just create snapshots of important data regularly. After rolling back to the previous snapshot, you can retrieve the lost files and start over from that point: Execute the rollback operation to restore the dataset to the state of the diff snapshot: ```sh # zfs rollback zroot/var/tmp@diff # ls -al /var/tmp total 19 drwxrwxrwt 3 root wheel 5 Apr 16 13:01 . drwxr-xr-x 25 root wheel 25 Apr 16 12:33 .. -r--r--r-- 1 root wheel 6070 Apr 16 12:56 COPYRIGHT -r--r--r-- 1 root wheel 6070 Apr 16 13:01 COPYRIGHT.copy drwxrwxrwt 2 root wheel 2 Apr 13 12:38 vi.recover ``` The rollback operation restores the dataset to the state of the last snapshot. Confirm that the snapshot list remains unchanged after rollback: ```sh # zfs list -rt snapshot zroot/var/tmp NAME USED AVAIL REFER MOUNTPOINT zroot/var/tmp@test 64K - 96K - zroot/var/tmp@test2 56K - 100K - zroot/var/tmp@diff 0B - 112K - ``` You can also roll back to an earlier snapshot, even if other snapshots exist after it. When attempting this, ZFS displays the following warning: ```sh # zfs rollback zroot/var/tmp@test cannot rollback to 'zroot/var/tmp@test': more recent snapshots or bookmarks exist use '-r' to force deletion of the following snapshots and bookmarks: zroot/var/tmp@test2 zroot/var/tmp@diff ``` This warning indicates that other snapshots exist between the target snapshot and the current dataset state. To complete the rollback, these snapshots must be deleted. Unlike virtual machine snapshots, by default, the `zfs rollback` command can only roll back to the most recent snapshot ([Reference Manual](https://docs.oracle.com/cd/E19253-01/819-7065/gbcxk/index.html), Oracle official ZFS rollback command documentation). Since snapshots are read-only, ZFS cannot track all changes to a dataset across different states unless the user uses the `-r` option to confirm this is the desired operation and destroy all snapshots newer than the target snapshot. After using the `-r` option, ZFS will destroy all snapshots newer than the target snapshot, thus allowing rollback to a non-latest snapshot. If this is indeed the intended operation and the user understands the consequences of deleting all intermediate snapshots, the following command can be executed: ```sh # zfs rollback -r zroot/var/tmp@test # zfs list -rt snapshot zroot/var/tmp NAME USED AVAIL REFER MOUNTPOINT zroot/var/tmp@test 0B - 96K - # ls -al /var/tmp total 10 drwxrwxrwt 3 root wheel 3 Apr 16 19:49 . drwxr-xr-x 25 root wheel 25 Apr 16 12:33 .. drwxrwxrwt 2 root wheel 2 Apr 13 12:38 vi.recover ``` From the `zfs list -t snapshot` output, you can confirm that after executing `zfs rollback -r`, the intermediate snapshots have been deleted. ZFS does not support recursively rolling back all child datasets at once; you must execute the rollback operation separately for each child file system. ```sh # zfs rollback -r zroot@test # Roll back zroot to test snapshot and destroy newer snapshots # zfs rollback -r zroot/ROOT@test # Roll back zroot/ROOT to test snapshot and destroy newer snapshots # zfs rollback -r zroot/ROOT/default@test # Roll back zroot/ROOT/default to test snapshot and destroy newer snapshots # zfs rollback -r zroot/tmp@test # Roll back zroot/tmp to test snapshot and destroy newer snapshots # zfs rollback -r zroot/usr@test # Roll back zroot/usr to test snapshot and destroy newer snapshots # zfs rollback -r zroot/home@test # Roll back zroot/home to test snapshot and destroy newer snapshots # zfs rollback -r zroot/usr/ports@test # Roll back zroot/usr/ports to test snapshot and destroy newer snapshots # zfs rollback -r zroot/var@test # Roll back zroot/var to test snapshot and destroy newer snapshots # zfs rollback -r zroot/var/log@test # Roll back zroot/var/log to test snapshot and destroy newer snapshots ``` > **Exercise** > > Explore better snapshot rollback solutions. You can refer to the Chinese community [ZFS Script Project](https://github.com/FreeBSD-Ask/zfs-snap), or submit feature requests and Pull Requests to the OpenZFS project. ### Recovering Individual Files from Snapshots Snapshots are stored in a hidden directory under the parent dataset at **.zfs/snapshots/snapshotname**. By default, these directories are not displayed even when executing the `ls -a` command. Although invisible, they can still be accessed like regular directories. The `snapdir` property controls whether these hidden directories appear in directory listings. When this property is set to `visible`, they will appear in the output of `ls` and other commands involving directory contents. View the current setting of the snapdir property, then switch it to visible: ```sh # zfs get snapdir zroot/var/tmp NAME PROPERTY VALUE SOURCE zroot/var/tmp snapdir hidden default # ls -a /var/tmp . .. vi.recover # zfs set snapdir=visible zroot/var/tmp # ls -al /var/tmp total 10 drwxrwxrwt 4 root wheel 3 Apr 16 19:49 . drwxr-xr-x 25 root wheel 25 Apr 16 12:33 .. dr-xr-xr-x+ 3 root wheel 3 Apr 16 19:46 .zfs # Note this line drwxrwxrwt 2 root wheel 2 Apr 13 12:38 vi.recover ``` By copying files from the snapshot back to the parent dataset, individual files can be restored to their previous state. The directory structure under **.zfs/snapshot** contains directories named after the previously created snapshots for easy identification. The following example shows how to recover files from the hidden **.zfs** directory — copying files back from the snapshot that contains the target files: After deleting a file, find and recover it through the .zfs/snapshot directory: ```sh # rm -rf /var/tmp/vi.recover # ls -a /var/tmp . .. .zfs # ls /var/tmp/.zfs/snapshot test # ls /var/tmp/.zfs/snapshot/test vi.recover # cp -r /var/tmp/.zfs/snapshot/test/vi.recover /var/tmp # ls /var/tmp/ .zfs vi.recover ``` Even if the `snapdir` property is restored to the default `hidden`, executing `ls .zfs/snapshot` will still list the directory contents. Administrators can decide whether to display these directories on their own; this setting applies to each dataset. After restoring snapdir to hidden, snapshots can still be accessed directly via the path: ```sh # zfs set snapdir=hidden zroot/var/tmp # ls /var/tmp/.zfs/snapshot/test vi.recover ``` Copying files or directories from the hidden **.zfs/snapshot** directory is very simple. Conversely, when trying to copy files to the snapshot directory, the following error occurs: Attempt to write a file to the snapshot directory to verify the read-only nature of snapshots: ```sh # cp /COPYRIGHT /var/tmp/.zfs/snapshot/test/ cp: /var/tmp/.zfs/snapshot/test/COPYRIGHT: Read-only file system ``` This error reminds users that snapshots are read-only and cannot be changed after creation. Since copying or deleting files to/from the snapshot directory would change the dataset state represented by the snapshot, these operations cannot be performed. Snapshots consume space as changes occur in the parent file system since the snapshot was created. The `written` property of a snapshot tracks the space consumed by that snapshot. To destroy a snapshot and reclaim space, use the `zfs destroy @` command. Adding the `-r` option recursively deletes all snapshots with the same name under the parent dataset. Using the `-n -v` options, the command lists the snapshots that will be deleted and their estimated reclaimed space without actually performing the destruction. ## Managing Clones A clone is a writable copy of a snapshot that is writable, mountable, and has its own properties. After creating a clone with `zfs clone`, the original snapshot cannot be destroyed. To reverse the parent-child relationship between a clone and a snapshot, use `zfs promote`. After promoting a clone, the snapshot becomes a child of the clone, and the space accounting under the original parent-child relationship changes accordingly. Clones can be mounted anywhere in the ZFS file system hierarchy. ### Creating Clones Here is a sample dataset demonstrating clone functionality: ```sh # zfs list -rt all zroot/home/ykla NAME USED AVAIL REFER MOUNTPOINT zroot/home/ykla 27.7M 45.9G 27.7M /home/ykla zroot/home/ykla@test 0B - 27.7M - ``` A typical use of clones is to experiment with a specific dataset while preserving the snapshot, so that you can recover if something goes wrong. Since snapshots cannot be changed, you can create a read-write clone of the snapshot. After the experiment yields satisfactory results, you can promote the clone to a dataset and delete the initial file system. Since clones and datasets can coexist without causing problems, deleting the original dataset is not necessary. Create a clone from a snapshot and verify that the clone has the same content as the original dataset: ```sh # zfs clone zroot/home/ykla@test zroot/home/ykla2 # ls -al /home/ykla* /home/ykla: total 28222 drwxr-xr-x 2 ykla ykla 11 Apr 16 11:55 . drwxr-xr-x 4 root wheel 4 Apr 16 13:30 .. -rw-r--r-- 1 ykla ykla 950 Apr 16 19:50 .cshrc -rw-r--r-- 1 ykla ykla 311 Apr 16 19:50 .login -rw-r--r-- 1 ykla ykla 79 Apr 16 19:50 .login_conf -rw------- 1 ykla ykla 289 Apr 16 19:50 .mail_aliases -rw-r--r-- 1 ykla ykla 255 Apr 16 19:50 .mailrc -rw-r--r-- 1 ykla ykla 966 Apr 16 19:50 .profile -rw-r--r-- 1 ykla ykla 1042 Apr 16 19:50 .shrc -rw-r--r-- 1 ykla ykla 37635246 May 10 13:23 Philosophische-untersuchungen.pdf -rw-r--r-- 1 root ykla 118 Apr 16 11:58 checksum.txt /home/ykla2: total 28222 drwxr-xr-x 2 ykla ykla 11 Apr 16 11:55 . drwxr-xr-x 4 root wheel 4 Apr 16 13:30 .. -rw-r--r-- 1 ykla ykla 950 Apr 16 19:50 .cshrc -rw-r--r-- 1 ykla ykla 311 Apr 16 19:50 .login -rw-r--r-- 1 ykla ykla 79 Apr 16 19:50 .login_conf -rw------- 1 ykla ykla 289 Apr 16 19:50 .mail_aliases -rw-r--r-- 1 ykla ykla 255 Apr 16 19:50 .mailrc -rw-r--r-- 1 ykla ykla 966 Apr 16 19:50 .profile -rw-r--r-- 1 ykla ykla 1042 Apr 16 19:50 .shrc -rw-r--r-- 1 ykla ykla 37635246 May 10 13:23 Philosophische-untersuchungen.pdf -rw-r--r-- 1 root ykla 118 Apr 16 11:58 checksum.txt ``` View the disk usage of the clone and the original dataset; they share the same data blocks: ```sh # df -h /home* Filesystem Size Used Avail Capacity Mounted on zroot/home 46G 96K 46G 0% /home # df -h /home/ykla* Filesystem Size Used Avail Capacity Mounted on zroot/home/ykla 46G 28M 46G 0% /home/ykla zroot/home/ykla2 46G 28M 46G 0% /home/ykla2 ``` ### Promoting Clones When a clone is created, it is a complete copy of the dataset at the time the snapshot was taken. After that, the clone can change independently of the original dataset. The two are linked by the snapshot, and ZFS records this association in the `origin` property. After using `zfs promote` to promote the clone, it becomes an independent dataset, the `origin` property value is cleared, and the association between the clone and the snapshot is also severed. The following example demonstrates this process: Before promotion, view the clone origin property to confirm it originates from the snapshot: ```sh # zfs get origin zroot/home/ykla2 NAME PROPERTY VALUE SOURCE zroot/home/ykla2 origin zroot/home/ykla@test - ``` Execute the promotion: ```sh # zfs promote zroot/home/ykla2 ``` After promotion, the origin property is cleared, and the clone becomes an independent dataset: ```sh # zfs get origin zroot/home/ykla2 NAME PROPERTY VALUE SOURCE zroot/home/ykla2 origin - - ``` After making some changes (such as copying **COPYRIGHT** to the promoted clone), the old directory is now outdated. At this point, the promoted clone can be used to replace it. To accomplish the replacement, first use `zfs destroy` to delete the old dataset, then use `zfs rename` to rename the clone to the old dataset name (or a completely different name). Add a new file to the clone: ```sh # cp /COPYRIGHT /home/ykla2 ``` Delete the original dataset: ```sh # zfs destroy -f zroot/home/ykla ``` Rename the clone to the original dataset name to complete the replacement: ```sh # zfs rename zroot/home/ykla2 zroot/home/ykla # ls /home/ykla .cshrc .profile .login .shrc .login_conf COPYRIGHT .mail_aliases Philosophische-untersuchungen.pdf .mailrc checksum.txt # df -h /home/ykla Filesystem Size Used Avail Capacity Mounted on zroot/home/ykla 46G 28M 46G 0% /home/ykla ``` At this point, the dataset cloned from the snapshot has become a regular dataset. It contains all the data from the original snapshot plus the newly added file (such as **COPYRIGHT**). Clones provide ZFS users with practical functionality in various scenarios. For example, you can provide jails with snapshots containing different application sets, and users can clone these snapshots and add their own applications as needed. Once satisfied with the changes, the clone can be promoted to a full dataset and delivered to end users, who can use it as a regular dataset. This greatly saves the time and management overhead required to provide jails. ## Replication Storing data only in a single location within a single pool exposes it to risks such as theft, natural disasters, or human error. Therefore, regular backups of the entire pool are essential. ZFS provides built-in serialization functionality that can send a stream representation of data to standard output. With this feature, data can be stored to another pool connected to the local system or sent over the network to another system. Snapshots are the foundation of this replication. The commands used to replicate data are `zfs send` and `zfs receive`. Create the required test storage pools: ```sh # mdconfig -a -t swap -s 1G md0 # mdconfig -a -t swap -s 1G md1 # zpool create mypool md0 # zpool create backup md1 # zpool list backup mypool NAME SIZE ALLOC FREE CKPOINT EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT backup 960M 372K 960M - - 7% 0% 1.00x ONLINE - mypool 960M 372K 960M - - 7% 0% 1.00x ONLINE - ``` Copy some **non-critical** files for testing: ```sh # cp /home/ykla/bsdbook.pdf /mypool/ ``` The following example shows how to use these two pools to complete ZFS replication: The storage pool **mypool** is the primary pool used for reading and writing regular data. The second pool **backup** serves as a backup in case the primary pool becomes unavailable. Note that ZFS does not perform automatic failover; administrators must manually intervene when necessary. Using snapshots provides a consistent version of the file system for replication and recovery. After creating a snapshot for **mypool**, it can be transferred to the **backup** pool by replicating the snapshot. This operation does not include changes made since the last snapshot. ```sh # zfs snapshot mypool@backup1 ``` Confirm the snapshot has been created: ```sh # zfs list -t snapshot mypool NAME USED AVAIL REFER MOUNTPOINT mypool@backup1 0B - 165M - ``` Now that a snapshot exists, you can use `zfs send` to create a stream representing the snapshot contents. Store this stream as a file, or receive it on another pool. The stream must be redirected to a file or pipe, otherwise the following error will occur: ```sh # zfs send mypool@backup1 Error: Stream can not be written to a terminal. You must redirect standard output. ``` ### Creating a Backup Stream To back up a dataset, use `zfs send` to redirect the stream to a file on a mounted backup pool. Make sure the pool has enough available space to accommodate the size of the snapshot being sent (the total amount of data contained in the snapshot, not the difference from the previous snapshot). ```sh # zfs send mypool@backup1 > /backup/backup1 ``` View the space usage changes of both pools after sending: ```sh # zpool list mypool backup NAME SIZE ALLOC FREE CKPOINT EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT backup 960M 161M 799M - - 8% 16% 1.00x ONLINE - mypool 960M 166M 794M - - 8% 17% 1.00x ONLINE - ``` `zfs send` has transferred all data from snapshot **backup1** to the storage pool **backup**. To automatically create and send these snapshots, you can configure cron jobs. ### Receiving a Backup Stream Rather than storing as an archive file, ZFS can receive data as an active file system, allowing direct access to backup data. To access the actual data in these streams, use `zfs receive` to restore the stream into files and directories. The following example combines `zfs send` with `zfs receive`, piping data from pool **mypool** to another pool **backup**. First, create a snapshot for mypool: ```sh # zfs snapshot mypool@replica1 ``` After the transfer is complete, the data can be used directly on the receiving pool. **You can only replicate to an empty dataset.** Execute a full send and receive: ```sh # zfs send -v mypool@replica1 | zfs receive backup/mypool full send of mypool@replica1 estimated size is 166M total estimated size is 166M TIME SENT SNAPSHOT mypool@replica1 ``` View the space usage of both pools after receiving: ```sh # zpool list mypool backup NAME SIZE ALLOC FREE CKPOINT EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT backup 960M 331M 629M - - 12% 34% 1.00x ONLINE - mypool 960M 166M 794M - - 8% 17% 1.00x ONLINE - ``` View the backup directory structure: ```sh # ls -al /backup/ total 169330 drwxr-xr-x 3 root wheel 4 Apr 16 11:56 . drwxr-xr-x 22 root wheel 24 Apr 16 11:53 .. -rw-r--r-- 1 root wheel 173869216 Apr 16 11:54 backup1 drwxr-xr-x 2 root wheel 3 Apr 16 11:53 mypool ``` Verify that the backup file matches the source file: ```sh # ls -al /backup/mypool/bsdbook.pdf -rw-r--r-- 1 root wheel 173330339 Apr 16 11:54 /backup/mypool/bsdbook.pdf ``` ### Incremental Backup `zfs send` can also determine the differences between two snapshots and send only the incremental changes between them. This saves both disk space and transmission time. Copy some more **non-critical** files to the **mypool** pool for testing: ```sh # cp /home/ykla/bsdbook.epub /mypool ``` For example: ```sh # zfs snapshot mypool@replica2 ``` View the REFER value changes across the three snapshots: ```sh # zfs list -t snapshot mypool NAME USED AVAIL REFER MOUNTPOINT mypool@backup1 0B - 165M - mypool@replica1 0B - 165M - mypool@replica2 0B - 356M - ``` View the capacity of both pools before incremental replication: ```sh # zpool list backup mypool NAME SIZE ALLOC FREE CKPOINT EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT backup 960M 331M 629M - - 12% 34% 1.00x ONLINE - mypool 960M 356M 604M - - 12% 37% 1.00x ONLINE - ``` The second snapshot **replica2** has been created above. This snapshot contains all changes to the file system since the previous snapshot **replica1**. Using `zfs send -i` and specifying the pair of snapshots generates an incremental replication stream containing only the changed data. This operation succeeds if the initial snapshot already exists on the receiving side. ```sh # zfs send -v -i mypool@replica1 mypool@replica2 | zfs receive backup/mypool send from mypool@replica1 to mypool@replica2 estimated size is 191M total estimated size is 191M TIME SENT SNAPSHOT mypool@replica2 ``` After the incremental send, view the space usage of both pools — the backup pool only increased by the differential data amount: ```sh # zpool list backup mypool NAME SIZE ALLOC FREE CKPOINT EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT backup 960M 522M 438M - - 13% 54% 1.00x ONLINE - mypool 960M 356M 604M - - 12% 37% 1.00x ONLINE - ``` List both pools and their datasets to view space allocation: ```sh # zfs list -r backup mypool NAME USED AVAIL REFER MOUNTPOINT backup 522M 310M 165M /backup backup/mypool 356M 310M 356M /backup/mypool mypool 356M 476M 356M /mypool ``` View the snapshot list in the backup pool: ```sh # zfs list -t snapshot backup mypool NAME USED AVAIL REFER MOUNTPOINT mypool@backup1 0B - 165M - mypool@replica1 0B - 165M - mypool@replica2 0B - 356M - ``` The incremental stream only copied the changed data, not the entire **replica1**. Sending only the differences significantly reduces transmission time and saves disk space by avoiding copying the entire pool each time. This advantage is particularly notable when replicating over slow networks or networks that charge by the byte transferred. At this point, the new file system **backup/mypool** is available, containing the data and files from the **mypool** pool. Confirm that the incremental backup file exists in the backup pool: ```sh # ls -al /backup/mypool/bsdbook.epub -rw-r--r-- 1 root wheel 199611278 Apr 16 11:57 /backup/mypool/bsdbook.epub ``` The `-p` option copies dataset properties, including compression settings, quotas, and mount points. The `-R` option replicates all child datasets of the dataset along with their properties. The send and receive process can be automated to create backups on a second pool on a regular basis. ### Sending Encrypted Backups via SSH Sending stream data over the network is an effective way to maintain remote backups, but it has a drawback: the data sent over the network link is unencrypted, and anyone can intercept the stream data and restore it to the original data without the sender authorization. This is unacceptable when sending data over the internet to a remote host. To ensure secure data transmission, SSH can be used to encrypt the sent data. Since ZFS requires the stream to be redirected from standard output, it can be easily piped through SSH. To ensure that file system contents remain encrypted at rest on the remote system, ZFS native encryption is recommended. Create an encrypted dataset on the receiving side: ```sh # zfs create -o encryption=aes-256-gcm -o keyformat=passphrase recvpool/backup ``` The received stream data will be automatically written to disk in encrypted form. ZFS native encryption has been mature and available since OpenZFS 0.8.0, seamlessly integrated with ZFS snapshots and send/receive, without requiring additional kernel modules. First, make some configuration changes and take security precautions. The following describes the steps required to perform `zfs send` operations: Configuration changes are as follows: * Use SSH keys for passwordless SSH access between the sending and receiving hosts * ZFS requires `root` user privileges to send and receive streams. This requires logging in to the receiving system as `root`. * For security reasons, `root` login is disabled by default. * Use the ZFS delegation system to allow non-`root` users on each system to perform the corresponding send and receive operations. On the sending system: ```sh # zfs allow -u username send,snapshot mypool ``` * To mount the pool, the non-privileged user must own the directory, and regular users need permission to mount file systems. On the receiving system: ```sh # sysctl vfs.usermount=1 vfs.usermount: 0 -> 1 # echo vfs.usermount=1 >> /etc/sysctl.conf # zfs create recvpool/backup # zfs allow -u username create,mount,receive recvpool/backup # chown username /recvpool/backup ``` Now, non-privileged users can also receive and mount datasets, and copy the *home* dataset to the remote system. It is recommended to use an IP address or fully qualified domain name. The receiver writes data to the *backup* dataset on the *recvpool* pool. ```sh % zfs snapshot -r mypool/home@monday % zfs send -R mypool/home@monday | ssh username@backuphost zfs recv -dvu recvpool/backup ``` First, create a recursive snapshot *monday* for the file system dataset *home* on the **mypool** pool. Then `zfs send -R` includes the dataset, all child datasets, snapshots, clones, and settings in the stream. The output is piped through SSH to `zfs receive` waiting on the remote host *backuphost*. In `zfs recv`: * The `-d` option overrides the receiving pool name with the snapshot name; * The `-u` option prevents the file system from being mounted on the receiving side; * The `-v` option displays more transfer details, including elapsed time and data volume. ## Bookmarks Bookmarks are lightweight references to snapshots that record the point-in-time position when the snapshot was created. Unlike snapshots, bookmarks do not hold data blocks, so they consume no additional disk space after creation and do not consume space as data changes, unlike snapshots. The typical use of bookmarks is as a source for `zfs send` incremental streams, enabling incremental replication without a complete snapshot. The bookmark feature requires the `bookmarks` feature flag to be enabled on the storage pool. In modern versions of ZFS, this flag is enabled by default. ### Creating Bookmarks Use the `zfs bookmark` command to create a bookmark for snapshot `zroot/var/tmp@test`: ```sh # zfs bookmark zroot/var/tmp@test zroot/var/tmp#mybookmark ``` You can also create a bookmark based on an existing bookmark: ```sh # zfs bookmark zroot/var/tmp#mybookmark zroot/var/tmp#mybookmark2 ``` Unlike snapshots which use `@` as a separator, bookmarks use `#` to separate the dataset name from the bookmark name. ### Listing Bookmarks Bookmarks are not displayed in normal `zfs list` output. To list bookmarks, use the `-t bookmark` option: ```sh # zfs list -t bookmark NAME USED AVAIL REFER MOUNTPOINT zroot/var/tmp#mybookmark - - 96K - zroot/var/tmp#mybookmark2 - - 96K - ``` Since bookmarks consume no space and cannot be mounted, the `USED`, `AVAIL`, and `MOUNTPOINT` columns all display `-`. The `REFER` column shows the amount of data referenced by the dataset at the time the bookmark was created. ### Using Bookmarks for Incremental Send The main value of bookmarks lies in incremental replication. If the snapshot on the source system has been destroyed but an incremental stream still needs to be sent to the backup system, bookmarks can replace snapshots as the incremental source: ```sh # zfs send -i zroot/var/tmp#mybookmark zroot/var/tmp@test | zfs receive backup/var/tmp ``` Administrators can therefore continue the incremental backup chain even after destroying intermediate snapshots, greatly saving storage space. ### Destroying Bookmarks Use the `zfs destroy` command to destroy bookmarks, with syntax similar to destroying snapshots but using the `#` separator: ```sh # zfs destroy zroot/var/tmp#mybookmark2 ``` After a bookmark is destroyed, if it was the only source for an incremental send chain, subsequent incremental sends will not be able to use that bookmark as a baseline. ### Bookmarks and Rollback When attempting to roll back to a non-latest snapshot, ZFS detects whether newer snapshots and bookmarks exist and prompts the user to use the `-r` option to force deletion: ```sh # zfs rollback zroot/var/tmp@test cannot rollback to 'zroot/var/tmp@test': more recent snapshots or bookmarks exist use '-r' to force deletion of the following snapshots and bookmarks: zroot/var/tmp@test2 zroot/var/tmp@diff ``` Although bookmarks do not hold data, they represent a point-in-time reference and therefore prevent rollback to snapshots earlier than them unless the user explicitly confirms the need to delete these bookmarks. ## Dataset, User, and Group Quotas Dataset quotas are used to limit the space a specific dataset can consume. Reference quotas are similar but only count the space used by the dataset itself, excluding snapshots and child datasets. Similarly, user quotas and group quotas prevent users or groups from exhausting all the space in a pool or dataset. ### Dataset Quotas Home directory datasets for new users are typically automatically created as `zroot/home/username`, with `mountpoint` generally set to **/home/username**. The example assumes the system has a user `ykla`. As follows: ```sh # zfs list -r zroot/home NAME USED AVAIL REFER MOUNTPOINT zroot/home 356M 48.7G 96K /home zroot/home/ykla 356M 48.7G 356M /home/ykla ``` To enforce a 1 GB dataset quota on **zroot/home/ykla**: ```sh # zfs set quota=1G zroot/home/ykla ``` To enforce a 2 GB reference quota on **zroot/home/ykla**: ```sh # zfs set refquota=2G zroot/home/ykla ``` To remove the 1 GB quota on **zroot/home/ykla**: ```sh # zfs set quota=none zroot/home/ykla ``` ### User Quotas The general format is `userquota@user=size dataset_or_pool`, and the user name can be in any of the following formats: | Format | Example | | --------------------- | --------------------------------------------- | | POSIX-compatible name | *ykla* | | POSIX numeric ID | *789* | | SID name | [*ykla@example.com*](mailto:ykla@example.com) | | SID numeric ID | *S-1-123-456-789* | For example, to enforce a 50 GB user quota for user **ykla**: ```sh # zfs set userquota@ykla=50G zroot ``` User quota properties are not displayed by `zfs get all`. Non-`root` users cannot view other users quotas unless granted the `userquota` privilege. Users with this privilege can view and set quotas for everyone. To view user quotas: ```sh # zfs get userquota@ykla zroot NAME PROPERTY VALUE SOURCE zroot userquota@ykla 50G local ``` To remove all quotas: ```sh # zfs set userquota@ykla=none zroot ``` Confirm the user quota has been cleared: ```sh # zfs get userquota@ykla zroot NAME PROPERTY VALUE SOURCE zroot userquota@ykla none local ``` ### Group Quotas The general format for setting group quotas is: `groupquota@group=size dataset_or_pool`. To set the quota for group *ykla* to 50 GB, use: ```sh # zfs set groupquota@ykla=50G zroot ``` Similar to user quota properties, non-`root` users can view quotas for groups they belong to. Users with the `groupquota` privilege or `root` can view and set quotas for all groups. Confirm the group quota is in effect: ```sh # zfs get groupquota@ykla zroot NAME PROPERTY VALUE SOURCE zroot groupquota@ykla 50G local ``` To remove the quota for group *ykla*, or to ensure no quota is set, use: ```sh # zfs set groupquota@ykla=none zroot ``` Confirm the group quota has been cleared: ```sh # zfs get groupquota@ykla zroot NAME PROPERTY VALUE SOURCE zroot groupquota@ykla none local ``` ### Viewing Quota Usage To display the space used by each user in a file system or snapshot along with all quotas, use `zfs userspace`. For group information, use `zfs groupspace`. Privileged users and `root` can list the quota for **zroot/home/ykla** by: ```sh # zfs get quota zroot/home/ykla NAME PROPERTY VALUE SOURCE zroot/home/ykla quota none local ``` ## Reservations Reservations ensure that a dataset always has available space (the reserved space is not allocated to any other dataset), which is very practical for guaranteeing space for critical datasets or log files. The format of the `reservation` property is `reservation=size`. The following command sets a 10 GB reservation for **zroot/home/ykla**: ```sh # zfs set reservation=10G zroot/home/ykla ``` The following command displays the existing reservation on **zroot/home/ykla**: ```sh # zfs get reservation zroot/home/ykla NAME PROPERTY VALUE SOURCE zroot/home/ykla reservation 10G local ``` To clear all reservations: ```sh # zfs set reservation=none zroot/home/ykla ``` Confirm the reservation has been cleared: ```sh # zfs get reservation zroot/home/ykla NAME PROPERTY VALUE SOURCE zroot/home/ykla reservation none local ``` ### Reference Reservation The `refreservation` property is used to set a reference reservation, applying the same principles, with the format `refreservation=size`. The following command sets a 1 GB reference reservation for **zroot/home/ykla**: ```sh # zfs set refreservation=1G zroot/home/ykla ``` The following command displays the existing reference reservation on **zroot/home/ykla**: ```sh # zfs get refreservation zroot/home/ykla NAME PROPERTY VALUE SOURCE zroot/home/ykla refreservation 1G local ``` To clear all reference reservations: ```sh # zfs set refreservation=none zroot/home/ykla ``` Confirm the reference reservation has been cleared: ```sh # zfs get refreservation zroot/home/ykla NAME PROPERTY VALUE SOURCE zroot/home/ykla refreservation none local ``` ## Compression ZFS data compression is implemented at the file system level and is transparent to upper-layer applications. When enabled, it typically reduces disk usage while improving read/write throughput. Administrators can view compression effectiveness through dataset properties. ```sh # zfs get used,compressratio,compression,logicalused zroot NAME PROPERTY VALUE SOURCE zroot used 3.94G - zroot compressratio 1.94x - zroot compression on local zroot logicalused 1.73G - ``` Set the data compression algorithm for the zroot file system to zstd level 5: ```sh # zfs set compression=zstd-5 zroot ``` > **Note** > > Compression property changes take effect immediately without restarting the system. However, this property only applies to newly written data and does not automatically compress existing data. List the data compression properties and their current settings for each ZFS file system again: ```sh # zfs get compression NAME PROPERTY VALUE SOURCE zroot compression zstd-5 local zroot/ROOT compression zstd-5 inherited from zroot zroot/ROOT/default compression zstd-5 inherited from zroot zroot/home compression zstd-5 inherited from zroot zroot/home/ykla compression zstd-5 inherited from zroot zroot/tmp compression zstd-5 inherited from zroot zroot/usr compression zstd-5 inherited from zroot zroot/usr/ports compression zstd-5 inherited from zroot zroot/usr/src compression zstd-5 inherited from zroot zroot/var compression zstd-5 inherited from zroot zroot/var/audit compression zstd-5 inherited from zroot zroot/var/crash compression zstd-5 inherited from zroot zroot/var/log compression zstd-5 inherited from zroot zroot/var/mail compression zstd-5 inherited from zroot zroot/var/tmp compression zstd-5 inherited from zroot ``` View the actual data compression ratio for each ZFS file system: > **Tip** > > `compressratio` represents the ratio of compressed data to uncompressed data. For example, 2.70x means the data is compressed to approximately 37% of its original size. ```sh # zfs get compressratio NAME PROPERTY VALUE SOURCE zroot compressratio 2.70x - zroot/ROOT compressratio 2.68x - zroot/ROOT/default compressratio 2.68x - zroot/home compressratio 1.00x - zroot/home/ykla compressratio 1.01x - zroot/tmp compressratio 1.00x - zroot/usr compressratio 2.73x - zroot/usr/ports compressratio 1.00x - zroot/usr/src compressratio 2.73x - zroot/var compressratio 1.47x - zroot/var/audit compressratio 1.00x - zroot/var/crash compressratio 1.01x - zroot/var/log compressratio 2.90x - zroot/var/mail compressratio 1.00x - zroot/var/tmp compressratio 1.00x - ``` ZFS provides different compression algorithms, each with its own advantages and disadvantages. Different compression algorithms vary in compression ratio, compression speed, and decompression speed, and should be selected based on workload characteristics. | Algorithm | Description | | --------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | LZ4 | Requires the pool to have the `lz4_compress` feature flag enabled (GUID: `org.illumos:lz4_compress`). Once enabled, it becomes the current default compression algorithm. LZ4 processes compressible data about 50% faster than LZJB, processes incompressible data more than three times faster, and decompresses about 80% faster than LZJB. On modern CPUs, LZ4 single-core compression speed typically exceeds 500 MB/s, and decompression speed exceeds 1.5 GB/s | | LZJB | Designed by Jeff Bonwick, one of the founders of ZFS. It is the default compression algorithm on older pools where the LZ4 feature flag is not enabled. LZJB provides good compression with lower CPU overhead than GZIP | | ZSTD | A high-performance compression algorithm (GUID: `org.freebsd:zstd_compress`) that combines high compression ratios with high speed. Compared to GZIP, it provides slightly better compression ratios at higher speeds; compared to LZ4, it provides better compression ratios with only slightly slower speed. The compression level can be specified via `zstd-N` (N=1\~19), where `zstd` is equivalent to `zstd-3`. Fast mode can be specified via `zstd-fast-N`, where N is an integer from `1-10, 20, 30, ..., 100, 500, 1000`, mapping to negative zstd levels; lower levels provide faster compression, and 1000 provides the fastest compression with the lowest compression ratio. `zstd-fast` is equivalent to `zstd-fast-1` | | GZIP | A popular compression algorithm whose main advantage is configurable compression levels. When setting the `compression` property, administrators can choose from `gzip-1` (fastest) to `gzip-9` (best compression ratio) to balance CPU time and disk space. `gzip` is equivalent to `gzip-6` (which is also the default level of gzip(1)) | | ZLE | Zero Length Encoding, which only compresses continuous zero blocks, suitable for datasets containing large numbers of zero blocks | When compression is combined with user quotas, unexpected side effects may occur. User quotas limit the space actually consumed by the user after compression. If a user has a 10 GB quota and writes 10 GB of compressible data, they can still store more data. If they subsequently update a file (such as a database) using more or less compressible data, their available space will change. This can create a strange situation where the user has not increased their actual data volume (`logicalused` property) but hits the quota limit due to changes in compression ratio. Similar unexpected effects can occur with the interaction between compression and backups. Quotas are typically used to limit data storage to ensure sufficient backup space is reserved. Since quotas do not account for compression, ZFS may write more data than an uncompressed backup. ## Deduplication Enabling deduplication can save storage space but requires a large amount of memory to support the deduplication table. Before enabling it, you should evaluate whether the workload truly needs deduplication — compression can provide most of the space savings without additional overhead. Not all data is suitable for deduplication. If the data in the pool has no redundancy, deduplication may not yield savings. ZFS can estimate potential space savings through simulated deduplication (it is recommended to export the pool first to ensure accurate results): ```sh # zdb -S mypool Simulated DDT histogram: bucket allocated referenced ______ ______________________________ ______________________________ refcnt blocks LSIZE PSIZE DSIZE blocks LSIZE PSIZE DSIZE ------ ------ ----- ----- ----- ------ ----- ----- ----- 1 285 35.5M 35.0M 35.0M 285 35.5M 35.0M 35.0M 2 278 34.8M 34.3M 34.3M 834 104M 103M 103M Total 563 70.3M 69.3M 69.3M 1.09K 140M 138M 138M dedup = 1.99, compress = 1.01, copies = 1.00, dedup * compress / copies = 2.02 ``` After `zdb -S` completes its analysis, it displays the potential space savings ratio from enabling deduplication. In this example, the `dedup` value of `2.02` is a relatively high value, but the savings mainly come from compression. If deduplication is enabled on this pool, it will not save any additional space, and the memory overhead required for deduplication would not be worth it. System administrators can use the following formula to plan storage allocation and determine whether the number of duplicate blocks in the workload justifies the corresponding memory investment. $$ \text{Final Storage Efficiency Ratio} = \frac{\dfrac{\text{Logical Space}}{\text{Allocated Space}} \times \dfrac{\text{Uncompressed Size}}{\text{Compressed Size}}}{\text{Copy Factor}} =============================================================================================================================================== \frac{\text{Dedup Ratio} \times \text{Compression Ratio}}{\text{Copy Factor}} $$ If the `dedup` value is close to 2, deduplication is recommended; if `dedup` is close to 1, deduplication is not recommended. If the data has good compressibility, the space savings can be significant. Since compression can also bring significant performance improvements, the best practice is to enable compression first. Only enable deduplication when it provides significant savings and there is sufficient memory to support the DDT. To enable deduplication, set the `dedup` property on the target pool: ```sh # zfs set dedup=on zroot ``` Deduplication only affects new data written to the pool; merely enabling this option does not apply to already written data. A pool with newly enabled deduplication will display as follows: ```sh # zpool list zroot NAME SIZE ALLOC FREE CKPOINT EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT zroot 51.5G 1.21G 50.3G - - 0% 2% 1.00x ONLINE - ``` The `DEDUP` column shows the actual deduplication ratio of the pool. `1.00x` indicates that the data has not been deduplicated. The following example generates a random block file and then copies it to two different filenames to observe the deduplication effect: ```sh # dd if=/dev/urandom of=/zroot/testfile bs=1M count=100 # cp /zroot/testfile /zroot/testfile.copy1 # cp /zroot/testfile /zroot/testfile.copy2 ``` To observe the deduplication effect on redundant data, use: ```sh zpool list zroot NAME SIZE ALLOC FREE CKPOINT EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT zroot 51.5G 1.31G 50.2G - - 0% 2% 2.99x ONLINE - ``` The `DEDUP` column shows a deduplication factor of `2.99x`. Detecting and deduplicating data copies used only one-third of the space. The potential for space savings is enormous, but sufficient memory is needed to track deduplicated blocks. You can also use the `zdb` command: ```sh # zdb -DD zroot DDT-sha256: version=1 [FDT]; flags=0x03 [FLAT LOG]; rootobj=154 DDT-sha256-zap-unique: dspace=221184; mspace=163840; entries=643 DDT-log-sha256-0: flags=0x01 [FLUSHING]; obj=155; len=393216; txg=593; entries=802 DDT-log-sha256-1: flags=0x00; obj=156; len=0; txg=601; entries=0 DDT histogram (aggregated over all DDTs): bucket allocated referenced ______ ______________________________ ______________________________ refcnt blocks LSIZE PSIZE DSIZE blocks LSIZE PSIZE DSIZE ------ ------ ----- ----- ----- ------ ----- ----- ----- 1 4 8.50K 8.50K 16K 4 8.50K 8.50K 16K 2 800 100M 100M 100M 2.34K 300M 300M 300M Total 804 100M 100M 100M 2.35K 300M 300M 300M dedup = 3.00, compress = 1.00, copies = 1.00, dedup * compress / copies = 3.00 ``` ## References * Jin Buguo. GPT Partition Details\[EB/OL]. \[2026-04-02]. . GPT partition basic technical documentation. * Congcong Guoke. How to Easily Change Partition Type ID? Try These 2 Methods!\[EB/OL]. (2022-11-20)\[2026-04-02]. . Introduction to the difference between partition type ID and partition UUID. * FreeBSD Project. zfs -- configures ZFS datasets\[EB/OL]. \[2026-04-14]. . ZFS dataset management tool manual page, covering snapshot, clone, and rollback operations. * FreeBSD Project. zpool -- configures ZFS storage pools\[EB/OL]. \[2026-04-14]. . ZFS storage pool management tool manual page. * FreeBSD Project. FreeBSD Handbook, Chapter 21: The Z File System (ZFS)\[EB/OL]. \[2026-04-14]. . Complete configuration and management guide for ZFS in the FreeBSD Handbook. * OpenZFS Project. zfs-rollback(8)\[EB/OL]. \[2026-04-18]. . Detailed explanation that the -r parameter is used to destroy newer snapshots, not to recursively roll back child datasets. * FreeBSD Project. zfsprops -- native and user-defined properties of ZFS datasets\[EB/OL]. \[2026-04-14]. . ZFS dataset properties manual page, defining compression, compressratio, and other properties. --- # Agent Instructions This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com. ## Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter: ``` GET https://book.bsdcn.org/ask/flat/chapter-28-the-zfs-file-system/di-28.5-jie-zfs-guan-li.md?ask= ``` The question should be specific, self-contained, and written in natural language. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.