Ctrlk
VitePress 镜像站QQ 群 787969044视频教程Ⅰ视频教程Ⅱ
discord
For the complete documentation index, see llms.txt. This page is also available as Markdown.

28.5 ZFS Administration

Creating and Destroying Datasets

Unlike traditional disk and volume managers, ZFS does not pre-allocate space. Traditional file systems cannot create new file systems after partitioning and allocating space without adding new disks. ZFS allows creating new file systems at any time. Each dataset possesses features such as compression, deduplication, caching, and quotas, along with practical attributes like read-only, case sensitivity, network file sharing, and mount points. Datasets can be nested, and child datasets inherit properties from their parent datasets. Each dataset can be delegated for management, replicated, snapshotted, included in a jail, or directly destroyed. Creating a separate dataset for each file type or file collection is a recommended practice. However, having too many datasets also has drawbacks: commands like zfs list will slow down, and mounting hundreds or even thousands of datasets will also slow down FreeBSD boot time.

View the current datasets:

# zfs list
NAME                 USED  AVAIL  REFER  MOUNTPOINT
zroot                916M  49.0G    96K  /zroot
zroot/ROOT           886M  49.0G    96K  none
zroot/ROOT/default   886M  49.0G   886M  /
zroot/home          27.8M  49.0G    96K  /home
zroot/home/ykla     27.7M  49.0G  27.7M  /home/ykla
zroot/tmp            128K  49.0G   128K  /tmp
zroot/usr            288K  49.0G    96K  /usr
zroot/usr/ports       96K  49.0G    96K  /usr/ports
zroot/usr/src         96K  49.0G    96K  /usr/src
zroot/var            636K  49.0G    96K  /var
zroot/var/audit       96K  49.0G    96K  /var/audit
zroot/var/crash       96K  49.0G    96K  /var/crash
zroot/var/log        156K  49.0G   156K  /var/log
zroot/var/mail        96K  49.0G    96K  /var/mail
zroot/var/tmp         96K  49.0G    96K  /var/tmp

Tip

The username ykla and path /home/ykla appearing in the examples in this section are for demonstration purposes. Please replace them with your actual username and home directory.

Create a new dataset and enable LZ4 compression:

Since this operation does not involve scanning files and updating corresponding metadata, destroying a dataset is much faster than deleting files within the dataset.

Warning

zfs destroy will permanently delete the dataset and all its data, and this action cannot be undone. Please make sure the target is correct. If necessary, use zfs destroy -n -v first to preview the datasets and snapshots that will be destroyed.

Destroy the created dataset:

In modern versions of ZFS, zfs destroy is asynchronous, and the freed space may take a few minutes to be reflected in the pool. Use zpool get freeing poolname to check the freeing property:

This property shows which datasets are currently releasing their blocks in the background. If there are child datasets (such as snapshots or other datasets), the parent dataset cannot be directly destroyed. To destroy a dataset and all its child datasets, use -r for recursive destruction. Using -n -v will list the datasets and snapshots that would be destroyed without actually destroying any data. When destroying snapshots, the reclaimed space will also be displayed.

Creating and Destroying Volumes

A volume is a special type of dataset. It is not mounted as a file system, but is exposed as a block device at the path /dev/zvol/poolname/dataset. Therefore, volumes can be used for other file systems, serve as virtual machine disks, or be provided to other hosts on the network through protocols such as iSCSI and HAST.

Volumes can be formatted and used with any file system, or they can store raw data without formatting. To the user, a volume is no different from a regular disk. Placing a regular file system on these zvols provides capabilities that regular disks or file systems do not have. For example, enabling the compression property on a 250 MB volume allows creating a compressed FAT file system.

Create a 3 GB ZFS volume and enable compression:

Tip

If the FAT32 capacity is set too low, it will not be possible to format properly due to insufficient file clusters.

Confirm the volume has been created and check its space usage:

The volume appears as a block device under the /dev/zvol/ path:

Warning

newfs_msdos will format the specified ZFS volume, and existing data on the volume will be permanently lost. Please confirm the device path /dev/zvol/ is correct to avoid accidentally formatting other devices.

Format the volume as a FAT32 file system:

Mount the volume and confirm the mount succeeded:

Copy some files for testing:

Destroying a volume is similar to destroying a regular file system dataset:

This operation completes almost instantly, but free space may take a few minutes to be reclaimed in the background.

Renaming Datasets

To change the name of a dataset, use the zfs rename command. To change the parent of a dataset, use the same command. Renaming a dataset to a different parent will change the property values it inherits from its parent. When renaming a dataset, it is unmounted and mounted at the new location (which is inherited from the new parent). To prevent this behavior, use the -u option.

Rename a dataset and move it to a different parent:

First, create a dataset for demonstration and confirm its location:

Create a snapshot for the dataset to demonstrate that snapshots are preserved after renaming:

Execute the rename, moving the dataset from zroot/usr to zroot/var:

Snapshots cannot change their parent. Create another snapshot to verify:

Renaming snapshots also uses the same command.

To recursively rename snapshots, use the -r option, which renames snapshots with the same name in all child datasets. -r also applies to recursively renaming a dataset and all its child datasets.

Setting Dataset Properties

Each ZFS dataset has a set of properties that control its behavior. Most properties are inherited from the parent dataset by default but can be individually overridden. Use the zfs set command to set dataset properties, with the syntax <property=value dataset>. Most properties have a limited range of valid values, and zfs get lists each property and its valid values. Use zfs get all <dataset> to view all current properties of a dataset (including inherited and locally set values), which is useful for troubleshooting and status confirmation. Use zfs inherit to restore most properties to their inherited values. Additionally, user-defined custom properties can be defined, which become part of the dataset configuration and can provide additional information about the dataset or its contents. To distinguish custom properties from ZFS built-in properties, use a colon (:) to create a custom namespace.

Set a custom property and view it:

To delete a custom property, use zfs inherit with the -r option. If the custom property is not defined in any parent dataset, this option will remove it (but the pool history will still record the change).

Delete the custom property and verify:

Getting and Setting Share Properties

Two commonly used and practical dataset properties are NFS and SMB share options. Setting these properties defines whether and how ZFS shares the dataset over the network. FreeBSD supports both NFS (sharenfs) and SMB (sharesmb) share properties, where SMB sharing requires Samba to be installed. Using NFS as an example, to get the current state of sharing, enter:

To enable sharing for the dataset, enter:

Additional options can be set for datasets shared via NFS, such as -alldirs, -maproot, and -network. To set share options, enter:

Warning

-maproot=root maps the NFS client root user to the server root, giving the remote root user full read-write access to the shared dataset. NFS enables root_squash by default (mapping client root to nobody), and using -maproot=root effectively disables this security mechanism. It is recommended to use this only in trusted internal networks, or switch to -maproot=nobody to restrict permissions, or map to a specified non-privileged user (e.g., -maproot=1000).

Managing Snapshots

ZFS snapshots use the copy-on-write (COW) mechanism, making creation instantaneous without consuming additional disk space. After a snapshot is created, when data blocks in the original file system are modified, ZFS writes the new data to a new location while the old data blocks remain in place, thus capturing the point-in-time state of the file system at the moment the snapshot was created. Snapshots provide a read-only, point-in-time copy of a dataset. Snapshots operate on the entire dataset rather than individual files, consuming no additional space at creation time, and consuming space as the blocks they reference change.

A typical use of snapshots is to quickly back up the current file system state before performing high-risk operations such as software installation or system upgrades. If the operation fails, you can roll back to the snapshot to restore the system state; if the upgrade succeeds, delete the snapshot to free space. Snapshot rollback is fast and causes almost no downtime. Snapshots cannot replace full pool backups, but they are extremely efficient for saving dataset copies at specific points in time.

The Chinese FreeBSD Community (CFC) provides ZFS scripts that can be used to view, create, delete, and restore ZFS snapshots. ZFS Script Project Repository. The script has been deployed at https://docs.bsdcn.org/zfs.sh and can be downloaded directly on a FreeBSD system using the fetch command.

Creating Snapshots

By default, the partition structure created using the Auto ZFS layout is as follows:

To create a snapshot, use the zfs snapshot <dataset>@<snapshotname> command. Adding the -r option creates snapshots recursively with the same name on all child datasets.

Create a recursive snapshot of the entire pool:

Snapshots are not displayed in normal zfs list output. To list snapshots, the -t snapshot option must be added to the zfs list command. -t all displays both file systems and snapshots.

Tip

In commands, snapshot can be abbreviated as snap. Other commands also have corresponding abbreviations, which you can look up in the documentation.

Snapshots are not directly mounted, so no path is displayed in the MOUNTPOINT column. Since snapshots are read-only after creation, ZFS does not display available space in the AVAIL column. You can compare snapshots with the original dataset using the following command:

Displaying both datasets and snapshots simultaneously reveals how snapshots work with copy-on-write (COW). They only save the changed portions (delta), rather than re-saving the entire file system contents. This means snapshots consume minimal space when changes occur.

By copying files to the dataset and then creating a second snapshot, you can more clearly observe the space usage:

View the space usage changes of each snapshot after creating the second snapshot:

The second snapshot only contains the changes to the dataset after the copy operation, thus greatly saving space.

Note

The size of snapshot zroot/var/tmp@test has also changed in the USED column, which reflects the difference between it and the snapshots created after it.

Destroying Snapshots

When destroying snapshots, the -r parameter can be used for recursive deletion:

Snapshot Holds

Sometimes it is necessary to prevent critical snapshots from being accidentally deleted — for example, before a backup process is complete, or when specific point-in-time copies must be retained due to regulatory requirements. ZFS hold mechanism applies named lightweight references to snapshots, preventing them from being deleted by zfs destroy until the hold is released.

Add a hold tag to a snapshot:

Tip

keeptest is just a label for easy identification and can be changed to other names.

View all holds on a snapshot:

Attempting to destroy a held snapshot returns an error:

Use the -r option to recursively apply holds to snapshots with the same name across all child datasets:

After releasing the hold tag, the snapshot can be destroyed normally:

The -h option of zfs send can include hold tags in the send stream. This is particularly useful for maintaining backup integrity in disaster recovery environments.

When the receiving side uses zfs receive to accept the stream, the snapshot holds will be automatically reconstructed on the receiver.

Comparing Snapshots

ZFS provides built-in commands for comparing differences between two snapshots. For scenarios involving long-term storage of numerous snapshots, this feature is extremely practical, allowing users to see how the file system has changed over time. For example, zfs diff can help users find the most recent snapshot and check whether it still contains a mistakenly deleted file. Comparing the two snapshots created in the previous sections yields the following output:

This command lists the changes between the specified snapshot (here zroot/var/tmp@test) and the current file system.

The first column shows the type of change:

Symbol
Function

+

Path or file added

-

Path or file deleted

M

Path or file modified

R

Path or file renamed

Comparing the output with the symbols above, ZFS added the COPYRIGHT file after creating the zroot/var/tmp@test snapshot, which also caused the parent directory mounted at /var/tmp to be modified.

Comparing two snapshots is very useful when using ZFS replication to transfer datasets to different hosts for backup.

Compare two snapshots by specifying the full dataset name and two snapshot names:

Backup administrators can compare two snapshots received from the sending host and determine the actual changes in the dataset.

Snapshot Rollback

As long as at least one snapshot exists, you can roll back to it at any time. The most common scenarios for rollback are: the current dataset state has become invalid, or an older version is more suitable. For example, local development testing errors, system update failures causing functionality damage, or the need to recover deleted files or directories — these situations are all common. To roll back to a snapshot, use the zfs rollback <snapshotname> command. If the amount of changes is large, the operation may take longer. During this period, the dataset remains in a consistent state, similar to a database performing a rollback following ACID principles. The entire process completes while the dataset is online and requires no downtime. After rollback, the dataset state is restored to the state at the time the snapshot was created. Rolling back to a snapshot discards all data in the dataset that does not belong to that snapshot. If you create a snapshot of the current state before rolling back to an earlier snapshot, you can easily roll back again if you later need some data. This way, users can switch between snapshots without losing data that still has value.

Snapshot Restore Testing

You can verify the effectiveness of snapshots by adding and deleting files. In a test environment, if snapshots have been created beforehand, even executing rm -rf /* can be successfully recovered; if the system uses UEFI, you need to restore the EFI boot according to the instructions in other chapters.

Note

This operation should only be performed in a test environment.

Suppose in the previous example, an accidental rm command deleted more data than expected, and you need to roll back to a snapshot:

First, view the currently available snapshots:

View the files in the current directory:

Simulate an accidental deletion by removing COPYRIGHT-related files:

At this point, the user realizes that extra files were accidentally deleted and wants to recover them. ZFS provides a simple recovery method: just create snapshots of important data regularly. After rolling back to the previous snapshot, you can retrieve the lost files and start over from that point:

Execute the rollback operation to restore the dataset to the state of the diff snapshot:

The rollback operation restores the dataset to the state of the last snapshot.

Confirm that the snapshot list remains unchanged after rollback:

You can also roll back to an earlier snapshot, even if other snapshots exist after it. When attempting this, ZFS displays the following warning:

This warning indicates that other snapshots exist between the target snapshot and the current dataset state. To complete the rollback, these snapshots must be deleted.

Unlike virtual machine snapshots, by default, the zfs rollback command can only roll back to the most recent snapshot (Reference Manual, Oracle official ZFS rollback command documentation). Since snapshots are read-only, ZFS cannot track all changes to a dataset across different states unless the user uses the -r option to confirm this is the desired operation and destroy all snapshots newer than the target snapshot. After using the -r option, ZFS will destroy all snapshots newer than the target snapshot, thus allowing rollback to a non-latest snapshot.

If this is indeed the intended operation and the user understands the consequences of deleting all intermediate snapshots, the following command can be executed:

From the zfs list -t snapshot output, you can confirm that after executing zfs rollback -r, the intermediate snapshots have been deleted.

ZFS does not support recursively rolling back all child datasets at once; you must execute the rollback operation separately for each child file system.

Exercise

Explore better snapshot rollback solutions. You can refer to the Chinese community ZFS Script Project, or submit feature requests and Pull Requests to the OpenZFS project.

Recovering Individual Files from Snapshots

Snapshots are stored in a hidden directory under the parent dataset at .zfs/snapshots/snapshotname.

By default, these directories are not displayed even when executing the ls -a command. Although invisible, they can still be accessed like regular directories.

The snapdir property controls whether these hidden directories appear in directory listings. When this property is set to visible, they will appear in the output of ls and other commands involving directory contents.

View the current setting of the snapdir property, then switch it to visible:

By copying files from the snapshot back to the parent dataset, individual files can be restored to their previous state. The directory structure under .zfs/snapshot contains directories named after the previously created snapshots for easy identification. The following example shows how to recover files from the hidden .zfs directory — copying files back from the snapshot that contains the target files:

After deleting a file, find and recover it through the .zfs/snapshot directory:

Even if the snapdir property is restored to the default hidden, executing ls .zfs/snapshot will still list the directory contents. Administrators can decide whether to display these directories on their own; this setting applies to each dataset.

After restoring snapdir to hidden, snapshots can still be accessed directly via the path:

Copying files or directories from the hidden .zfs/snapshot directory is very simple. Conversely, when trying to copy files to the snapshot directory, the following error occurs:

Attempt to write a file to the snapshot directory to verify the read-only nature of snapshots:

This error reminds users that snapshots are read-only and cannot be changed after creation. Since copying or deleting files to/from the snapshot directory would change the dataset state represented by the snapshot, these operations cannot be performed.

Snapshots consume space as changes occur in the parent file system since the snapshot was created. The written property of a snapshot tracks the space consumed by that snapshot.

To destroy a snapshot and reclaim space, use the zfs destroy <dataset>@<snapshot> command. Adding the -r option recursively deletes all snapshots with the same name under the parent dataset. Using the -n -v options, the command lists the snapshots that will be deleted and their estimated reclaimed space without actually performing the destruction.

Managing Clones

A clone is a writable copy of a snapshot that is writable, mountable, and has its own properties. After creating a clone with zfs clone, the original snapshot cannot be destroyed. To reverse the parent-child relationship between a clone and a snapshot, use zfs promote. After promoting a clone, the snapshot becomes a child of the clone, and the space accounting under the original parent-child relationship changes accordingly. Clones can be mounted anywhere in the ZFS file system hierarchy.

Creating Clones

Here is a sample dataset demonstrating clone functionality:

A typical use of clones is to experiment with a specific dataset while preserving the snapshot, so that you can recover if something goes wrong. Since snapshots cannot be changed, you can create a read-write clone of the snapshot. After the experiment yields satisfactory results, you can promote the clone to a dataset and delete the initial file system. Since clones and datasets can coexist without causing problems, deleting the original dataset is not necessary.

Create a clone from a snapshot and verify that the clone has the same content as the original dataset:

View the disk usage of the clone and the original dataset; they share the same data blocks:

Promoting Clones

When a clone is created, it is a complete copy of the dataset at the time the snapshot was taken. After that, the clone can change independently of the original dataset. The two are linked by the snapshot, and ZFS records this association in the origin property. After using zfs promote to promote the clone, it becomes an independent dataset, the origin property value is cleared, and the association between the clone and the snapshot is also severed. The following example demonstrates this process:

Before promotion, view the clone origin property to confirm it originates from the snapshot:

Execute the promotion:

After promotion, the origin property is cleared, and the clone becomes an independent dataset:

After making some changes (such as copying COPYRIGHT to the promoted clone), the old directory is now outdated. At this point, the promoted clone can be used to replace it. To accomplish the replacement, first use zfs destroy to delete the old dataset, then use zfs rename to rename the clone to the old dataset name (or a completely different name).

Add a new file to the clone:

Delete the original dataset:

Rename the clone to the original dataset name to complete the replacement:

At this point, the dataset cloned from the snapshot has become a regular dataset. It contains all the data from the original snapshot plus the newly added file (such as COPYRIGHT). Clones provide ZFS users with practical functionality in various scenarios. For example, you can provide jails with snapshots containing different application sets, and users can clone these snapshots and add their own applications as needed. Once satisfied with the changes, the clone can be promoted to a full dataset and delivered to end users, who can use it as a regular dataset. This greatly saves the time and management overhead required to provide jails.

Replication

Storing data only in a single location within a single pool exposes it to risks such as theft, natural disasters, or human error. Therefore, regular backups of the entire pool are essential. ZFS provides built-in serialization functionality that can send a stream representation of data to standard output. With this feature, data can be stored to another pool connected to the local system or sent over the network to another system. Snapshots are the foundation of this replication. The commands used to replicate data are zfs send and zfs receive.

Create the required test storage pools:

Copy some non-critical files for testing:

The following example shows how to use these two pools to complete ZFS replication:

The storage pool mypool is the primary pool used for reading and writing regular data. The second pool backup serves as a backup in case the primary pool becomes unavailable.

Note that ZFS does not perform automatic failover; administrators must manually intervene when necessary. Using snapshots provides a consistent version of the file system for replication and recovery.

After creating a snapshot for mypool, it can be transferred to the backup pool by replicating the snapshot. This operation does not include changes made since the last snapshot.

Confirm the snapshot has been created:

Now that a snapshot exists, you can use zfs send to create a stream representing the snapshot contents. Store this stream as a file, or receive it on another pool.

The stream must be redirected to a file or pipe, otherwise the following error will occur:

Creating a Backup Stream

To back up a dataset, use zfs send to redirect the stream to a file on a mounted backup pool.

Make sure the pool has enough available space to accommodate the size of the snapshot being sent (the total amount of data contained in the snapshot, not the difference from the previous snapshot).

View the space usage changes of both pools after sending:

zfs send has transferred all data from snapshot backup1 to the storage pool backup. To automatically create and send these snapshots, you can configure cron jobs.

Receiving a Backup Stream

Rather than storing as an archive file, ZFS can receive data as an active file system, allowing direct access to backup data. To access the actual data in these streams, use zfs receive to restore the stream into files and directories.

The following example combines zfs send with zfs receive, piping data from pool mypool to another pool backup.

First, create a snapshot for mypool:

After the transfer is complete, the data can be used directly on the receiving pool. You can only replicate to an empty dataset.

Execute a full send and receive:

View the space usage of both pools after receiving:

View the backup directory structure:

Verify that the backup file matches the source file:

Incremental Backup

zfs send can also determine the differences between two snapshots and send only the incremental changes between them. This saves both disk space and transmission time.

Copy some more non-critical files to the mypool pool for testing:

For example:

View the REFER value changes across the three snapshots:

View the capacity of both pools before incremental replication:

The second snapshot replica2 has been created above. This snapshot contains all changes to the file system since the previous snapshot replica1.

Using zfs send -i and specifying the pair of snapshots generates an incremental replication stream containing only the changed data. This operation succeeds if the initial snapshot already exists on the receiving side.

After the incremental send, view the space usage of both pools — the backup pool only increased by the differential data amount:

List both pools and their datasets to view space allocation:

View the snapshot list in the backup pool:

The incremental stream only copied the changed data, not the entire replica1. Sending only the differences significantly reduces transmission time and saves disk space by avoiding copying the entire pool each time. This advantage is particularly notable when replicating over slow networks or networks that charge by the byte transferred.

At this point, the new file system backup/mypool is available, containing the data and files from the mypool pool.

Confirm that the incremental backup file exists in the backup pool:

The -p option copies dataset properties, including compression settings, quotas, and mount points. The -R option replicates all child datasets of the dataset along with their properties. The send and receive process can be automated to create backups on a second pool on a regular basis.

Sending Encrypted Backups via SSH

Sending stream data over the network is an effective way to maintain remote backups, but it has a drawback: the data sent over the network link is unencrypted, and anyone can intercept the stream data and restore it to the original data without the sender authorization. This is unacceptable when sending data over the internet to a remote host. To ensure secure data transmission, SSH can be used to encrypt the sent data. Since ZFS requires the stream to be redirected from standard output, it can be easily piped through SSH. To ensure that file system contents remain encrypted at rest on the remote system, ZFS native encryption is recommended. Create an encrypted dataset on the receiving side:

The received stream data will be automatically written to disk in encrypted form. ZFS native encryption has been mature and available since OpenZFS 0.8.0, seamlessly integrated with ZFS snapshots and send/receive, without requiring additional kernel modules.

First, make some configuration changes and take security precautions. The following describes the steps required to perform zfs send operations:

Configuration changes are as follows:

  • Use SSH keys for passwordless SSH access between the sending and receiving hosts

  • ZFS requires root user privileges to send and receive streams. This requires logging in to the receiving system as root.

  • For security reasons, root login is disabled by default.

  • Use the ZFS delegation system to allow non-root users on each system to perform the corresponding send and receive operations. On the sending system:

  • To mount the pool, the non-privileged user must own the directory, and regular users need permission to mount file systems.

On the receiving system:

Now, non-privileged users can also receive and mount datasets, and copy the home dataset to the remote system. It is recommended to use an IP address or fully qualified domain name. The receiver writes data to the backup dataset on the recvpool pool.

First, create a recursive snapshot monday for the file system dataset home on the mypool pool.

Then zfs send -R includes the dataset, all child datasets, snapshots, clones, and settings in the stream.

The output is piped through SSH to zfs receive waiting on the remote host backuphost.

In zfs recv:

  • The -d option overrides the receiving pool name with the snapshot name;

  • The -u option prevents the file system from being mounted on the receiving side;

  • The -v option displays more transfer details, including elapsed time and data volume.

Bookmarks

Bookmarks are lightweight references to snapshots that record the point-in-time position when the snapshot was created. Unlike snapshots, bookmarks do not hold data blocks, so they consume no additional disk space after creation and do not consume space as data changes, unlike snapshots. The typical use of bookmarks is as a source for zfs send incremental streams, enabling incremental replication without a complete snapshot.

The bookmark feature requires the bookmarks feature flag to be enabled on the storage pool. In modern versions of ZFS, this flag is enabled by default.

Creating Bookmarks

Use the zfs bookmark command to create a bookmark for snapshot zroot/var/tmp@test:

You can also create a bookmark based on an existing bookmark:

Unlike snapshots which use @ as a separator, bookmarks use # to separate the dataset name from the bookmark name.

Listing Bookmarks

Bookmarks are not displayed in normal zfs list output. To list bookmarks, use the -t bookmark option:

Since bookmarks consume no space and cannot be mounted, the USED, AVAIL, and MOUNTPOINT columns all display -. The REFER column shows the amount of data referenced by the dataset at the time the bookmark was created.

Using Bookmarks for Incremental Send

The main value of bookmarks lies in incremental replication. If the snapshot on the source system has been destroyed but an incremental stream still needs to be sent to the backup system, bookmarks can replace snapshots as the incremental source:

Administrators can therefore continue the incremental backup chain even after destroying intermediate snapshots, greatly saving storage space.

Destroying Bookmarks

Use the zfs destroy command to destroy bookmarks, with syntax similar to destroying snapshots but using the # separator:

After a bookmark is destroyed, if it was the only source for an incremental send chain, subsequent incremental sends will not be able to use that bookmark as a baseline.

Bookmarks and Rollback

When attempting to roll back to a non-latest snapshot, ZFS detects whether newer snapshots and bookmarks exist and prompts the user to use the -r option to force deletion:

Although bookmarks do not hold data, they represent a point-in-time reference and therefore prevent rollback to snapshots earlier than them unless the user explicitly confirms the need to delete these bookmarks.

Dataset, User, and Group Quotas

Dataset quotas are used to limit the space a specific dataset can consume. Reference quotas are similar but only count the space used by the dataset itself, excluding snapshots and child datasets. Similarly, user quotas and group quotas prevent users or groups from exhausting all the space in a pool or dataset.

Dataset Quotas

Home directory datasets for new users are typically automatically created as zroot/home/username, with mountpoint generally set to /home/username.

The example assumes the system has a user ykla. As follows:

To enforce a 1 GB dataset quota on zroot/home/ykla:

To enforce a 2 GB reference quota on zroot/home/ykla:

To remove the 1 GB quota on zroot/home/ykla:

User Quotas

The general format is userquota@user=size dataset_or_pool, and the user name can be in any of the following formats:

Format
Example

POSIX-compatible name

ykla

POSIX numeric ID

789

SID name

ykla@example.com

SID numeric ID

S-1-123-456-789

For example, to enforce a 50 GB user quota for user ykla:

User quota properties are not displayed by zfs get all. Non-root users cannot view other users quotas unless granted the userquota privilege. Users with this privilege can view and set quotas for everyone.

To view user quotas:

To remove all quotas:

Confirm the user quota has been cleared:

Group Quotas

The general format for setting group quotas is: groupquota@group=size dataset_or_pool.

To set the quota for group ykla to 50 GB, use:

Similar to user quota properties, non-root users can view quotas for groups they belong to. Users with the groupquota privilege or root can view and set quotas for all groups.

Confirm the group quota is in effect:

To remove the quota for group ykla, or to ensure no quota is set, use:

Confirm the group quota has been cleared:

Viewing Quota Usage

To display the space used by each user in a file system or snapshot along with all quotas, use zfs userspace. For group information, use zfs groupspace.

Privileged users and root can list the quota for zroot/home/ykla by:

Reservations

Reservations ensure that a dataset always has available space (the reserved space is not allocated to any other dataset), which is very practical for guaranteeing space for critical datasets or log files.

The format of the reservation property is reservation=size. The following command sets a 10 GB reservation for zroot/home/ykla:

The following command displays the existing reservation on zroot/home/ykla:

To clear all reservations:

Confirm the reservation has been cleared:

Reference Reservation

The refreservation property is used to set a reference reservation, applying the same principles, with the format refreservation=size.

The following command sets a 1 GB reference reservation for zroot/home/ykla:

The following command displays the existing reference reservation on zroot/home/ykla:

To clear all reference reservations:

Confirm the reference reservation has been cleared:

Compression

ZFS data compression is implemented at the file system level and is transparent to upper-layer applications. When enabled, it typically reduces disk usage while improving read/write throughput.

Administrators can view compression effectiveness through dataset properties.

Set the data compression algorithm for the zroot file system to zstd level 5:

Note

Compression property changes take effect immediately without restarting the system. However, this property only applies to newly written data and does not automatically compress existing data.

List the data compression properties and their current settings for each ZFS file system again:

View the actual data compression ratio for each ZFS file system:

Tip

compressratio represents the ratio of compressed data to uncompressed data. For example, 2.70x means the data is compressed to approximately 37% of its original size.

ZFS provides different compression algorithms, each with its own advantages and disadvantages. Different compression algorithms vary in compression ratio, compression speed, and decompression speed, and should be selected based on workload characteristics.

Algorithm
Description

LZ4

Requires the pool to have the lz4_compress feature flag enabled (GUID: org.illumos:lz4_compress). Once enabled, it becomes the current default compression algorithm. LZ4 processes compressible data about 50% faster than LZJB, processes incompressible data more than three times faster, and decompresses about 80% faster than LZJB. On modern CPUs, LZ4 single-core compression speed typically exceeds 500 MB/s, and decompression speed exceeds 1.5 GB/s

LZJB

Designed by Jeff Bonwick, one of the founders of ZFS. It is the default compression algorithm on older pools where the LZ4 feature flag is not enabled. LZJB provides good compression with lower CPU overhead than GZIP

ZSTD

A high-performance compression algorithm (GUID: org.freebsd:zstd_compress) that combines high compression ratios with high speed. Compared to GZIP, it provides slightly better compression ratios at higher speeds; compared to LZ4, it provides better compression ratios with only slightly slower speed. The compression level can be specified via zstd-N (N=1~19), where zstd is equivalent to zstd-3. Fast mode can be specified via zstd-fast-N, where N is an integer from 1-10, 20, 30, ..., 100, 500, 1000, mapping to negative zstd levels; lower levels provide faster compression, and 1000 provides the fastest compression with the lowest compression ratio. zstd-fast is equivalent to zstd-fast-1

GZIP

A popular compression algorithm whose main advantage is configurable compression levels. When setting the compression property, administrators can choose from gzip-1 (fastest) to gzip-9 (best compression ratio) to balance CPU time and disk space. gzip is equivalent to gzip-6 (which is also the default level of gzip(1))

ZLE

Zero Length Encoding, which only compresses continuous zero blocks, suitable for datasets containing large numbers of zero blocks

When compression is combined with user quotas, unexpected side effects may occur. User quotas limit the space actually consumed by the user after compression. If a user has a 10 GB quota and writes 10 GB of compressible data, they can still store more data. If they subsequently update a file (such as a database) using more or less compressible data, their available space will change. This can create a strange situation where the user has not increased their actual data volume (logicalused property) but hits the quota limit due to changes in compression ratio.

Similar unexpected effects can occur with the interaction between compression and backups. Quotas are typically used to limit data storage to ensure sufficient backup space is reserved. Since quotas do not account for compression, ZFS may write more data than an uncompressed backup.

Deduplication

Enabling deduplication can save storage space but requires a large amount of memory to support the deduplication table. Before enabling it, you should evaluate whether the workload truly needs deduplication — compression can provide most of the space savings without additional overhead.

Not all data is suitable for deduplication. If the data in the pool has no redundancy, deduplication may not yield savings. ZFS can estimate potential space savings through simulated deduplication (it is recommended to export the pool first to ensure accurate results):

After zdb -S completes its analysis, it displays the potential space savings ratio from enabling deduplication.

In this example, the dedup value of 2.02 is a relatively high value, but the savings mainly come from compression. If deduplication is enabled on this pool, it will not save any additional space, and the memory overhead required for deduplication would not be worth it.

System administrators can use the following formula to plan storage allocation and determine whether the number of duplicate blocks in the workload justifies the corresponding memory investment.

Final Storage Efficiency Ratio=Logical SpaceAllocated Space×Uncompressed SizeCompressed SizeCopy Factor=Dedup Ratio×Compression RatioCopy Factor\text{Final Storage Efficiency Ratio} = \frac{\dfrac{\text{Logical Space}}{\text{Allocated Space}} \times \dfrac{\text{Uncompressed Size}}{\text{Compressed Size}}}{\text{Copy Factor}} = \frac{\text{Dedup Ratio} \times \text{Compression Ratio}}{\text{Copy Factor}}

If the dedup value is close to 2, deduplication is recommended; if dedup is close to 1, deduplication is not recommended.

If the data has good compressibility, the space savings can be significant.

Since compression can also bring significant performance improvements, the best practice is to enable compression first. Only enable deduplication when it provides significant savings and there is sufficient memory to support the DDT.

To enable deduplication, set the dedup property on the target pool:

Deduplication only affects new data written to the pool; merely enabling this option does not apply to already written data.

A pool with newly enabled deduplication will display as follows:

The DEDUP column shows the actual deduplication ratio of the pool. 1.00x indicates that the data has not been deduplicated.

The following example generates a random block file and then copies it to two different filenames to observe the deduplication effect:

To observe the deduplication effect on redundant data, use:

The DEDUP column shows a deduplication factor of 2.99x. Detecting and deduplicating data copies used only one-third of the space. The potential for space savings is enormous, but sufficient memory is needed to track deduplicated blocks.

You can also use the zdb command:

References

Previous28.4 Updating ZFS Storage PoolsNext28.6 ZFS Tuning

Last updated