Restore Your Linux System Instantly with ZFS Snapshots!

Do you ever find yourself in a situation where you need to restore your Linux system to its previous state? It can be quite a challenge, especially when you have a complex system with lots of configurations and data. Fortunately, there is an easy and efficient solution for this problem—ZFS Snapshots!

In this blog, I’ll tell you all about ZFS Snapshots, their benefits, how to create them, how to rollback to a snapshot, and the best practices for managing and automating them. So, let’s get started!

Introduction to ZFS Snapshots

ZFS Snapshots are a feature of the ZFS file system, which is a highly advanced file system for Linux systems. There’s an ongoing debate on the inclusion of ZFS in the Linux kernel due to some potential licensing issues. I am not going to discuss about that; just be aware of the situation and check if your desired linux distribution supports ZFS natively (Ubuntu does, at least on 22.10 which I’m currently using), if not you’ll need to look at ZFS on Linux and do some manual steps to use ZFS. ZFS Snapshots are used to “freeze” the state of a file system at a given point in time, allowing you to quickly restore your system to that point in time if something goes wrong. ZFS Snapshots can be used for a variety of purposes, such as system recovery, data protection, and system migration. In this blog, we’ll focus on the use of ZFS Snapshots for system recovery.

Benefits of Using ZFS Snapshots

The main benefit of using ZFS Snapshots is that they allow you to restore your system to a previous state in a matter of seconds. This is especially useful in cases where you need to recover from a system crash or other unforeseen events. A good use case example is the following: you install a new package but that screws your system. If you have a snapshot strategy in place, you can restore your system in minutes (I had this in January 2023, while trying to install the displaylink driver, and restored my machine in less than five minutes).

Another great benefit of ZFS Snapshots is that they are very efficient. Since they only store the differences between the current state and the snapshot state, they are much more space-efficient than traditional backups. Said so, snapshots are not a substitute for backups but rather an integration. Keep backing up your important files!

Finally, since ZFS Snapshots are taken at a file system level, they can capture all of your system configurations and data, making them a great tool for system recovery.

What are ZFS Snapshots?

ZFS Snapshots are “frozen” versions of your file system, taken at a given point in time. Each snapshot captures the state of all of your system configurations and data, so that if something goes wrong, you can quickly restore your system to the state it was in when the snapshot was taken.

ZFS Snapshots are stored on the same file system as the original files, so they are highly space-efficient. They also have the added benefit of being able to capture the state of all of your system configurations, including any software that was installed at the time of the snapshot.

How to Create ZFS Snapshots

Creating ZFS Snapshots is relatively straightforward. The first step is to open a terminal window and navigate to the file system that you want to take a snapshot of. Then, you can use the “zfs snapshot” command to create a snapshot. For example:

zfs snapshot my_filesystem@snapshot_name

This will create a snapshot of your file system, with the name “snapshot_name”. You can take multiple snapshots of the same file system with different names, if you want to keep track of different points in time.

How to Rollback to a ZFS Snapshot

Rolling back to a ZFS Snapshot is just as easy as creating one. To do this, you simply use the “zfs rollback” command. For example:

zfs rollback my_filesystem@snapshot_name

This will restore your file system to the state it was in when the snapshot was taken. This is a great way to quickly restore your system to a previous state if something goes wrong.

ZFS Snapshot Management

Once you’ve created a few ZFS Snapshots, you’ll want to know how to manage them. Fortunately, ZFS has a few built-in tools that make managing snapshots easy.

First, you can use the zfs list -t snapshot command to list all of your snapshots. This will give you a list of all of your snapshots, along with their names and dates.

You can also use the zfs diff command to compare two snapshots. This will show you all of the differences between the two snapshots, so you can easily see what has changed since the snapshot was taken.

Finally, you can use the zfs destroy command to delete a snapshot. This is useful if you want to free up disk space or if you no longer need a particular snapshot.

ZFS Snapshot Best Practices

When using ZFS Snapshots, there are a few best practices that you should follow. The first is to only take snapshots when your system is in a stable state. Otherwise, you may end up with an inconsistent snapshot that won’t be useful for restoring your system.

It’s also a good idea to keep track of your snapshots. You can do this by giving each snapshot a unique name that describes its purpose. This will make it easier to find the snapshot you need if you ever need to restore your system.

Finally, it’s a good idea to regularly delete old snapshots. This will help to keep your file system free of clutter and ensure that you have enough disk space for new snapshots.

How to Automate ZFS Snapshots

If you want to make sure that you always have a recent snapshot of your system, you can automate the process of creating ZFS Snapshots. There are different options you can chose from, personaly I use a modified version of “zfs-auto-snapshot”.

The original version of zfs-auto-snapshot is available here: GitHub - zfsonlinux/zfs-auto-snapshot: ZFS Automatic Snapshot Service for Linux, while you can find the customized version I use here: GitHub - dzacca/zfs-auto-snapshot: ZFS Automatic Snapshot Service for Linux

The “zfs-auto-snapshot” script allows you to specify a schedule for creating snapshots. You can specify how often the snapshots should be taken, as well as how many snapshots should be kept at any given time. This makes it easy to ensure that you always have a recent snapshot of your system.

In my modified version I added an automation to create a snapshot everytime apt makes changes to packages (install, upgrade, remove).

An example on how to restore a corrupted system

This is a real example of what I did in January 2023 to restore my system after the issue I had with displaylink which I mentioned earlier. After installing that driver I was asked to reboot my system and, at that point, the system was not booting correctly anymore. Ubuntu proposed me to enter the emergency mode (or whatever it’s called, can’t really recall it now). Once into the recovery mode I listed the available snapshots (below I already filtered the relevant ones via grep):

zfs list -t snapshot |grep apt|grep 2023-01-19-1453|grep ROOT

rpool/ROOT@zfs-auto-snap_apt-2023-01-19-1453                                              0B      -       96K  -
rpool/ROOT/ubuntu_8foh88@zfs-auto-snap_apt-2023-01-19-1453                              180M      -     11.2G  -
rpool/ROOT/ubuntu_8foh88/srv@zfs-auto-snap_apt-2023-01-19-1453                            0B      -       96K  -
rpool/ROOT/ubuntu_8foh88/usr@zfs-auto-snap_apt-2023-01-19-1453                            0B      -       96K  -
rpool/ROOT/ubuntu_8foh88/usr/local@zfs-auto-snap_apt-2023-01-19-1453                      8K      -     6.01G  -
rpool/ROOT/ubuntu_8foh88/var@zfs-auto-snap_apt-2023-01-19-1453                            0B      -       96K  -
rpool/ROOT/ubuntu_8foh88/var/games@zfs-auto-snap_apt-2023-01-19-1453                      0B      -       96K  -
rpool/ROOT/ubuntu_8foh88/var/lib@zfs-auto-snap_apt-2023-01-19-1453                        0B      -       96K  -
rpool/ROOT/ubuntu_8foh88/var/lib/AccountsService@zfs-auto-snap_apt-2023-01-19-1453        0B      -      364K  -
rpool/ROOT/ubuntu_8foh88/var/lib/NetworkManager@zfs-auto-snap_apt-2023-01-19-1453         0B      -      148K  -
rpool/ROOT/ubuntu_8foh88/var/lib/apt@zfs-auto-snap_apt-2023-01-19-1453                  184K      -     87.0M  -
rpool/ROOT/ubuntu_8foh88/var/lib/dpkg@zfs-auto-snap_apt-2023-01-19-1453                 616K      -     55.6M  -
rpool/ROOT/ubuntu_8foh88/var/log@zfs-auto-snap_apt-2023-01-19-1453                     2.74M      -     1.40G  -
rpool/ROOT/ubuntu_8foh88/var/mail@zfs-auto-snap_apt-2023-01-19-1453                      80K      -      248K  -
rpool/ROOT/ubuntu_8foh88/var/snap@zfs-auto-snap_apt-2023-01-19-1453                       8K      -     2.39M  -
rpool/ROOT/ubuntu_8foh88/var/spool@zfs-auto-snap_apt-2023-01-19-1453                    128K      -     4.70M  -
rpool/ROOT/ubuntu_8foh88/var/www@zfs-auto-snap_apt-2023-01-19-1453                        0B      -       96K  -

The entries I’m interested in are the ones that shows a “USED” size greater than zero. In my case I rolled back:

rpool/ROOT/ubuntu_8foh88@zfs-auto-snap_apt-2023-01-19-1453   
rpool/ROOT/ubuntu_8foh88/var/lib/apt@zfs-auto-snap_apt-2023-01-19-1453 
rpool/ROOT/ubuntu_8foh88/var/lib/dpkg@zfs-auto-snap_apt-2023-01-19-1453 

with these commands:

zfs rollback rpool/ROOT/ubuntu_8foh88@zfs-auto-snap_apt-2023-01-19-1453   
zfs rollback rpool/ROOT/ubuntu_8foh88/var/lib/apt@zfs-auto-snap_apt-2023-01-19-1453 
zfs rollback rpool/ROOT/ubuntu_8foh88/var/lib/dpkg@zfs-auto-snap_apt-2023-01-19-1453 

that’s it. I rebooted the system and I was back to the previous stable state.

P.S. Why didn’t I roll back all the snapshots? Well, simply as I didn’t care about restoring things like the logs at the previous state, nor my home directory, as I knew there was no problem there.

Conclusion

ZFS Snapshots are a great tool for restoring your Linux system to a previous state. They are fast, efficient, and easy to use. Plus, with the ability to automate the process of creating snapshots, you can make sure that you always have a recent snapshot of your system.

So, if you ever find yourself needing to restore your Linux system, be sure to give ZFS Snapshots a try. With their help, you’ll be able to restore your system in a matter of seconds!