Installing Rocky-8.4 (RHEL) into a live ZFS filesystem

I have been using ZFS sinze 2009 on various operating systems from FreeNAS over Illumos, FreeBSD and Ubuntu. Hence my ZFS filesystem lives on, even if the operating systems get updated/changed or even replaced.

And who cares what the name of the Operating system is, those day are over, we just want to get our own job done, and my data is what is relevant in the end. — With ZFS we can finally keep our filesystem even when we change our Operating system.

Last Christmas I got a few Emulex 10Gbit Ethernet cards which I couldn’t get running under Ubuntu, 10 Gbit Ethernet interconnecting my servers. I had given up on these cards but it seems  like there had been support for them in RedHat Enterprise Linux, and now that Rocky Linux again is making it possible for normal people to run RHEL binaries, I though I would give it a try.

The plan here is:

  • Install Rocky Linux on a virtual machine
  • Get the bits and pieces and configure Rocky Linux for ZFS
  • Create a full tar dump of Rocky Linux on the virtual machine
  • Create a ZFS dataset and import the above tarfile on the virtual box
  • Teach GRUB2 how to boot Rocky Linux from the ZFS-dataset on the virtual box
  • Create a ZFS dataset and import the above tarfile on the real server
  • Use the same SSH credentials to avoid as the host already running this ZFS filesystem to avoid lot of fuss
  • The real server has allready been taught how to boot various Operatig Systems

Most the work is done on the Virtual machine, once it works, the last few steps are repeated on the Ubuntu system that hosts the ZFS filesystem with very little down-time.

Rocky Linux on a virtual box with ZFS support

I took a shortcut and downloaded a precoocked virtual box image Rocky Linux 8.4 VirtualBox Image from LinuxVMimages.com  you can choose between a minimal 0.6Gb or desktop version 1.6Gb, There no screen on my server so I took the small one.

The proper way is ofcourse to the get the real .ISO from https://rockylinux.org install it and then  do what is explained in Getting started on OpenZFS

I normally configure my virtual machines to be on a bridged network, and do all the work over a ssh login. for so many reasons, f.ex. I can have a full log of what I did.  so handy when you come back years later and forgot to write a blog-post about your endeavour then.

So here is what I did:

$ ssh vbox-rocky
$ sudo -s
# vi /etc/selinux/config                            ## set SELINUX=permissive
# dnf update
# source /etc/os-release
# dnf install epel-release
# RHEL_ZFS_REPO=https://zfsonlinux.org/epel/zfs-release.el${VERSION_ID/./_}.noarch.rpm
# dnf install -y $RHEL_ZFS_REPO
# dnf install kernel-devel zfs zfs-dracut tar
# echo 'add_dracutmodules+=" zfs "' > /etc/dracut.conf.d/zfs.conf
# echo zfs >/etc/modules-load.d/zfs.conf
# echo nvme >/etc/modules-load.d/nvme.conf          ## need driver for my ssd
# vi /lib/dracut/modules.d/90zfs/zfs-generator.sh   ## fixing a bug, see below
# dracut --force --no-host-only
# echo -n >/etc/zfs/zpool.cache
# tar -cvzf - --one-file-system / /boot /home | ssh peter@t470t dd of=rocky.tgz

To get Rocky-8.4 to boot from a legacy dataset I had to create and apply the patch below

--- /lib/dracut/modules.d/90zfs/zfs-generator.sh.orig	2021-09-16 20:22:33.000000000 -0400
+++ /lib/dracut/modules.d/90zfs/zfs-generator.sh	2021-10-29 19:34:19.271560868 -0400
@@ -28,11 +28,26 @@
 	exit 0
 
 rootfstype=zfs
-case ",${rootflags}," in
-	*,zfsutil,*) ;;
-	,,)	rootflags=zfsutil ;;
-	*)	rootflags="zfsutil,${rootflags}" ;;
-esac
+mountpoint=""
+if [ x"$(zpool list -Ho name)" = "x" ]; then
+	echo "zfs-generator: zpool is not imported yet so we cannot see if mountpoint is legacy" >> /
dev/kmsg
+	if [ "${root##ZFS=}" != "$root" ]; then
+		echo "zfs-generator: root=$root - assume legacy if root=ZFS=" >> /dev/kmsg
+		mountpoint="legacy"
+	fi
+else
+	mountpoint="$(zfs get -Ho value mountpoint ${root##ZFS=})"
+fi
+if [ "$mountpoint" = legacy ]; then
+	echo "zfs-generator: $root is legacy mount" >> /dev/kmsg
+else
+	echo "zfs-generator: $root is not legacy setting zfutil" >> /dev/kmsg
+	case ",${rootflags}," in
+		*,zfsutil,*) ;;
+		,,)	rootflags=zfsutil ;;
+		*)	rootflags="zfsutil,${rootflags}" ;;
+	esac
+fi

Mounting legacy root filesystems under Rocky-8.4 does not work out-of-the-box. and I believe this is the case for all RHEL based systems

On Debian and Ubuntu I use root=ZFS=pool/ROOT/ubuntu-20.04 on the cmdline to boot into Linux with root-on-ZFS, the dataset has the mountpoint set to legacy. I have been using this setup for 10 year. My ZFS-filesystem can hence be booted from different OS-es including FreeBSD and Illumos in the past.

The culprit here is dracut (initramfs) which always want to mount the rootfs-dataset with -o zfutils, which again means you cannot use legacy mount.

I have raised this issue on github.com/openzfs

Now the only thing left is to create a ZFS dataset and unpack the tar-archive we extracted from the virtual machine.

Keep ssh credentials

I copy over the ssh credential, so ssh-wise it looks like the same host.

rsync -av --exclude "ssh*_config" /pool/ROOT/ubuntu-20.04/etc/ssh /pool/ROOT/rocky-8.4/etc/ssh

Done – Rocky Linux can now bring up my ZFS file-system

I have made a small addition to grub /etc/grub.d/09-zfs so Rocky-Linux will automagically be in my boot menu, a version of that script was described in a blog-post 5 years ago Installing FreeBSD into a live ZFS filesystem from f.ex. Linux more about that in my next blog post, it is much simpler now.

This entry was posted in Linux, ZFS. Bookmark the permalink.