Ubuntu 24.04 Supports Easy Installation Of OpenZFS Root File-System With Encryption

[deusexmachina]

Why are people using void? nixos is the distro to use with zfs

zfsbootmenu devs use void and wrote its install guide for zfs on root. NixOS is a bikeshed that's mutated into a prefab mansion assembly line. Isn't it on zfs version 2.1 anyway? void & arch are on 2.2.3 and are both a more stable experience than NixOS.

[royce]

Ubuntu's setup is just ZFS over LUKS.

Is it? I've used this before, admittedly a couple of years back, and it was using native ZFS encryption. Has this changed with this release?

[pong]

It might (??) not be the "filesystem" maybe just something in a driver or the resume logic that doesn't deal gracefully with the wake up of that drive, maybe I/O is attempted before the drive is ready or before it is mounted / unlocked / whatever.

Of course the file system higher layers could be more resilient with retries or administerable settings to delay / verify / retry / recover or such.

Some systems have different power management modes you can set the disks to go in during various idle / sleep conditions so it's possible tuning something
there could help unless you mean cold-off hibernate kind of sleep at which point the ACPI/ASPM and driver re-init etc. has to work to get it ready before anything else happens.

Fun real life recent experience here. I've been mostly fine with Btrfs for root on Tumbleweed / Leap / Fedora for years now. But I just had Btrfs absolutely shit itself for seemingly no good reason on my main desktop two weeks ago. It locked hard when resuming from sleep. I had to power cycle the machine but it came back up. The same thing happened on the next sleep cycle. After that, I couldn't even log into a TTY or ssh session due to i/o errors. It starting spewing this over and over again ...

BTRFS error (device dm-1: state EA): bdev /dev/mapper/system-root errs: wr 113, rd XXXXXXXX, flush 0, corrupt 0, gen 0
BTRFS error (device dm-1: state EA): bdev /dev/mapper/system-root errs: wr 113, rd XXXXXXXX, flush 0, corrupt 0, gen 0
BTRFS error (device dm-1: state EA): bdev /dev/mapper/system-root errs: wr 113, rd XXXXXXXX, flush 0, corrupt 0, gen 0
BTRFS error (device dm-1: state EA): bdev /dev/mapper/system-root errs: wr 113, rd XXXXXXXX, flush 0, corrupt 0, gen 0

I suspected the drive was fine (SK Hynix Gold P31 NVMe). Sure enough, even putting some unnecessary wear on the NAND with a 4 pass run of badblocks showed the drive was perfectly fine. I've never had a ZFS pool / filesystem crap out like that.

[pong]

So does the ubuntu install / boot process support things like generating a LUKS-2 compatible image and work with things like FIDO tokens and such being able to be used to open / boot the luks encrypted zfs root and other drives / partitions?

It was moderately recently (last year?) IIRC that there were some improvements to / problems with some versions of GRUB to deal with luks 2, fido, etc.
IIRC some worked around the issue by using dracut instead of grub.

[aviallon]

zfsbootmenu devs use void and wrote its install guide for zfs on root. NixOS is a bikeshed that's mutated into a prefab mansion assembly line. Isn't it on zfs version 2.1 anyway? void & arch are on 2.2.3 and are both a more stable experience than NixOS.

Well, actually, you get to choose between the stable ZFS version (2.1) or the unstable version (2.2) on NixOS.
Also, I'm quite sure you've never ever used NixOS in production. Why? Because it just works. All the time.
My backup servers are using NixOS with ZFS storage. They are using unattended upgrades, and reboot automatically when needed.
I absolutely never had a single issue in four years of 24/7 use.
That's quite stable I think.

[cutterjohn]

This is a real ace in the sleeve of ubuntu. I don't support zfs due to their licensing shit, but for those that insist on it i think it's nice to have a popular distro offer great support.

Admittedly, you are putting yourself kind of in a vendor lock-in in the sense that it's not plug&play anymore to migrate.

BS ZFS is the best 'stable' CoW FS available to linux right now... ots we;; tested and stable! The only 'problem' os the BS 'licensing' 'problem' from a freer license! I never got the loud minority whinging about freer licenses like BSD and the like... TBH... I thought that they were akk about 'free', oh wait they meant only their narrow minded definition of 'free'...

[antonyshen]

Do they have plan to add BTRFS and subvol support back?

[Jumbotron]

and that's why i applauded them for that part. Am i a zealot for expressing my concern, and preference for something else? You sure are bound to see zealots everywhere you go outside of your little zfs fanclub if you get that hostile whenever someone else voices a differing opinion.

I couldn’t give a shit about ZFS. I’ve been ok with with EXT4 for years and years. Until EXT4 has run its course for my use case I won’t change. But I understand why some folks want ZFS on Ubuntu. So good on Canonical for offering it and making it relatively easy to install. And yes after 20 years using Linux and going on forums like this one I have the receipts to say in your words “ I see zealots everywhere”. And for your information junior I wasn’t calling you personally a zealot. I used the word “you” in a rhetorical manner. Meaning anyone out there who might be anti ZFS because of Oracle or whatever .

[darkbasic]

Zsys never really took off outside of Ubuntu for that same reason because it was based on ZFS using GRUB with a split boot & root partition scheme that may or many not have been on top of LUKS. None of that utilizes ZFS's strengths. It's just how Ubuntu does things and Zsys acts as a way to place ZFS into the role of a traditional file system and framework the bootloader, encryption, and other features around that.

They're not exactly the same, but Zsys is to ZFS is as what Stratis is to XFS. They both framework features around a file system. The difference with Stratis is that RHEL and Fedora have made Stratis so easy for others to adopt that it's gone from AUR to now being part of the Arch repos while Zsys hasn't even made it to the AUR. It says a lot about a project without saying anything at all when something as high profile and well knows as Zsys doesn't make the AUR.

It didn't hit the AUR because it quickly became unmaintained and has the habit of eating your data. Otherwise there is not even a single alternative capable or rolling back to an older snapshot without losing newer ones. Not even zfsbootmenu.

[Agno]

Ubuntu's setup is just ZFS over LUKS.

It is not.

This is not ZFS on root, but ZFS wrapped up in LUKS.

The only thing that use LUKS is a small zdev. No zpool use a LUKS encrypted partition.

Is it? I've used this before, admittedly a couple of years back, and it was using native ZFS encryption. Has this changed with this release?

It has not changed. You are right. The ZFS installation on Ubuntu 24.04 (tried yesterday on a VM) is the same as it was on 22.04. Same LUKS-encrypted keystore zdev. Same limitations.

So does the ubuntu install / boot process support things like generating a LUKS2 compatible image and work with things like FIDO tokens and such being able to be used to open / boot the luks encrypted zfs root and other drives / partitions?

Not sure. But the kernel and initrd are not encypted. You can use everything LUKS supports. But it will happen in the initrd, not in grub.

The way Ubuntu set up ZFS with encryption is the following:

• 2 zpool: a small one for grub boot environment (not sure why do you need this...), a main one for everything else;
• the installer will use 1 disk, partitioning it in 3 (no partition is LUKS-encrypted): one tiny partition for bootloader(bios)/esp(uefi), one small partition as a boot zpool vdev, one big partition as a main/root zpool vdev;
• the boot zpool is not encrypted
• the main zpool contains a small zdev called keystore that is LUKS-encrypted and contains a small ext4-formatted file system with a single key file
• the root filesystem in the main zpool (and all its children created by the Ubuntu installer) are encrypted using native zfs encryption with the key saved in the keystore

The boot process is the following:

• grub boots from the boot zpool, unlocking the encryption is done by scripts in the initramfs
• initramfs scripts import the main zpool
• initramfs scripts decrypt /dev/zd0 into /dev/mapper/keystore-POOLNAME
• initramfs scripts mount /dev/mapper/keystore-POOLNAME at /run/keystore/POOLNAME/ and load the key
• the root filesystem and all children are mounted

IMHO I think Ubuntu developers came up with this idea to overcome the limitations of zfs native encryption: with a LUKS-encrypted keystore you can support everything LUKS supports(passwords, TPM2, tokens, ....) and still have a zfs native encrypted system.