The Dracut Initramfs Generator Is Slow - Could Be Much Faster As Shown By Distri's Minitrd

[skeevy420]

I wonder if using zstd for compressing the image would result in a significant gain in compression/decompression time?

Yes, it does, but one has to use zstd-fast 1000 (yes, that's a real setting) to beat LZ4 speeds and compression. You also have to use a custom patched kernel because Zstd compressed images aren't supported on mainline Linux (at least they weren't when I last checked last month).

[skeevy420]

You can rebuild dracut images with just "dracut -f".

Fedora is even discussing moving to prebuilt dracut images, and Silverblue already uses them.

Hmm. I don't know how that would work on Arch; especially Manjaro which I use.

Wouldn't Manjaro have to wrap up all their prebuilt dkms modules, like Nvidia and ZFS, and ship them out for everybody? For a fixed system like Silverblue, prebuilt is perfect. For a dynamic system, prebuilt seems like it could be a fustercluck.

[Britoid]

Hmm. I don't know how that would work on Arch; especially Manjaro which I use.

Wouldn't Manjaro have to wrap up all their prebuilt dkms modules, like Nvidia and ZFS, and ship them out for everybody? For a fixed system like Silverblue, prebuilt is perfect. For a dynamic system, prebuilt seems like it could be a fustercluck.

iirc there's a method of making a dracut image load another image which can contain added kernel modules, the efi vars would contain the parameters for loading this extra image.

dkms is pretty awful anyway and you shouldn't be relying on it to boot your system. Nvidia can already boot with a prebuilt initramfs on Silverblue fine, the driver just gets loaded later on during the boot process.

prebuilt dracut images means the entire boot process can be signed and things like hibernation (with secure boot) and using the tpm to store encryption keys becomes a lot more feasible.

[skeevy420]

iirc there's a method of making a dracut image load another image which can contain added kernel modules, the efi vars would contain the parameters for loading this extra image.

dkms is pretty awful anyway and you shouldn't be relying on it to boot your system. Nvidia can already boot with a prebuilt initramfs on Silverblue fine, the driver just gets loaded later on during the boot process.

prebuilt dracut images means the entire boot process can be signed and things like hibernation (with secure boot) and using the tpm to store encryption keys becomes a lot more feasible.

Image loading image make sense. Currently, nothing on my system relies on dkms modules to boot...minor functionality and external ZFS drives, yeah, but nothing boot critical.

Why can't we just use /boot/bin and /boot/lib for crap like this and not generate initramfs images at all? I've always thought the /usr merge was kind of dumb since it wasn't accompanied with a /boot merge and that the move to initramfs just added unnecessary complexity.

[Britoid]

Image loading image make sense. Currently, nothing on my system relies on dkms modules to boot...minor functionality and external ZFS drives, yeah, but nothing boot critical.

Why can't we just use /boot/bin and /boot/lib for crap like this and not generate initramfs images at all? I've always thought the /usr merge was kind of dumb since it wasn't accompanied with a /boot merge and that the move to initramfs just added unnecessary complexity.

Because you need the tools to mount the root filesystem, and if you have something like encryption you need a pre-boot envrionment to take care of that.

/usr merge made a lot of sense. /usr has been effectively repurposed as a /System -eque partition.

[q2dg]

Is there anyone here from the dracut team who wants to share his opinion?

[skeevy420]

Because you need the tools to mount the root filesystem, and if you have something like encryption you need a pre-boot envrionment to take care of that.

/usr merge made a lot of sense. /usr has been effectively repurposed as a /System -eque partition.

On my system, GRUB/LUKS with a password handles my /boot encryption which then hands it off to keyfiles to unlock my ZFS drives. But my initial unlock doesn't require an initramfs and my other unlocks can be scripted into any environment that has some ZFS modules.

That doesn't explain why we need an initramfs. /boot/bin,lib can contain the same exact stuff an initramfs image contains without having to deal with images and other annoyances when messing with the bootstack. LUKS, Grub, and all that can already handle pre-boot encryption and the rest of the tools are on /boot/bin. It really is no different than what an initramfs does only it removes the compressed archive step.

Yep. I've narrowed it down to 5 top level folders/drives.

/admin for /etc and /srv (which, IMHO, should contain systemd services instead of /etc/systemd)
/system for /usr
/active for read/write folders like /var (needs a better name)
/users for users and home directories
/boot for the bootstack

Or 4 if /users and /var are combined like Silverblue does. Or 3 if that is done and /admin is moved to /active/admin...but I don't like the idea of config files that should, ideally, be read only being stored on a read-write drive/partition/dataset.

Guess who is working on their own ZFS read-only root standard and has thought way too much about where stuff is stored and why?

Can a systemd dev answer this-- How come /srv doesn't contain any service files? Why is /etc/systemd used?
I could read-only my /etc up there and move /srv to /active/services if it weren't for systemctl.

[Vistaus]

Is this going to eventually mean that "mkinitcpio -p linux-lts" and /etc/mkinitcpio.conf are going to be something different in the future? I hope not.

For me, I don't really care if the initramfs generates faster or not. 99.999% of the time I'm building one it's during a kernel upgrade. It takes around 5 minutes to build all the dkms modules I use and no initramfs generator is going to speed the "compiling software" part of the process up.

You must have an old PC as my 2018 mid-range laptop generates initramfs in less than 30 seconds.

[BwackNinja]

On my system, GRUB/LUKS with a password handles my /boot encryption which then hands it off to keyfiles to unlock my ZFS drives. But my initial unlock doesn't require an initramfs and my other unlocks can be scripted into any environment that has some ZFS modules.

That doesn't explain why we need an initramfs. /boot/bin,lib can contain the same exact stuff an initramfs image contains without having to deal with images and other annoyances when messing with the bootstack. LUKS, Grub, and all that can already handle pre-boot encryption and the rest of the tools are on /boot/bin. It really is no different than what an initramfs does only it removes the compressed archive step.

Yep. I've narrowed it down to 5 top level folders/drives.

/admin for /etc and /srv (which, IMHO, should contain systemd services instead of /etc/systemd)
/system for /usr
/active for read/write folders like /var (needs a better name)
/users for users and home directories
/boot for the bootstack

Or 4 if /users and /var are combined like Silverblue does. Or 3 if that is done and /admin is moved to /active/admin...but I don't like the idea of config files that should, ideally, be read only being stored on a read-write drive/partition/dataset.

Guess who is working on their own ZFS read-only root standard and has thought way too much about where stuff is stored and why?

Can a systemd dev answer this-- How come /srv doesn't contain any service files? Why is /etc/systemd used?
I could read-only my /etc up there and move /srv to /active/services if it weren't for systemctl.

Having initrd gives the flexibility to have the rootfs in many different configurations. It could be encrypted, remote, selectable, require drivers not in the kernel, or not even exist. it isn't strictly necessary, particularly in a standard desktop system, but is normally easier and more generic than the alternatives.

/srv traditionally included files for services, like /srv/http or /srv/ftp, the former is /var/www in debian-based systems.

As for big changes to the filesystem hierarchy, you'll have a good amount of software to patch. To start with:

• /etc/passwd, /etc/hosts, /etc/resolv.conf and a few others are hardcoded into the libc and other programs
• #!/bin/sh or #!/bin/bash is often the shebang in shell scripts, and you'd have similar problems with python whether you use #!/usr/bin/env python or #!/usr/bin/python
• /etc/mtab is often used because it's not Linux-specific like it's normal symlink end, /proc/mounts
• /var/run is similarly used by software, and normally a symlink to /run on the filesystem
• You'll need to set your include dir for building software because its normally expected in /usr/include by the compiler
• /lib/ld-linux.so.2 is the linker hardcoded in most binaries, so no 3rd-party binaries can be expected to work without it

Beyond that there's also deeper philosophical questions like:

• Where is the line between data and configuration? - This is more visibly blurred where software has its default configuration in /usr/share, and overrides (by the distro or user) in /etc
• How do you organize configuration and other data? Particularly in the case where it is shared by multiple programs. (again, /etc/resolv.conf and others come to mind)
• Is there a separation between the base system and applications? As it is with the FHS, there isn't really a notion of a core system -- all packages installed are integrated into the system rather than being treated as an addon to the system.

I have also put a lot of thought into. My main conclusion is that if a standard works well enough, it won't be changed by any big player because of the costs vastly outweigh the benefits. The filesystem is just another implementation detail, people care more about how everything works on top of it and how easy those things are to use. A small project can certainly make those changes because they can explicitly ignore compatibility concerns.

[zdzichu]

So I'm looking into /usr/bin/dracut on Fedora 31 and I see already:

    for i in pigz gzip lz4 lzop zstd lzma xz lbzip2 bzip2 cat; do
        command -v "$i" &>/dev/null || continue
        compress="$i"
        break
    done

In other words, it tries to use pigz, then gzip, then some others, then zstd... It looks like selecting best compressor is already implemented.