Red Hat Developers Announce Work On New "Composefs" File-System

[phoronix]

Phoronix: Red Hat Developers Announce Work On New "Composefs" File-System

Red Hat engineers this morning for the first time publicly announced their work on Composefs, a new opportunistically-sharing and verified image file-system...

https://www.phoronix.com/news/Composefs

[arun54321]

why not just use erofs? NIH syndrome?

[sinepgib]

why not just use erofs? NIH syndrome?

Does it allow sharing between mounts?
I'm actually surprised regular OverlayFS (IIUC the article) doesn't. I mean, it's common sense, the shared layers should share page cache. I assumed that was already how it worked. Or is it about file granularity sharing for intersecting-but-not-same images? Isn't the idea for those to build on top of each other without duplicating files?

[archkde]

Does it allow sharing between mounts?
I'm actually surprised regular OverlayFS (IIUC the article) doesn't. I mean, it's common sense, the shared layers should share page cache. I assumed that was already how it worked. Or is it about file granularity sharing for intersecting-but-not-same images? Isn't the idea for those to build on top of each other without duplicating files?

As far as I understand, it's about sharing with intersecting images. But I don't quite understand what composefs gives you over read-only overlayfs here.

[hsiangkao]

why not just use erofs? NIH syndrome?

Actually someone asked me, and my answer was why not consider finer sharing chunk-based blobs rather than file-based OSTree-like blobs. I just asked him to ask the community about this path first but soon they insisted on reinventing this :-)

Does it allow sharing between mounts?

Yes, see: https://www.phoronix.com/news/Linux-6.1-EROFS

[Britoid]

why not just use erofs? NIH syndrome?

I believe they do very different things

composefs isn't a physical file system, it doesn't change how data is stored on disk and uses a backing filesystem (e.g. XFS, btrfs etc). It's basically a way to say get these files, put them here and there, without copying them, mark them read-only with these properties and have a verifiable signature when loading them.

It's basically for OStree and means it no longer needs to build a bazillion hard links that are susceptible to modification if you mount the directory as rw. It's also useful in containers since the container root filesystem can be a compiled composefs image and use a git-like object store for deduplication in user space.

[hsiangkao]

composefs isn't a physical file system, it doesn't change how data is stored on disk and uses a backing filesystem (e.g. XFS, btrfs etc). It's basically a way to say get these files, put them here and there, without copying them, mark them read-only with these properties and have a verifiable signature when loading them.

EROFS can get a lot blobs from backing-fses with new fscache feature as well, but in the form of chunk-based sharing, which means one blob can have a part of a file _or_ an entire file (just like what OSTree now does) _or_ several files. I wonder why OSTree per-file sharing is an effective model after introducing a filesystem with some actual metadata since we can do finer chunk-based sharing with a brand new filesystem (actually that is what composefs does as well) rather than insist on hardlink-like per-file sharing as OSTree

[Danielsan]

So eventually at Team Red someone realized that Snap made by Team Orange was somehow good but just made poorly...

We should invent a slogan for Linux...

Linux where everyday something magical happens...

[Britoid]

So eventually at Team Red someone realized that Snap made by Team Orange was somehow good but just made poorly...

We should invent a slogan for Linux...

This is way more similar to how Flatpak does things than Snap.

[Vistaus]

So first Stratis and now Composefs? I know they are different but still, two filesystems in development by the same company seems like a bit much to me.