Linus Torvalds Comments On Bcachefs Prospects For Linux 6.6

[avis]

Performance is up to par for the most part, however, data integrity is often an issue due to garbled code which is being 'maintained' yet not refactored.

Any evidence to back it up? This is the first time I'm hearing this. The Linux kernel is running close to 2 billion Android devices. I've not heard of any data integrity issues yet. I'm inclined to think you're making stuff up.

Data integrity issues instantly attract a ton of attention because data has been money for the past several millennia.

[Errinwright]

Would you please provide examples about it at least?

What garbled code?

What data integrity issues happened because of it?

Do you consider Bcachefs have these issues too?

Is an issue of code from VFS subsystem(s), filesystems itself or something else?

Talking about passive aggressiveness. My examples are just anecdotal, for instance game files turning corrupt on XFS and going over various 'older code' on github. BcacheFS likely has many of the same issues until it matures, however, changes to the code is surmountable based on the size and the active involvement of the main developer; a port to Rust is already in the works for the distant future. Furthermore, it ticks all major features required of modern filesystems, despite equivalent (or better) performance: this point is paramount to a solid foundation to build upon. Lastly, there aren't any such alternatives on the horizon. Take it as a biased point of view if you will. I think the fragmentation of filesystems is not good for Linux -especially given the proposed VFS maintainer 'burn out'.

[avis]

Talking about passive aggressiveness. My examples are just anecdotal, for instance game files turning corrupt on XFS and going over various 'older code' on github. BcacheFS likely has many of the same issues until it matures, however, changes to the code is surmountable based on the size and the active involvement of the main developer; a port to Rust is already in the works for the distant future. Furthermore, it ticks all major features required of modern filesystems, despite equivalent (or better) performance: this point is paramount to a solid foundation to build upon. Lastly, there aren't any such alternatives on the horizon. Take it as a biased point of view if you will. I think the fragmentation of filesystems is not good for Linux -especially given the proposed VFS maintainer 'burn out'.

This sounds like a perfect example of anecdotal evidence.

Where's an appropriate bug report? Got an URL? Where's a reproducible test case? Are you sure your HW is alright? I've seen IO errors but they've always been caused by faulty HW.

[timofonic]

Talking about passive aggressiveness.

Sorry, I didn't want to sound passive-aggressive. English isn't my first language. It was totally unintended!

My examples are just anecdotal, for instance game files turning corrupt on XFS and going over various 'older code' on github.

Despite being anecdotal, it would be nice to know more details about it if you can

What 'olde code'?

BcacheFS likely has many of the same issues until it matures, however, changes to the code is surmountable based on the size and the active involvement of the main developer;

And other filesystems such as Btrfs doesn't have the same activity despite having an army of active full time paid developers? I'm curious about it, no offense intended.

a port to Rust is already in the works for the distant future.

Does Kent want to rewrite Bcachefs parts in Rust? It seems a massive amount of code right now.

And Rust isn't still available everywhere. To me, it lacks a proper equivalent implementation in GCC

Furthermore, it ticks all major features required of modern filesystems, despite equivalent (or better) performance: this point is paramount to a solid foundation to build upon.

This seems interesting to know

Lastly, there aren't any such alternatives on the horizon. Take it as a biased point of view if you will. I think the fragmentation of filesystems is not good for Linux -especially given the proposed VFS maintainer 'burn out'.

What kind of fragmentation? Too many filesystems in active development?

From my ignorance, I think Linux code management structure requires more developers and improve the management. It seems to have too much legacy methodologies from the 90s.

I consider not everything new is better, but relying on mailing lists for development is a mess. I consider Git should evoñve and integrate features available in software like GitLab, such as issue trackers and merge/pull requests. Then Git dependent software must adapt to this standarized storage of information and avoid vendor lock-in too (migrations are extremely time consuming, requiring custom software and lots of manual labor).

Maybe just one VFS maintainer isn't enough and the task should be more divided?

[Errinwright]

This sounds like a perfect example of anecdotal evidence.

Where's an appropriate bug report? Got an URL? Where's a reproducible test case? Are you sure your HW is alright? I've seen IO errors but they've always been caused by faulty HW.

It is anecdotal. It is not a hardware defect, as I test hardware daily. Calling older filesystems 'atrocious' may have been an overly biased exaggeration, however, the state of their code is objectively a hindrance for maintenance.

What kind of fragmentation? Too many filesystems in active development?

Yes, indeed. I agree with the notion that new isn't always better, however, in this case it BcacheFS was built bottom-up with a purpose of being a drop-in replacement that covers all modern features with the potential for unhindered expansion due to legacy code. Having an enthused and invested developer assisting in future maintenance is arguably a bonus - one not distracted by other projects or bureaucracy.

The lack of being included in linux-next is arguably the biggest blunder with this ordeal. Going by the compiler issues encountered I suspect 6.7 is now no longer a valid target.

[timofonic]

It is anecdotal. It is not a hardware defect, as I test hardware daily. Calling older filesystems 'atrocious' may have been an overly biased exaggeration, however, the state of their code is objectively a hindrance for maintenance.

Were you able to do some debug info? Did you store some logs that may be shared by censoring sensible information first?

Would you please be able to elaborate about some specific parts of that code you're referring to?

My curiosity is sincere. I want to know more about this, if possible.

Yes, indeed. I agree with the notion that new isn't always better, however, in this case it BcacheFS was built bottom-up with a purpose of being a drop-in replacement that covers all modern features with the potential for unhindered expansion due to legacy code. Having an enthused and invested developer assisting in future maintenance is arguably a bonus - one not distracted by other projects or bureaucracy.

What kind of bureaucracy do you consider to be negative in this case?

The lack of being included in linux-next is arguably the biggest blunder with this ordeal. Going by the compiler issues encountered I suspect 6.7 is now no longer a valid target.

Yes, I agree. I don't understand why Kent Overstreet failed this way.

Ignoring VFS maintainers and others caused too much friction too.

According to MAINTAINERS Git file:

FILESYSTEMS (VFS and infrastructure)
M: Alexander Viro <[email protected]>
M: Christian Brauner <[email protected]>
L: [email protected]
S: Maintained
F: fs/* F: include/linux/fs.h F: include/linux/fs_types.h
F: include/uapi/linux/fs.h
F: include/uapi/linux/openat2.h

But I did read on LKML that Christian is no longer active. Is that true?

I also don't understand Kent attitude too. Despite certain nonsense, he shouldn't behave that way but a more polite and diplomatic one. Maybe that's my humble point of view, I frequently deal with a*holes and extreme weirdos in my daily life.

[fitzie]

Does Kent want to rewrite Bcachefs parts in Rust? It seems a massive amount of code right now.

And Rust isn't still available everywhere. To me, it lacks a proper equivalent implementation in GCC

Kent is waiting for gcc-rust, from: https://lore.kernel.org/lkml/2023081...oria.home.lan/

Personally, I think we'd be better served by putting what manpower we
can spare into starting on an incremental Rust rewrite; at least that's
my plan for bcachefs, and something I've been studying for awhile (as
soon as the gcc rust stuff lands I'll be adding Rust code to
fs/bcachefs, some code already exists). For xfs/ext4, teasing things
apart and figuring out how to restructure data structures in a way to
pass the borrow checker may not be realistic, I don't know the codebases
well enough to say - but clearly the current approach is not working,
and these codebases are almost definitely still going to be in use 50
years from now, we need to be coming up with _some_ sort of plan.

He was quite the advocate when rust was proposed for linux. I don't think this is a near term effort, it seems he wants to do a FUSE port for some reason next.

Here is some discussion on why he wants to do the FUSE port: https://www.patreon.com/posts/status-update-11721750

Not sure what operating system would want a 3rd party filesystem.

[fitzie]

But I did read on LKML that Christian is no longer active. Is that true?

I also don't understand Kent attitude too. Despite certain nonsense, he shouldn't behave that way but a more polite and diplomatic one. Maybe that's my humble point of view, I frequently deal with a*holes and extreme weirdos in my daily life.

You may be confusing Christoph (former iomap maintainer) with Christian (still active VFS maintainer)

[avis]

From    Linus Torvalds <>
Date    Thu, 7 Sep 2023 13:51:50 -0700
Subject    Re: [GIT PULL] bcachefs
    

On Thu, 7 Sept 2023 at 13:37, Kent Overstreet <> wrote:
>
> Linus, I specifically asked you about linux-next in the offlist pre-pull
> request thread back in May. It would have been nice if I could've gotten
> an answer back then; instead, I'm only getting a definitive answer on
> that now.

I may not have answered because it was so *obvious*.

Was there really any question about it?

Anyway, the fact that the code doesn't even build for me is kind of a
show-stopper regardless. It's the kind of things linux-next is
supposed to notice early, so that we aren't in the situation where the
code has not even been build-tested properly.

The fact that the kernel is developed using a mailing list where there's no real responsibility or duties shows.

Yep, Linus often ignores private emails and oh, boy, I've seen hundreds of unreplied emails on LKML with people having quite serious issues.

Yet, most kernel developers oppose to using any formal bug tracker. It just sucks for a project of this magnitude and importance. At least I did my best to save the kernel bugzilla.

[timofonic]

From Linus Torvalds <>
Date Thu, 7 Sep 2023 13:51:50 -0700
Subject Re: [GIT PULL] bcachefs


On Thu, 7 Sept 2023 at 13:37, Kent Overstreet <> wrote:
>
> Linus, I specifically asked you about linux-next in the offlist pre-pull
> request thread back in May. It would have been nice if I could've gotten
> an answer back then; instead, I'm only getting a definitive answer on
> that now.

I may not have answered because it was so *obvious*.

Was there really any question about it?

Anyway, the fact that the code doesn't even build for me is kind of a
show-stopper regardless. It's the kind of things linux-next is
supposed to notice early, so that we aren't in the situation where the
code has not even been build-tested properly.

The fact that the kernel is developed using a mailing list where there's no real responsibility or duties shows.

Yep, Linus often ignores private emails and oh, boy, I've seen hundreds of unreplied emails on LKML with people having quite serious issues.

Yet, most kernel developers oppose to using any formal bug tracker. It just sucks for a project of this magnitude and importance. At least I did my best to save the kernel bugzilla.

It's insane. I don't understand why they don't use a bug tracker! I even don't understand why they don't use something similar to GitLab.

I understand Linus inbox may be massive too.