Linux 6.1-rc1 Released With Rust Now In The Kernel, MGLRU Added, New Hardware Support

[mdedetrich]

Now, back to the discussion. It is a shame that the LTS release is the first one with Rust this year. That seems like it was badly planned. I would have preferred it being the next cycle, so that Rust had more real world usage under its belt.

You can always make that argument, when is enough usage of Rust going to be enough for the kernel?

To a lot of people that have actually programmed in Rust (and evidently in the kernel itself) Rust has already been used enough to demonstrate its worth. To me its kind of clear that its just Rust haters/doubters that are coming up with every excuse imaginable to complain about something they don't like.

[Volta]

Some notes:[LIST][*]It is hard to improve performance of C code by converting it from a single-threaded to multi-threaded version because C makes to harder (compared to Rust) to compute/verify whether the multi-threaded code is correct

• Rust moves some of the complexity of code-checking from the human brain to the Rust compiler, which frees the brain and enables it to think more efficiently

Does it apply to core kernel as well? How Rust compiler can know how to properly implement multi-threaded code into the core kernel itself? Is it aware of kernel primitives, RCU and so on?

[cynic]

Some notes:

• If you are using the Btrfs filesystem: Btrfs has some serious long-standing issues with file fragmentation (sqlite files). Somebody will have to take a serious look at the Btrfs implementation and fix those issues - yet the fact is that nobody has fixed those fragmentation issues for many years. It is unlikely that 1 year of additional time will somehow "magically" change the broken Btrfs C code into correct C code.

all the other notes have been removed, but the following considerations apply to all of them.

btrfs fragmentation issues are not related to bad programming: it is a "features" of COW filesystem, and no, Rust is not going to magically fix it.

I like Rust, I've studied and used it for some personal projects, but please just stop to depict it as if it is going to solve all the world's problem.

[zboszor]

Some notes:

• It is hard to improve performance of C code by converting it from a single-threaded to multi-threaded version because C makes to harder (compared to Rust) to compute/verify whether the multi-threaded code is correct
  • Rust moves some of the complexity of code-checking from the human brain to the Rust compiler, which frees the brain and enables it to think more efficiently
• You aren't accounting for the possibility that some kernel developers might just be waiting for "a good-enough reason" to justify a complete rewrite of selected parts of the Linux kernel. Enabling Rust in the kernel provides 1 such reason.
• If you are using the Btrfs filesystem: Btrfs has some serious long-standing issues with file fragmentation (sqlite files). Somebody will have to take a serious look at the Btrfs implementation and fix those issues - yet the fact is that nobody has fixed those fragmentation issues for many years. It is unlikely that 1 year of additional time will somehow "magically" change the broken Btrfs C code into correct C code.
  • There might be tens of large long-standing performance issues in the Linux kernel
• It is currently impossible to directly and safely access kernel memory from user-space without a CPU context switch. Given the current state/features of C compilers, plain C code will never be able to directly&safely access kernel memory from user-space. I am not claiming that Rust enables such memory access, but Rust definitely is a small step towards such a goal.
• In 20 years, C might be considered to be an ancient (outdated) programming language for writing operating systems
• It is possible that some kernel developers will be using a C-to-Rust source code conversion tools. Because the tool will be making only a partial/imperfect conversion, the developer will need to read both the original C source code and the generated Rust source code. The process of reviewing the generated Rust source code provides an opportunity to improve its performance.

Most of these are bullsh*t and wishful thinking.

Inhererently single threaded tasks (like traversing a linked list) won't magically become multi-threaded just by switching to a different language. Tasks that can be multi-threaded can be safely separated by not using global variables. Locks and critical sections exist for a reason.

Hiding the details behind programming language features will make you forget about thinking the complex details of multi-threading. Rust will just rust (sic) when the last developer forgets about the hard complexities of multi-threading. There won't be anyone who can improve Rust.

Kernel memory is not to be accessed directly from user space. The very definition of an OS kernel is that (1) it provides services to user space and (2) it protects itself from user space. Kernel memory is always something to be protected. User space may nicely ask data from the kernel, which in turn will copy data to user space.

[Weasel]

It is currently impossible to directly and safely access kernel memory from user-space without a CPU context switch. Given the current state/features of C compilers, plain C code will never be able to directly&safely access kernel memory from user-space. I am not claiming that Rust enables such memory access, but Rust definitely is a small step towards such a goal.

It's actually funny how delusional Rust proponents/devs truly are. This is one of those examples.

Do you actually think that Rust provides protection from MALICE? (i.e. intentionally "bad" code or input)

If you can access kernel memory, it's over. It doesn't fucking matter if Rust complains when you do it, a hacker isn't going to be a Rust purist idiot, he will just write it in C or assembly and exploit it. Or even use unsafe in Rust, but I doubt any hacker will ever use Rust to begin with.

[karolherbst]

Rust haters missing the point: It's pain to code in C and fun to code in Rust. Specifically if you do multithreading.

[cynic]

You are mistaken in this case.

well, if it is the case (which I don't think it is), can you provide a resource where this is discussed?

[mdedetrich]

Rust haters missing the point: It's pain to code in C and fun to code in Rust. Specifically if you do multithreading.

Not even just multithreading, even dealing with memory is much easier in Rust (after all the reason that Rust makes multithreading a breeze is because it verifies how memory is accessed so for example it gives you a compiler error when multiple threads access the same shared data structure at once without having some sought of resolution mechanism, i.e. mutex or new reference on an immutable data structure).

[darkonix]

Rewriting kernel algorithms from C to Rust will bring performance improvements, not because Rust code runs faster than C, but because a rewrite from language X to language Y offers an opportunity to rethink the algorithm. In my opinion, major rewrites are unlikely to happen before kernel version ~6.10 though (~2 years from today).

That is likely true for other projects that are porting some utilities to Rust, but the core kernel is not being rewritten in Rust. The only things that's being added is experimental infrastructure support for implementing modules/drivers in Rust. Now, some projects are reported to rewrite some drivers to Rust, I'm guessing that's just as proof of concept pilots and not to replace anything at this point.

[cynic]

I have reported sqlite issues to btrfs kernel mailing list several times. btrfs developers fixed some smaller issues. There seems however to be no hope that they will fix fragmentation and over-allocation issues in their entirety. In very simplified terms: defragmentation in btrfs is currently implemented so that it can handle the case of appending lines to large log files over extended periods of time - btrfs fails to handle the case of an application writing pages to random offsets in a very large file over extended periods of time.

this 100% confirms what I stated before. This is a fs design "issue" (this is how COW fs works), not a programming issue and there's no magic programming language that is ever going to fix it.

btrfs autodefrag is not a solution but rather a workaround and is never going to fix it either.