New Linux Kernel Patches Begin Plumbing Rust Support Into Bcachefs Driver

[intelfx]

But we are also free to hate this decision, express our hatred for that and steer clear of their work.

Look, if you consider yourself free to do this, then we are likewise free to call you a bunch of pathetic clowns. Sounds fair to me, no?

[reba]

Initially, Python does not use a compiler but an interpreter. With that out of the way, do i like being forced to use the Python interpreter? No, but I can decide in favor or against using it based on the scripts that I might want to run. For example, every computer I use has python. My Gentoo desktops even have rustc, because of Firefox. On the other hand I have nowhere installed java, because there is no java app worth having. You don't get the point though.

I never said installing rustc is a big problem. I said that it might be a small problem, but it is a problem. Always adding a dependency that did not exist, is a problem. Do the benefits for the developer outweigh the problems for some users? Maybe for some cases. Do they for bcachefs Rust (or Rust for any part of the kernel in this regard)? I don't think so, since it causes the extra complications that I described above.

Okay, fair.
I don't program in Python and am not really a fan of it because of all that pip, pipx, pypi, pipenv, etc pp., the 2-vs-3 drama, vanishing dependencies and other problems and thought "*.pyc" were compiled python files, my bad.

And I can understand you want to minimize dependencies where possible. Fair, too.

But I do think Rust is a valuable addition to a project because of the intrinsic properties it brings with it other languages just can not provide.
Therefore from the pragmatic view I welcome any project that adds Rust to itself, replacing C/C++, either piece-by-piece or at a whole as these intrinsic gains then become part of the project.

Granted, for mixed-language projects this increases the number of dependencies until everything has been rewritten or packetized to be binary included.

But to be fair rustc is a dependency of so many projects it's just like gcc.

As a whole I personally see adding Rust to a project as a plus, but we're nearing a hornets nest here...

tldr: I understand exploding dependencies are a problem; however I see it as a trade of of what is gained and - for me - it outweights the gained dependencies, especially in areas where preventable errors are not negotiable like filesystems, drivers, kernel stuff in general, codecs and other high performance, parallelized or parallelizable and therefore race condition exposed areas, where buffer overflows are critical (anything kernel of course) and robustness is of prime concern.

[smitty3268]

thought "*.pyc" were compiled python files, my bad.

They are compiled byte-code, similar to how Java works. And it is possible to compile all the way down to an executable, although that's not typically done.

I suspect marios would consider Java a compiler, though.

[hwertz]

Rust isn't that bad. It's essentially an ultra-paranoid version of C, you can really can start out with some C code, make some light modifications to it to add the type information etc. that Rust requires, and if your C code doesn't have any security holes it'll compile. It catches switch statements where yo don't cover all cases, potential buffer overflows, potential race conditions and concurrency issues, memory use after free, etc., at compile time throwing a compile time error rather than having potentially insecure code. Since the bcachefs command line tools are already written in Rust, the in-kernel Rust could of course be using some code from there. But I imagine largely it could use the existing C code (modified to meet Rust requirements), and any potential security or concurrency issues Rust finds fixes applied to the Rust code could actually be applied back as fixes to the C code too.

It doesn't ultimately make sense to have both C and Rust code for this. But it does make sense to have both while the Rust code is under development, it's nice to have 2 options when something is in development rather than being forced into being a beta tester.

As for other platforms -- I agree, Rust should be available on them. I don't think Rust is exceptionally hard to port, I think the only part that is particularly CPU-specific is the concurrency primitives for threading (i.e. to be able to ensure safe locking/mutex behavior on multi-threaded applications.) (Edit: Well, yeah it also has to generate code for the CPU. I assume like gcc and clang, you have essentially a template you can fill in the instruction set and some characeristics of the CPU to help the compiler.) I mean, they do (per the docs) they don't appear to have an x86 build, but have x86-64, various quite old through modern ARM, MIPS, SPARC, Motorola 68K, a target for Nvidia GPU, RISCV, and PowerPC; oh and WebAssembly. I guess that leaves out like PA-RISC, Alpha, and S390 but really, if it's running on 68K I'm sure it can be brought up and running on those 3.

[oleid]

About it being inconsequential to users? Look at marios' posts. It does affect users in a negative way, even if they don't code a single line of Rust.

By that logic you'd be inconsequential to users as well by requiring a C compiler if people don't code in C, since you could simply use bare assembler

[oleid]

Wayland was stillborn. It was barely alive before 2015 in the first place

How can something be stillborn and then barely alive afterwards? That makes no sense.

It is talked about, but everyone is *still* using X11 programs almost 10 years later...

Who is everybody? By the end of last year I installed an X-Free Gentoo system to my old laptop with graphical login manager and Wayland compositor. Even without Xwayland. No libX11-anything. It's a possibility.

You realize we are having this same discussion almost 10 years after 2015 and Wayland has made almost no progress since.

Hmmmm, it depends on the definition of progress.

[oleid]

Yes, but rust is not in the list. But using 2 compilers for a single binary is usually a big no for me (especially if the binary is the kernel). It forces to choose between two bad choices in this case. Either use 2 different toolchains, an llvm based for Rust and another of my choice for C (does not like the best idea and might break things, i.e. LTO) or use an llvm toolchain compatible with rustc for both C and Rust which is too restrictive (even the latest llvm might be incompatible with rustc, let alone gcc).

Just for informational purposes:

If you're interested in Link time optimisation, clang is your best choice (due to build performance and memory requirements). If you use the latest clang and latest rustc your distribution provides, they will most likely use the same llvm version. You'll be happy to hear that Mozilla uses clang+rustc with cross language lto for Firefox. The same should be possible for the kernel. I didn't try it, yet.

If you still prefer an gcc based Toolchain, you'll be happy to hear that rustc with gcc backend (via libgccjit) is not far away - Q3 2024 would be my guesstimate. I'm positive it will be available before rust is required for bcachefs. It will also allow lto and afaik also cross language lto.

[Weasel]

By that logic you'd be inconsequential to users as well by requiring a C compiler if people don't code in C, since you could simply use bare assembler

That's correct for a project that started off in assembly...?

Linux was always C so it required it since day 1.

And btw, building C can be quite a nightmare in some cases, not the kernel though because it doesn't have dependencies.

[marios]

If you're interested in Link time optimisation, clang is your best choice (due to build performance and memory requirements).

The reasoning in the parenthesis is irrelevant (unless the performance of the output binary is identical among alternatives), LTO sacrifices lots of build performance (which happens once) in order to gain runtime performance (which is expected to be reaped among lots of runs). So always choose the compiler that produces the (even marginally) fastest binary. That being said, currently for Linux, clang/llvm LTO is indeed the way to go, because for some annoyingly stupid reason gcc LTO has not been mainlined.

If you use the latest clang and latest rustc your distribution provides, they will most likely use the same llvm version.

Not true for rustc. A specific version of clang uses the same version of llvm. Actually, the llvm repository includes llvm, clang, lld and other stuff. Rustc however is out of tree and often does not support the latest llvm. It has happened more than once on my Gentoo pc. Things would have been different if rustc was in the llvm tree, but it isn't.

You'll be happy to hear that Mozilla uses clang+rustc with cross language lto for Firefox. The same should be possible for the kernel. I didn't try it, yet.

I know that about firefox. Actually my firefox is compiled that way. I never doubted the possibility of doing it in the kernel. What I said is that it restricts the toolchain choice, even excluding versions of the llvm toolchain. See my above point.

[oleid]

The reasoning in the parenthesis is irrelevant (unless the performance of the output binary is identical among alternatives), LTO sacrifices lots of build performance (which happens once) in order to gain runtime performance (which is expected to be reaped among lots of runs). So always choose the compiler that produces the (even marginally) fastest binary. That being said, currently for Linux, clang/llvm LTO is indeed the way to go, because for some annoyingly stupid reason gcc LTO has not been mainlined.

You must mean something different, I guess, because gcc LTO is mainline for years.
When I said clang is the way to go, I especially mean for developers, end users not that much. With clang you can easily use multi threaded link time optimization that scales very well with the cores. But most importantly, gcc is a lot more memory hungry when it comes to LTO. So memory may be a limiting factor here.

Not true for rustc. A specific version of clang uses the same version of llvm. Actually, the llvm repository includes llvm, clang, lld and other stuff. Rustc however is out of tree and often does not support the latest llvm. It has happened more than once on my Gentoo pc. Things would have been different if rustc was in the llvm tree, but it isn't.

Well, yes, if you live on the bleeding edge, the latest llvm version might not get used for rustc right away - they stabilize and test first to prevent regressions. Nevertheless, it is quite easy to pick a combo of clang+rustc that allows cross-language LTO. That's what I wanted to say.

I know that about firefox. Actually my firefox is compiled that way. I never doubted the possibility of doing it in the kernel. What I said is that it restricts the toolchain choice, even excluding versions of the llvm toolchain. See my above point.

I understand your point, but I was wondering if it is really a problem in praxis.