Rust Developers Move Ahead With Preparing To Upstream More Code Into The Linux Kernel

[moltonel]

Can you elaborate a little, please? I was under the impression Rust drivers would use the existing APIs (if wrapped inside unsafe blocks).

I'm not sure what your current (mis)understanding of the plans are, so I hope this clears up a few things:

• Driver code is kernel code, you can't claim "Rust will not be in the kernel at all" just because "it's only drivers". The 6.1 kernel is only getting support code with no driver or anything user-facing, but it's still a few thousand lines of Rust
• Rust code does of course call into existing C apis. But the wrappers are a bit more sophisticated than a straight unsafe binding: they often offer a more idiomatic Rust API and are safe to call. The goal is that most drivers can be written without any unsafe.
• While Rust can't be used for core parts of the kernel (yet), it's not limited to drivers. There's already a Binder (Android's message bus) implementation, a 9P (network filesystem) implementation, and talks about LSM, parsers, and many others. C code can call into Rust code.
• Rust probably wouldn't have been added to Linux if it couldn't (at least in theory) be used in every part of the kernel. Expect a flurry of experimentations by the time 6.2 comes out, as people challenge this theory. Only a fraction of this will get proposed for mainline, but it's important to uncover any fundamental language issue early.

[bug77]

I'm not sure what your current (mis)understanding of the plans are, so I hope this clears up a few things:

• Driver code is kernel code, you can't claim "Rust will not be in the kernel at all" just because "it's only drivers". The 6.1 kernel is only getting support code with no driver or anything user-facing, but it's still a few thousand lines of Rust
• Rust code does of course call into existing C apis. But the wrappers are a bit more sophisticated than a straight unsafe binding: they often offer a more idiomatic Rust API and are safe to call. The goal is that most drivers can be written without any unsafe.
• While Rust can't be used for core parts of the kernel (yet), it's not limited to drivers. There's already a Binder (Android's message bus) implementation, a 9P (network filesystem) implementation, and talks about LSM, parsers, and many others. C code can call into Rust code.
• Rust probably wouldn't have been added to Linux if it couldn't (at least in theory) be used in every part of the kernel. Expect a flurry of experimentations by the time 6.2 comes out, as people challenge this theory. Only a fraction of this will get proposed for mainline, but it's important to uncover any fundamental language issue early.

Ok, thanks, that makes sense.

I guess the misunderstanding is simply about what "in the kernel" means. To me, the drivers are not "in the kernel" because the kernel can exist (as in be compiled and boot up fine) without them. It makes them add-ons in my head. I guess it would more accurate to say that Rust is in optional kernel modules maybe?

I have my fingers crossed for Rust proving itself and taking over more critical parts. With memory safety taken care of, I think developers would have a much easier life. Then again, even if Rust proves itself, it's not like all the C code will automagically rewrite itself. Rust will have to coexist with C for a long, long time, so another guess would be that getting admitted into the kernel will depend a lot on how it integrates with C.

[NobodyXu]

I guess the misunderstanding is simply about what "in the kernel" means. To me, the drivers are not "in the kernel" because the kernel can exist (as in be compiled and boot up fine) without them. It makes them add-ons in my head. I guess it would more accurate to say that Rust is in optional kernel modules maybe?

Yes, Rust currently is opt-in since there're a few CPU archtecture it doesn't support and still being experimental support, but with rustc_codegen_gcc, it will be there.

But do note that M1's GPU driver is written in Rust, so if you want to build Asahi Linux kernel with GPU support, you might have to turn on the Rust support.

I have my fingers crossed for Rust proving itself and taking over more critical parts. With memory safety taken care of, I think developers would have a much easier life. Then again, even if Rust proves itself, it's not like all the C code will automagically rewrite itself. Rust will have to coexist with C for a long, long time, so another guess would be that getting admitted into the kernel will depend a lot on how it integrates with C.

Yes, it will take a long time to migrate.

[krzyzowiec]

I think you are confusing dynamic vs static typing with strong vs weak typing. Ruby, similar to Python, is strongly typed which means that if you do something like trying add 2 objects it will fail immediately. Now because its dynamically typed it happens at runtime but unlike Javascript, its not going to silently coerce the object to something else and actually try to add 2 nonsensical things.

wrt to nil (not null, Ruby doesn't have null) although there are problems with it, they still at least have syntax helpers to help deal with the problem, i.e. https://stackoverflow.com/questions/...uby-on-rails-2

It's not like trying to add two objects is a serious problem in JavaScript. Either you know about it and know what you should actually do, or you don't know the language at all.

[mdedetrich]

It's not like trying to add two objects is a serious problem in JavaScript. Either you know about it and know what you should actually do, or you don't know the language at all.

That has to be the stupidest and dumbest argument, especially considering the creator of Javascript said it was mistake from a language they rushed out in a week due to time pressure.

This kind of handwaving away retarded design decisions needs to stop, obviously this should never be done but considering how much Javascript is used, this mistake and others has likely wasted collective billions of hours because since Javascript is a dynamically typed language you may not even know that a reference is an object (or not) at runtime and rather than telling you immediately it can just silently run on giving you an even more cryptic and dumb error message later.

And this isn't even the only case, there are further ones that are even more cryptic and subtle to catch.

[moltonel]

I guess the misunderstanding is simply about what "in the kernel" means. To me, the drivers are not "in the kernel" because the kernel can exist (as in be compiled and boot up fine) without them. It makes them add-ons in my head.

I disagree here for various reasons: First, some drivers are necessary for booting even a "blind and mute" kernel : hardware buses and sometimes even cpu/memory need to be initialized for the kernel to properly use them. Second, the whole point of a kernel is to provide access to hardware: a kernel without drivers is strictly useless. Thrid, drivers are not the only components that are optional and runtime-loadable: network schedulers, filesystems, virtualization, you name it. Last, 2/3rds of the mainline kernel tree is drivers: go tell driver developers that their work isn't "in the kernel" and see what happens.

I guess it would more accurate to say that Rust is in optional kernel modules maybe?

Optional yes (though that will hopefully change someday, when compiler(s) support all the required platforms, and the language has proved itself in kernel developer's eyes). Modules not necessarily, as rust code doesn't have to be compilable as a module (AFAICT, none of the rust code merged in 6.1 can), and almost any code that can be compiled as modules can also be compiled built-in.

[bug77]

I disagree here for various reasons: First, some drivers are necessary for booting even a "blind and mute" kernel : hardware buses and sometimes even cpu/memory need to be initialized for the kernel to properly use them. Second, the whole point of a kernel is to provide access to hardware: a kernel without drivers is strictly useless. Thrid, drivers are not the only components that are optional and runtime-loadable: network schedulers, filesystems, virtualization, you name it. Last, 2/3rds of the mainline kernel tree is drivers: go tell driver developers that their work isn't "in the kernel" and see what happens.



Optional yes (though that will hopefully change someday, when compiler(s) support all the required platforms, and the language has proved itself in kernel developer's eyes). Modules not necessarily, as rust code doesn't have to be compilable as a module (AFAICT, none of the rust code merged in 6.1 can), and almost any code that can be compiled as modules can also be compiled built-in.

A kernel will also not boot without hardware to run on and power for that hardware. But we're not saying hardware and electricity are in the kernel, are we?
Anyway, I think we're in agreement, we're just splitting hairs now.

[cj.wijtmans]

Well maybe you happen to be lucky. But that is an exception, not a rule. Postgresql won't build on freebsd with clang-14, Mozilla has been complaining how trying to accommodate 3 different C++ compilers for their official Firefox builds (VC++ for Windows, G++ for Linux, Clang++ for MacOS) generates substantial additional work, Chromium prefers to ship its own clang, Linux won't work with anything but gcc (and now clang too, but that was a MAJOR undertaking), a number of packages written in C or C++ in the BSD ports tree still have a hard requirement on GCC, etc.

The reality is that trying to write code that is compiler independent takes extra effort which is hardly justified. It was another story when there was a much larger number of relevant platforms than today and each had its own proprietary C compiler (or compilers). But for a new programming language that comes with a compiler which is free, that anyone can have, use, port, adapt, fork etc. at will, which already runs on the most important OSes and architectures (although yes, arch support is still a weakness of Rust compared to C), I really don't see the point of demanding multiple subtly different implementations just because reasons. What practical problem would that solve? I don't see any, but I see a number of problems it would introduce.

The only thing I can think of in terms of "accommodating" is compiler flags. Either that or they are using a lot of C++ extensions that are non-standard. Most likely in order to try and optimize their code. But then you might as well complain that when you use assembly to optimize that you have to accommodate different architectures. Mean while they most likely refuse to use new C++ standards that standardized some perfomance related things like hinting. It's a non-argument because they created the issue themselves in the first place. For example, Attribute specifier sequence(since C++11) - cppreference.com. "Attributes provide the unified standard syntax for implementation-defined language extensions, such as the GNU and IBM language extensions __attribute__((...)), Microsoft extension __declspec(), etc."

Let's also not forget another factor: writing pure "standards compliant" code was much less of a problem back in the 80s or even the 90s. But user expectations have moved on since then and software complexity has increased by orders of magnitude - and for good, justified reasons, so KISS is not the answer. By the same token some of the then-standards have became basically immaterial (like POSIX). You can still write "hello world" in pure standards compliant C, but if you are developing say LibreOffice, choosing your compiler for the project is as important as choosing the UI framework it will use. After all, if someone wants to rebuild it himself, telling him "you must use g++" is a perfectly valid choice.

This has to be a joke.