Google Wants To See Rust Code In The Linux Kernel, Contracts The Main Developer

[andreano]

panic (…) Linus made it pretty clear this is a big problem and THE biggest showstopper for integrating Rust

So I heard, but it would surprise me if this can't be solved. My understanding is that this is exactly the same problem as exceptions in C++: Proving that you don't have any is not generally possible, other than by grepping all the source (good riddance, external dependencies).

Think what you would do in C++: Disable exceptions, of course. I have only found limited ways to disable panic, but I think this just has to become possible. I see no other way around it in the long term. Maybe as an emergency solution, you could have a custom prelude to (re)define the panic macro to something that doesn't compile or link.

And before anyone mentions the #[no_panic] attribute, adding an attribute to every single function – except the ones you forget – is obviously no solution.

I think Rust made the same mistake as C++ (noexcept) here: Strayed from the otherwise so pervasive principle of doing the safest thing by default: They should obviously have made a #[panics] attribute instead, which very few functions would need. Huge mistake!

[NobodyXu]

So I heard, but it would surprise me if this can't be solved. My understanding is that this is exactly the same problem as exceptions in C++: Proving that you don't have any is not generally possible, other than by grepping all the source (good riddance, external dependencies).

Think what you would do in C++: Disable exceptions, of course. I have only found limited ways to disable panic, but I think this just has to become possible. I see no other way around it in the long term. Maybe as an emergency solution, you could have a custom prelude to (re)define the panic macro to something that doesn't compile or link.

And before anyone mentions the #[no_panic] attribute, adding an attribute to every single function – except the ones you forget – is obviously no solution.

I don’t think this is going to be a problem in linux kernel.

FIrst off, it is unlikely that linux kernel uses any external crate.
The only thing external it is going to use is core standard library, which excludes anything that panic or allocate anything.

Second, even if panic is used in kernel, it can be easily handled using https://doc.rust-lang.org/std/panic/fn.set_hook.html, which allows linux to call into the c implementation of panic and do whatever the kernel is configured to do.

Linux usually either print a stacktrace under panic (with symbol name if you configured it to include symbols) or just kill the machine straight away.
To ability to panic is already in linux for a long time, so IMHO this is not as serious as it seems.

The only problem I see with panic is that it involves allocation of memory, which means it is not a good idea to call it upon OOM situation.
If the rust can removes the memory allocation required in panic, then I think it can be used in linux kernel, given that linux uses a custom panic set_hook.

[NobodyXu]

Warning: long read.

Thank you very much for the link.

It seems like swift took a vtable approach that provide a getter for every field.
Doing so allows the layout to be modified to the point that you can easily replace a field with a getter method that returns a temporary value.
Since getter can return a reference while there is no field backing it, it performs “write-back” after modifing the value.
And here the “write-back” seems to be just bitwise-copy that doesn’t concern the copy/move constructor/assignment.

Templates in swift in an external library seems to be a polynomial function which accept polynomial types.

So in general, swift’s solution for dynamic library is just like java, it uses polynomial for ABI stability.
But unlike java, these polynomial behavior is only used between library/binary boundaries, but not inside the library, where the compiler can still optimize as it pleases.

This gives swift good performance/small code, while does not have to resort to JVM, which is a memory hug and battery eater.

I’m not suprised to see rust did not adopt this approach.

After all, it doesn’t fit with rust’s zero-cost abstraction in that it implicitly introduces heap allocation/vtable/ref counting etc.
And, many crates in rust are really small utility functions and making them polynomial would badly hurt performance and even generate larger binary size.

Though for big libraries, like GUI libraries, it is indeed preferrable to use polynomial interface to have a stable ABI.

It seems to be one that this kind of polynomial, stable ABI can definitely be made in rust.
It is also possible to write a macro to create a getter to return a “reference wrapper” for every field to simulate swift’s field access.

But due to the fact that rustc still isn’t stable and might introduces breaking changes, such practice is currently adviced against.
Once rust have stable ABI, I think it is possible to make such ABI.

[silmeth]

It seems to be one that this kind of polynomial, stable ABI can definitely be made in rust.
It is also possible to write a macro to create a getter to return a “reference wrapper” for every field to simulate swift’s field access.

But due to the fact that rustc still isn’t stable and might introduces breaking changes, such practice is currently adviced against.
Once rust have stable ABI, I think it is possible to make such ABI.

Rust in general does have built-in mechanism for runtime polymorphism, called trait objects – but as all non-zero-cost abstractions it is explicit and opt-in. Behind the scenes trait-object pointers compile down to pair of data pointer and vtable pointer.

But even if Rust had a guaranteed stable ABI, trait objects would still not fix the dynamic libraries problem in general, because in Rust this is fundamentally different mechanism than monomorphised generics (accepting a generic parameter and accepting a trait object in Rust are two different concepts, and not all traits can be made into trait objects, see the section on object safety of traits in the linked book).

And then there’s also all the other information encoded in Rust function and type signatures, like lifetime annotations, pointer types (mutable reference, shared reference, owned heap pointer, raw pointer…) that are parts of Rust APIs essential for ensuring safety and correct usage that get compiled away – though that stuff could be included in some metadata part of the compiled dynamic library.

[Charlie68]

Both Go and Rust are released under open source licenses. The reason why Google prefers Rust for the kernel is because it is more suited to the specific purpose. This isn't the first time that Google has been supporting open source software, I really don't see why it's surprised.
Google may like it or not, but it is certainly a company that has often contributed to open source software and I hope that more and more companies do, rather than wanting to lock themselves up in a world of "community" software.
Among other things, the news should not surprise anyone, it is not the first time that Google has declared that it wants to use Rust in the kernel.

[ssokolow]

I think Rust made the same mistake as C++ (noexcept) here: Strayed from the otherwise so pervasive principle of doing the safest thing by default: They should obviously have made a #[panics] attribute instead, which very few functions would need. Huge mistake!

The problem is that, aside from the noteworthy "Linux overcommit yanked the rug out from under our memory allocator" or "Bus error" cases which are hand-waved as programmer error in the standard library for userland code, panicking is meant to be used like unsafe... a sign of an invariant that, if broken, is a programmer-induced bug, and something that should not propagate upward in the API because everything depends on it eventually.

For example, for parsing a string constant into an IP address struct and then panicking on failure. Returning a Result<T, E> from that would be useless noise.

Think about how useless a "transitively calls unsafe code" annotation would be, given that any call to an OS API will depend on it.

The proper solution is to revive development of Rustig, which analyzes code paths in the built binary and allows you to control what it complains about with a whitelist.

[ultimA]

We were talking about aliasing, not alignment, were we?

You're right. Brainslip on my part.

[moltonel]

No, a proper rust frontend for gcc or gtfo.

I'll bite : how is rustc not a proper frontend for gcc ? I've listed a few technical and social reasons why the gccrs frontend can never be as good as the rustc frontend, but you seem to have a strong redeeming argument in favor of gccrs and it's not clear what that argument is.

It's ok to not like Rust (to each his own), and the current lack of a Gcc backend causes real problems, which can induce a cocktail of bad emotions. But IMHO if you're going to reluctantly live with Rust, you should pragmatically choose the most trouble-free way to do so. Gccrs is going to be a rough ride, whichever way you look at it.

[NobodyXu]

Rust in general does have built-in mechanism for runtime polymorphism, called trait objects – but as all non-zero-cost abstractions it is explicit and opt-in. Behind the scenes trait-object pointers compile down to pair of data pointer and vtable pointer.

But even if Rust had a guaranteed stable ABI, trait objects would still not fix the dynamic libraries problem in general, because in Rust this is fundamentally different mechanism than monomorphised generics (accepting a generic parameter and accepting a trait object in Rust are two different concepts, and not all traits can be made into trait objects, see the section on object safety of traits in the linked book).

And then there’s also all the other information encoded in Rust function and type signatures, like lifetime annotations, pointer types (mutable reference, shared reference, owned heap pointer, raw pointer…) that are parts of Rust APIs essential for ensuring safety and correct usage that get compiled away – though that stuff could be included in some metadata part of the compiled dynamic library.

Maybe libraries who want generics to be polynomial as in swift can make the generics a wrapper of the actual implementation that takes trait objects.
They can even implemented a macro for it that automatically wraps the actual implementation with a generic, or they can just export the polynomial version directly.

Regarding the use of `Self` of Traits, I think the use of it is not common. And if one is decided to use polynomial, they can simply replace it with `&dyn Trait` for `&Self` or `Box<dyn Trait>` for `Self`.

Regarding the lifetime, I don’t think they ever need to be encoded in the ABI.
They only need to present in the module (not dynamic library) and let the compiler know about the lifetime at compile time.

On the pointers, I am pretty sure they are not part of the ABI.
I just compiled my personal crate in debug mode and `objdump -d | rustfilt` it and none of these functions include any argument type like in C++, but they do includes namespaces.

I think this is because rust doesn’t support function overloading, so no need to have argument type in ABI to make it longer.

At the ABI level, I guess all reference and primitive pointers are represented using pointers, passed in the registers and the Box are treated like a struct, which usually either get divided into multiple parts to fit in a register, or pass in the stack if it is too large.

Since a Box<dyn Trait> consists of two pointers, one pointer to the object and the other to the vtable, I guess it will be passed in registers.

I just did a little bit of googling and it turns out rust support downcasting using a crate called any, so it is actually possible to simulate the “resilient layout” by passing `Any` to the API and the library internally cast it to back to its actual type, so it can still optimize as it pleases.

If it is a generic, then it would have to use the vtable.
Still, the compiler can replaces some dynamic dispatching with switching + inlining code for classes known to the library plus a default fallback to vtable.
Basically just like how C++ do the stable ABI via polynomial objects or by using PImpl https://en.cppreference.com/w/cpp/language/pimpl for struct methods.

[ssokolow]

I just did a little bit of googling and it turns out rust support downcasting using a crate called any, so it is actually possible to simulate the “resilient layout” by passing `Any` to the API and the library internally cast it to back to its actual type, so it can still optimize as it pleases.

It's not perfect... and it's actually part of the standard library. (The std::any module)

Building a runtime reflection system for Rust 🦀️ (Part 1) is a good starting point.