Rust For Linux Kernel v9 Patches Trim Things Down Greatly For Easier Upstreaming

[NobodyXu]

Just because a project implements dynamic dispatching (and in C of all cases) doesn't make it OOP-style.

Read this please:

Object-oriented design patterns in the kernel, part 1 - https://lwn.net/Articles/444910/

Compilers are actually known to optimize away even vtables completely if they can prove there are no current and future users for it. In a kernel application and after devirtualization this probably means compiling with LTO, which has only been possible since relatively recently with the Linux kernel, but nevertheless possible. We can also reasonably speculate that it would have gotten implemented much earlier if there had been a language binding in the kernel greatly benefiting from it.

Not if the object is reference counted or is put into some kinds of container, which is extremely hard for compiler to remove the vtable pointer.
And the Linux kernel supports kernel module and anything used in the API must be stable and the compiler cannot just remove a field from it.

Thus, in practice, this devirtualization might not work that well.
When you opt-in to use abstract class or virtual method in C++, you must consider its cost and cannot expect it to be zero-cost without some analysis on its specific use cases.

[ultimA]

#include <utility>

template <class T, class ...Args>
auto initialize(Args &&...args) -> T {
T{std::forward(args)...}
}

People often write generic code that requires initialization of an object and having overloaded meaning when writing "T{...}" really isn't helping.
If the type "T" has an initializer list constructor, things get much more complicated and cause confusion.

So people just keep using the old "T(...)" instead.

Now you are getting into a different territory. First, the kind of code you showcase would be a library function even in the kernel and would be replacing macros today, except being typesafe.

Second, you really think people won't write hard-to-read code in Rust too? The code you show as an example is not something people would normally write even in C++.

I also strongly disagree that it would be better to use template magic in C++ to emulate traits in Rust. I have shown in my posts above that you can perfectly and efficiently emulate traits in C++ with a simple, easy-to-implement and easy-to-read syntax (using inheritance and compiler optimizations). Why the hell would anyone want to use templates for this? This seems very contrived only to try to prove C++ must be harder to understand than Rust even in cases where it really isn't.

[ultimA]

Not if the object is reference counted or is put into some kinds of container, which is extremely hard for compiler to remove the vtable pointer.
And the Linux kernel supports kernel module and anything used in the API must be stable and the compiler cannot just remove a field from it.

Thus, in practice, this devirtualization might not work that well.
When you opt-in to use abstract class or virtual method in C++, you must consider its cost and cannot expect it to be zero-cost without some analysis on its specific use cases.

The way I have described is easy in code and works reliably with current language and compiler tools. Hard for the compiler or not doesn't change the fact that it is already implemented. The only reason this doesn't work in the kernel is because there is no support for compiling C++ code in the kernel today at all. Of course if a vtable must be used in C++ due to actually needing dynamic dispatch then there will be an overhead, but Rust cannot get around that either. Even in Rust you either have a vtable in some way (manual or automatic) or you must refrain from using dynamic dispatch. There's no magic for that. But you can have interfaces without dynamic dispatch in C++ too as shown.

[Sergey Podobry]

#include <utility>

template <class T, class ...Args>
auto initialize(Args &&...args) -> T {
T{std::forward(args)...}
}

That's definitely not for initializing 10 elements with 0 because it has multiple arguments. Of course some language constructions may be misused due to lack of knowledge and experience. It's a valid point for any programming language (including Rust).

[Sergey Podobry]

Of course if a vtable must be used in C++ due to actually needing dynamic dispatch then there will be an overhead, but Rust cannot get around that either.

Moreover the Linux kernel already has vtables but implemented in C. With the same overhead as in C++. So it's not a big deal at all if used wisely.

[NobodyXu]

The way I have described is easy in code and works reliably with current language and compiler tools. Hard for the compiler or not doesn't change the fact that it is already implemented. The only reason this doesn't work in the kernel is because there is no support for compiling C++ code in the kernel today at all.

As I have said in my previous reply, I don't think it will work well because many data structures are reference counted or put into a container in the kernel.

Of course if a vtable must be used in C++ due to actually needing dynamic dispatch then there will be an overhead, but Rust cannot get around that either. Even in Rust you either have a vtable in some way (manual or automatic) or you must refrain from using dynamic dispatch. There's no magic for that.

If you use dynamic dispatching, then yes, Rust of course use a vtable.

My point about vtable not being zero-cost is that if you merely declare a method as "virtual" in C++ or inherit from an abstract base class but just use it as a regular object without dymamic dispatching, then the object still needs to be 8-bytes larger.

While compiler might be able to optimize it out, it does not guarantee that and it might not be able to optimize it if it is reference counted, put into a container, exposed in a public API or etc.

Hence it is not zero-cost.

[NobodyXu]

That's definitely not for initializing 10 elements with 0 because it has multiple arguments. Of course some language constructions may be misused due to lack of knowledge and experience. It's a valid point for any programming language (including Rust).

Yes, every language can be misused, but C++ is one that are particular easy to misuse and shoot yourself.
And its language design is inconsistent and lack of consideration.

[ultimA]

std::vector<int> v = std::vector{10};

Yes, in C++ you can init a vector not just with the number of elements, but also with specific values for each element. And you are misusing this feature as a bug to claim it is a design deficiency?

The coder could have written:

auto v = std::vector<int>(10);

Or, to explicitly communicate intent and avoid any misunderstandings:

std::vector<int> v;
v.insert(v.begin(), 10, 0);

Or, probably what he really wanted:

std::array<int, 10> arr;

There are parts of the C++ language that are actually weird, hard or confusing (but that doesn't mean they need to be used or cannot be abstracted away). What you show in your example though is not one of those parts, it is just a coding bug.

[ultimA]

As I have said in my previous reply, I don't think it will work well because many data structures are reference counted or put into a container in the kernel.
If you use dynamic dispatching, then yes, Rust of course use a vtable.

The case with multiple polymorphic objects of different types in the same container is a use-case that simply needs vtables. Rust would need it too in this case, so this is an invalid critique. Same goes for reference counting, you only need vtables there if you want to provide polymorphic reference counted pointers, and in that case Rust cannot get around that either. In C++ all you'd need is a smart pointer type that is not yet part of the STL, and bam you have refcounting without dynamic dispatch. Both C and Rust have or will have standard utils/library functions to support kernel devs, so a pointer class for C++ to be added like this wouldn't be unprecedented.

My point about vtable not being zero-cost is that if you merely declare a method as "virtual" in C++ or inherit from an abstract base class but just use it as a regular object without dymamic dispatching, then the object still needs to be 8-bytes larger.

While compiler might be able to optimize it out, it does not guarantee that and it might not be able to optimize it if it is reference counted, put into a container, exposed in a public API or etc.

Hence it is not zero-cost.

As I said, you can reliably trigger this optimization on any compiler today, be it clang, gcc or msvc (not as if msvc would be relevant for kernel dev). So what's the problem with relying on it? All released code today, including the Linux kernel, already rely heavily on existing compiler optimizations to achieve acceptable performance. Try compiling the kernel with optimizations turned off and you'll be shocked by your system's performance after that. As long as optimizations work more-or-less reliably, there is nothing wrong relying on them, and we already do this in all performance critical applications and libraries (including the kernel). Why make an exception for function devirtualization? Why is it bad to rely on function devirtualization but not bad to rely on (for example) constant propagation, branch elimination or instruction scheduling? Again, you are artificially placing constraints just so you can prove C++ is worse in this specific regard.

[NobodyXu]

The case with multiple polymorphic objects of different types in the same container is a use-case that simply needs vtables. Rust would need it too in this case, so this is an invalid critique. Same goes for reference counting, you only need vtables there if you want to provide polymorphic reference counted pointers, and in that case Rust cannot get around that either. In C++ all you'd need is a smart pointer type that is not yet part of the STL, and bam you have refcounting without dynamic dispatch. Both C and Rust have or will have standard utils/library functions to support kernel devs, so a pointer class for C++ to be added like this wouldn't be unprecedented.

I agree that in these cases, you need vtables none the less.

Though my point is just that virtual function in C++ is not zero-cost, since the objects have the 8-bytes vtable even if you don't use it.
It can be optimized out, but that optimization isn't guarantee to happen.

Consider the following code:

struct C {
    public virtual void f() {
        /* Contains so much code that compiler does not inline this function */
    }
}

void large_func(C &c) {
    /* Contains so much code that compiler does not inline this function */
}

In this case, the compiler would not be able to inline large_func and cannot inline C::f, so the compiler cannot optimize out the vtable pointer here.

As I said, you can reliably trigger this optimization on any compiler today, be it clang, gcc or msvc (not as if msvc would be relevant for kernel dev). So what's the problem with relying on it? All released code today, including the Linux kernel, already rely heavily on existing compiler optimizations to achieve acceptable performance. Try compiling the kernel with optimizations turned off and you'll be shocked by your system's performance after that. As long as optimizations work more-or-less reliably, there is nothing wrong relying on them, and we already do this in all performance critical applications and libraries (including the kernel). Why make an exception for function devirtualization? Why is it bad to rely on function devirtualization but not bad to rely on (for example) constant propagation, branch elimination or instruction scheduling? Again, you are artificially placing constraints just so you can prove C++ is worse in this specific regard.

You misunderstand me again.

I'm not saying that you cannot depend on optimization, I am simply saying that in C++, virtual function is not zero-cost.
Optimization here does not change the fact that it isn't zero-cost as it cannot eliminate the unused vtable pointer in all scenarios.