Linus Torvalds: Rust For The Kernel Could Possibly Be Merged For Linux 5.20

[mdedetrich]

The lack of language support for HKTs is also frustrating. You can kinda hack them in and somehow works, but it's far from ideal. I was hoping the likes of Amazon and Microsoft getting involved would speed things up, but sadly it seems a bit like Rust development has somewhat stagnated.

As someone that programs in languages that have HKT's (Scala, bit of Haskell in the past) you are unlikely to find an ergonomic implementation of HKT's in Rust because it would require a lot of boxing behind the scene's which is against one of the main design goals of Rust/C++ (zero cost abstraction).

Afaik its also the same reason where there is so many issues with async + traits in Rust currently.

I am also very keen on Rust, but I think these shortcomings that a lot of pure functional programmers notice you will just have to accept because the whole point of Rust is that its a low level language that gives you precise control of memory allocation and also doesn't do silent techniques behind the scenes (such as boxing) just for ergonomic reasons.

If you are expecting these things in Rust than Rust is not the language for you (not saying Rust shouldn't try to adopt HKT's, but just adjust your expectations around it).

[oleid]

Some people asked on twitter about this https://twitter.com/fasterthanlime/s...34725185191937, can you explain more about why Tokio is far from ideal for single threaded async ?

Check out this discussion:
https://www.reddit.com/r/rust/commen...e_the/ibqc0bw/

Short summary:

• When configuring Tokio to use a single threaded executor your futures still need to be Send and 'static (although we are in a single threaded context where this shouldn't have to be required).
• You can, however, use a workaround via LocalSet, however that's a 2nd class citizen and more verbose than for other executors

[Developer12]

It is highly probable that the Native GCC frontend is done enough in the beginning of 2023. I don't see why the extreme rush to merge rust support still in 2022.

They want it merged because continuing to port it to new kernels sucks.

It has language bindings that rely on every single kernel subsystem not changing, or they break. Each time there's a change anywhere the bindings need to be adapted to match it. This is a pain that is intentionally inflicted on all out-of-tree code but rust pays for it across every part of the tree it touches, which is way more code than usual compared to some random driver.

The sooner they're in-kernel, the sooner they don't face a hailstorm of breaking changes from every other kernel developer combined.

[NobodyXu]

Same. I really like Rust but async is FUBAR to the point is being better ignoring it. Problem in the userland level is that loads of basic crates you NEED will force async on you, in most cases pushing for a specific runtime too. Good luck with the x-lib interop on that.

Doesn matter for the Linux kernel, since it has its own async executor/locking mechanism/io types, so it only has one async environment for rust to support.

[Almindor]

I like Rust but it sucks that it doesn't yet have async functions in traits. 😭️
It also sucks that async is shattered islands with non-interoperable packages that depend on a specific async runtimes. 😭️

Same. I really like Rust but async is FUBAR to the point is being better ignoring it. Problem in the userland level is that loads of basic crates you NEED will force async on you, in most cases pushing for a specific runtime too. Good luck with the x-lib interop on that.

[sinepgib]

Heeee ? Every high level language gets compiled to turn into assambler the machine language, thats what compilers are for, because nobody nowdays knows how to realy code assembler anymore, and even if it would take a horrible amount of time to write a simple programm in assembler, so i guess thats just not true. Thats why you also need multiple tools compiler truns it to assembler then the linker runs that code again and forms the actual binary wich then is just a jumbo mumbo of hex data, basically unreadable by a human being.

Since you generally* don't read the assembly output, the compiler can turn your C code into the jumbo mumbo of hex data without going through the intermediate text representation. Which is also a bit of a lie because compilers use their own low level representation in many cases.

*Unless you specifically want to, in which case you pass a flag to the compiler so it outputs it.

Besides, there are sections of code where you can't avoid assembly. Some parts of the interrupt handling and setting of memory mappings are often done in assembly (tho I think GCC does provide extensions for most of it), and some performance sensitive code is written with explicit usage of SIMD extensions in plain assembly.

[sabian2008]

Heeee ? Every high level language gets compiled to turn into assambler the machine language, thats what compilers are for, because nobody nowdays knows how to realy code assembler anymore, and even if it would take a horrible amount of time to write a simple programm in assembler, so i guess thats just not true. Thats why you also need multiple tools compiler truns it to assembler then the linker runs that code again and forms the actual binary wich then is just a jumbo mumbo of hex data, basically unreadable by a human being.

Although I am not a programmer, I think he probably means that you might have C code which is irrelevant and hence the compiler when enabling optimizations might discard it at the IR level, effectively generating no Assembly code for that block of code.

He was probably being pedantic, like this whole discussion about whether Linux has one or two languages (and which in, btw, if you count Assembly, you probably have several different Assembly languages -- I imagine that all the supported micro-architectures must have at least a loader written in Assembly--).

[erniv2]

C code doesn't always gets turned into assembly, it just depends on the compiler

We disagree on that view.

Heeee ? Every high level language gets compiled to turn into assambler the machine language, thats what compilers are for, because nobody nowdays knows how to realy code assembler anymore, and even if it would take a horrible amount of time to write a simple programm in assembler, so i guess thats just not true. Thats why you also need multiple tools compiler truns it to assembler then the linker runs that code again and forms the actual binary wich then is just a jumbo mumbo of hex data, basically unreadable by a human being.

[sinepgib]

Some people asked on twitter about this https://twitter.com/fasterthanlime/s...34725185191937, can you explain more about why Tokio is far from ideal for single threaded async ?

Not sure what OP means, but while "far from ideal" is something I have no clue about, it does not special case it, so everything it does to avoid races, deadlocks, etc, when running the runtime with multiple threads is still present when running a single one, i.e. you pay some extra overhead despite not having these issues.
You'd generally use an async runtime for two reasons:
1. It's enough for IO to be faster, and for that it's even more efficient than threading (that's why asyncio exists in Python, even tho CPython can't do multicore);
2. It's a nice abstraction to provide thread-per-core in the background (reducing OS scheduling overhead).

If you have only one core, you want a single thread, and you only use async for the case 1. But because of that your await calls should be sufficient synchronization (since otherwise you're already risking a deadlock).

[evasb]

There's nothing special about a major version of the linux kernel. There's no more difference between 4.9 and 5.0 than between 4.5 and 4.6. Linus merely rolls the major digit over whenever they hit 19-20 to avoid having linux x.20 getting confused with x.2 or some stupid reason like that.

As for mandating rust? Not even remotely going to happen. First there needs to be rust support for all the targets linux supports. Right now rust code can only be compiled using LLVM, which isn't even close. There's work on a GCC backend, but there's no way it'll be ready by the following kernel release.

It is highly probable that the Native GCC frontend is done enough in the beginning of 2023. I don't see why the extreme rush to merge rust support still in 2022.