Linux's VGEM Kernel Driver Being Rewritten In Rust

[sinepgib]

You are right to say that it happens mainly in web development, which seems to be driven almost entirely by fads anyway. But I don't think that it applies to system languages. Of those that vanished, PL/1 and BCPL were never fads and D was never a fad either (and it's still alive and well, albeit on the sidelines). There was the Modula-2 family and that's pretty much it. I suppose one could say that Lisp-as-a-system-language was also a fad of sorts, but even that never really became mainstream. As far as Rust is concerned, I think it's now past the Too Big To Fail milestone. While I agree with being cautious about passing fads, being overly skeptical and holding off indefinitely until something is sufficiently "proven" (which by definition never happens) isn't a sensible policy either IMHO. Even if Rust were to somehow disappear or become totally forgotten, rewriting those parts in C or in whichever other language would pop up wouldn't be wasted effort. In fact it would almost certainly result in a better quality implementation than if it was written in C from the start.

I agree with most of this part, except about rewriting not being wasted effort if Rust disappeared. You often lose track of edge cases you found during the lifetime of a significant piece of code and the rewrites tend to ditch that, so you have to discover those bugs all over again. With Rust it makes sense because the kind of bugs it tends to prevent are also typically of much higher severity, but for a rewrite from Rust you'd need at least something equally safe and then much better in some other aspect to not call it a waste of time.

I'm not aware of any in-depth study on the subject, but from some anecdotal tests and benchmarks I've done, Rust often autovectorises better than Clang or Clang++, presumably because of its const-by-default and one-single-mutating-pointer-at-a-time rules.

It would make sense, but I also saw no experiments to judge. It would make an interesting read if you had some of those handy.

[jacob]

I see how it is a bit fallacious, but you have to acknowledge that many languages were oh so popular for a few years and then faded away in that time. IMO that happens mostly in the web backend space, but I see reasons to be skeptical in history. Rust is at its height in popularity right now. I don't think it's going away and I don't think critical infrastructure should be written in C (or COBOL lol). If anything, writing critical infrastructure in Rust is what will make it stay around after the ADHD of our collective mindshare finds some new shiny toy.

You are right to say that it happens mainly in web development, which seems to be driven almost entirely by fads anyway. But I don't think that it applies to system languages. Of those that vanished, PL/1 and BCPL were never fads and D was never a fad either (and it's still alive and well, albeit on the sidelines). There was the Modula-2 family and that's pretty much it. I suppose one could say that Lisp-as-a-system-language was also a fad of sorts, but even that never really became mainstream. As far as Rust is concerned, I think it's now past the Too Big To Fail milestone. While I agree with being cautious about passing fads, being overly skeptical and holding off indefinitely until something is sufficiently "proven" (which by definition never happens) isn't a sensible policy either IMHO. Even if Rust were to somehow disappear or become totally forgotten, rewriting those parts in C or in whichever other language would pop up wouldn't be wasted effort. In fact it would almost certainly result in a better quality implementation than if it was written in C from the start.

I didn't mean that in comparison to C or C++ FWIW. I don't think it's really that much worse (if worse at all) compared to those, after all it's the same code generator and the iteratoanecdoticallyr style probably helps in generating more obviously vectorizable code (citation needed). But compilers in general tend to need quite a bit of help to vectorize the non-trivial, you need to structure your code thinking about how you can lead it to vectorize or directly tell it how via intrinsics. For that case it just makes about as much sense to hand-roll your own assembly.

I'm not aware of any in-depth study on the subject, but from some anecdotal tests and benchmarks I've done, Rust often autovectorises better than Clang or Clang++, presumably because of its const-by-default and one-single-mutating-pointer-at-a-time rules.

[ssokolow]

All hand vectorization is niche. But the thing about niches is that they're irrelevant until you need them but they become critical when you do. Note this is not at all a criticism to Rust. Besides, Rust has decent support for inline assembly from what I'm told (haven't had to use it, so I can't confirm).

Yes. It took them several years to stabilize though, because they wanted to pin down a syntax that wouldn't be tied to the underlying LLVM implementation details and could be made part of the compiler-independent language spec language.

As an aside, I wouldn't recommend inline assembly in C at all unless you definitely support one compiler only like Linux does or it's trivial enough to write an implementation per compiler. It's implementation defined and often incompatible between compiler, it's not very well defined, etc. Just write a function in a .s and link it like civilized people do.

*nod* The only place I use inline assembly is my DOS retro-hobby project, which currently depends on Open Watcom C/C++ as its only mandatory dependency (eg. Build automation must use only Watcom Make and, if that's not enough, helper "scripts" written in C).

[sinepgib]

...and the use-cases for actual assembly language instead of compiler intrinsics (which are exposed by Rust) have become so niche that MSVC doesn't allow inline assembly on 64-bit targets.

All hand vectorization is niche. But the thing about niches is that they're irrelevant until you need them but they become critical when you do. Note this is not at all a criticism to Rust. Besides, Rust has decent support for inline assembly from what I'm told (haven't had to use it, so I can't confirm).
As an aside, I wouldn't recommend inline assembly in C at all unless you definitely support one compiler only like Linux does or it's trivial enough to write an implementation per compiler. It's implementation defined and often incompatible between compiler, it's not very well defined, etc. Just write a function in a .s and link it like civilized people do.

[sinepgib]

That is a fallacy, although I'm surprised how common it is. But by that logic, critically important new software should be written in COBOL because it has been around much longer than C. I'm not saying that Rust should always be the automatic choice, there are still cases where C is the better option (platform support, extreme memory constraints or the need to be entirely binding-friendly to the lowest common denominator), but the fact that C is older is neither here or there.

I see how it is a bit fallacious, but you have to acknowledge that many languages were oh so popular for a few years and then faded away in that time. IMO that happens mostly in the web backend space, but I see reasons to be skeptical in history. Rust is at its height in popularity right now. I don't think it's going away and I don't think critical infrastructure should be written in C (or COBOL lol). If anything, writing critical infrastructure in Rust is what will make it stay around after the ADHD of our collective mindshare finds some new shiny toy.

It definitely isn't there yet, but in many cases Rust already does a better job than C or C++ at autovectorising.

I didn't mean that in comparison to C or C++ FWIW. I don't think it's really that much worse (if worse at all) compared to those, after all it's the same code generator and the iterator style probably helps in generating more obviously vectorizable code (citation needed). But compilers in general tend to need quite a bit of help to vectorize the non-trivial, you need to structure your code thinking about how you can lead it to vectorize or directly tell it how via intrinsics. For that case it just makes about as much sense to hand-roll your own assembly.

[ssokolow]

It definitely isn't there yet, but in many cases Rust already does a better job than C or C++ at autovectorising.

...and the use-cases for actual assembly language instead of compiler intrinsics (which are exposed by Rust) have become so niche that MSVC doesn't allow inline assembly on 64-bit targets.

[jacob]

There are logical reasons to restrict Rust though. As with pretty much everything in engineering, there is a tradeoff and different programmers will tilt toward different decisions. For example, lifespan is a reasonable argument: compared to C, Rust is young, which means C is a (slightly, IMO) better bet for expecting it to be around in 20-30 years from now, simply because it already lived such long times.

That is a fallacy, although I'm surprised how common it is. But by that logic, critically important new software should be written in COBOL because it has been around much longer than C. I'm not saying that Rust should always be the automatic choice, there are still cases where C is the better option (platform support, extreme memory constraints or the need to be entirely binding-friendly to the lowest common denominator), but the fact that C is older is neither here or there.

Yeah, I don't know how I missed this part really, as it is the most obvious one. But the parts that absolutely need assembly language tend to be few and small and are already not C (I mostly assume we only mean to rewrite the C parts, I may be wrong). Some will also be kept as assembly for performance, the sufficiently smart compiler that vectorizes complicated algorithms isn't yet here AFAIK.

It definitely isn't there yet, but in many cases Rust already does a better job than C or C++ at autovectorising.

[mether]

This effort exists but has not been publicly announced to my knowledge.

I am not talking about a rust compiler. I am talking about a standards-compliant compiler that compiles C code and has been written in Rust.

If it is not public knowledge, then you don't have a source you can point to? There isn't anything interesting to discuss in private/secret projects. Many toy projects to compile C using Rust already does exist publicly.

[Developer12]

No such effort exists of course. a) mrustc (entirely in C++) and GCC RS (reusing Rust libraries outside of the compiler) are written in C++ as I have noted earlier in the thread b) Even if there was an effort, C is a pretty common language to write compilers. Google's Go compiler itself was originally written in C and Python's reference compiler, CPython continues to be written in C today as the name indicates although alternatives like pypy are self hosting.

This effort exists but has not been publicly announced to my knowledge.

I am not talking about a rust compiler. I am talking about a standards-compliant compiler that compiles C code and has been written in Rust.

[Almindor]

There are logical reasons to restrict Rust though. As with pretty much everything in engineering, there is a tradeoff and different programmers will tilt toward different decisions. For example, lifespan is a reasonable argument: compared to C, Rust is young, which means C is a (slightly, IMO) better bet for expecting it to be around in 20-30 years from now, simply because it already lived such long times.



Yeah, I don't know how I missed this part really, as it is the most obvious one. But the parts that absolutely need assembly language tend to be few and small and are already not C (I mostly assume we only mean to rewrite the C parts, I may be wrong). Some will also be kept as assembly for performance, the sufficiently smart compiler that vectorizes complicated algorithms isn't yet here AFAIK.

I bet COBOL was in the same ship as C is now, albeit with much less actual software written in it.

As for the assembly, yes there are a few places where it is needed. The initial runtime setup needs to be in assembly for example (because basic memory isn't even "set up" yet e.g. stack pointer or frame pointer). The nice thing about doing these in Rust though is that you minimize the asm surface as much as possible, link it together and unsafe only things like CSR access, preferably with very well tested generated code.

Most other things can then be safe rust which makes for great safety guarantees.