Rust Support In The Linux Kernel Undergoing Another Round Of Discussions

[Alexmitter]

As long as Rust can not be compiled via GCC, any discussion on adding rust code to the kernel is a dead end and waste of time.

[uid313]

What parts of the Linux kernel would benefit the most from Rust?

I can imagine WireGuard would be nice to have in Rust. What else?

[M@yeulC]

What parts of the Linux kernel would benefit the most from Rust?

I can imagine WireGuard would be nice to have in Rust. What else?

Device drivers. There are a lot of them, and not everyone tests all of them. Some extra safety guarantees would be nice.

Edit: also, they are optional, so one could leave Rust disabled-by-default and optional for a while.

[M@yeulC]

Is memory safety so hard people had to create another language to address it?

Yes. C is not memory-safe by design. Some other, rather old programming languages are memory safe too: ADA/SPARK, and most scripting, garbage-collected languages

[bug77]

The real problem that is hard, is making a program that memory safe and thread safe. Multi-threading without unnecessary copying is hard.

But what is "unnecessary copying"?

Let's say you develop some library that operates on a vector. You split processing in two, one thread handles the first half of the vector, another thread takes care of the rest. It's "unnecessary" to copy the vector parts, because you simply pass the start and stop indices to each thread. But then some user of your library figures out how to manipulate those indices. Is the copying still "unnecessary"?

Not to mention copying in perfectly acceptable in some context. Just look at how Erlang went about things.

(NB I fully understand a systems programming language has to meet different requirements from more higher level languages.)

[9Strike]

Whether it's Unix friendly or unfriendly is neither here or there. It's user friendly, performance friendly and security friendly. That matters infinitely more than whether it conforms to an OS design that was crappy by 1972 standards, yet alone today.

No, it is user friendly. Yes it is if you compile stuff on your own download binaries randomly from the internet, it might not. But if you want to use stable software and install stuff from your repo, it is user unfriendly.

Also, if you say the shared libraries model from *all* distros is so crappy, you have no idea what you are talking about. It is one of the most important reasons Linux is so much more secure than Windows.

I have yet to see a single distro maintainer that thinks the Rust build and dependency system is good. It's not unfixeable, but the Rust people just don't care at all (but hey, who cares about security anyway, right?).

[ssokolow]

As long as Rust can not be compiled via GCC, any discussion on adding rust code to the kernel is a dead end and waste of time.

It's being worked on. (Through two different avenues, actually. The former being to rewrite a Rust compiler frontend for GCC in the same kind of C++ used for GCC itself so it'll be a first-class GCC citizen like any other native GCC frontend, and the latter being to glue the existing rustc frontend onto the GCC backend to get maximally-compatible-with-the-language results quickly.)

But what is "unnecessary copying"?

Let's say you develop some library that operates on a vector. You split processing in two, one thread handles the first half of the vector, another thread takes care of the rest. It's "unnecessary" to copy the vector parts, because you simply pass the start and stop indices to each thread. But then some user of your library figures out how to manipulate those indices. Is the copying still "unnecessary"?

In this context, I assume "unnecessary copying" would be the compiler just "making it work" rather than throwing a "Please clarify how you intended this to work" error.

[angrypie]

"Is coding so hard that people had to create another language to address it?" --> Criticisms of C by an Assembly programmer

"Is compiling so hard that people had to create another language to address it?" --> Criticisms of dynamic/scripting languages by a C programmer

"Is arguing so hard lyamc had to resort to slippery slope to say anything at all?"

[ssokolow]

No, it is user friendly. Yes it is if you compile stuff on your own download binaries randomly from the internet, it might not. But if you want to use stable software and install stuff from your repo, it is user unfriendly.

Also, if you say the shared libraries model from *all* distros is so crappy, you have no idea what you are talking about. It is one of the most important reasons Linux is so much more secure than Windows.

I have yet to see a single distro maintainer that thinks the Rust build and dependency system is good. It's not unfixeable, but the Rust people just don't care at all (but hey, who cares about security anyway, right?).

Let's be real about dependencies (seriously, that's the title). The Rust model is more amenable to having a single, auditable version of bits and pieces C and C++ applications rewrite or patch, which aren't suitable to be their own distro-level packages.

While we’re here, let’s look a little deeper into some of these programs. Part of my theory is that C programs commonly omit deps by re-implementing the bits they need themselves, because for small stuff it’s far easier to just write or copy-paste your own code than to actually use a library. Let’s rummage through some of these programs, briefly, and see if that’s the case.

• dash: Well, there’s a handful of linked lists and a simple memory allocator, but no gratuitous re-implementations of memcpy or anything like that.
• lighttpd: Aha, here we are. There’s a SHA1 impl, a base64 impl, a resizeable array, a URL parser, a CRC32 implementation, a layer over the poll(2) equivalent on various platforms, a LALR parser generator called LEMON (specifically designed to be vendor’ed into other code bases), an MD5 impl, a RNG that tries to get secure randomness from several different sources depending on the system, a safe_memclear() that hopefully hasn’t been broken by compiler changes since it was written, a splay tree, and another, different, resizeable array. Or, in terms of Rust crates: sha1, base64, std::vec::Vec, url, crc, mio, lalrpop or something, md5, rand, actually I can’t find a simple safe way to zero memory in Rust, std::collections::BTreeSet, and again std::vec::Vec. These add up to about 8000 significant lines of code, roughly 15% of lighttpd.
• vlc: Why not, I ain’t scared! This won’t be a complete list, but a skim finds a command line parser, some chunks of libc that are apparently commonly missing on some platforms (mostly Unicode stuff), what looks like a threadpool, a block memory allocator, a thread-safe FIFO, a subsystem for recognizing file magic numbers, a HTTP cookie library, an MD5 implementation, a small MIME type guesser, a bunch of time parsing stuff, a thread-safe wrapper around the POSIX drand48(3) function, a pile of PGP key stuff, an XML parser, a bunch of string functions, a base64 decoder… You get the idea.

So, yeah. An argument I’ve heard against the Go/Rust paradigm of statically linking everything is “you end up with bunches of copies of the same library code compiled into each different program!” All I can say to that is… lol. That said, with these vendored utility libraries there’s a strong incentive to keep them small, simple and task-specific, which is good. On the flip side, I wonder how many times vlc’s XML parser has been fuzzed?

Yes, embedding dependency version information in the compiled binaries to improve auditability is still a work in progress, and a statically linked model will result in more bandwidth being used to push updates, but it's questionable whether the absolute bandwidth usage will rise, given that stronger compile-time checking is applied to memory safety, the slice of the pie that accounts for 65-90% of security vulnerabilities.

It's perfectly possible to build a security update model around a combination of:

• Setting a common CARGO_TARGET_DIR so all the distro's Rust packages share a common cache for intermediate build artifacts.
• Running something like cargo-audit on each package's Cargo.lock file to automate checking for Rust dependencies that currently depend on a crate that's had a CVE issued against it.

You're just applying the dependency management and auditing before rather than after the decision of what to break up into which distro packages.

...and, again, there already exist C++ libraries that have to be handled this way because they either make heavy use of templates or are header-only libraries that can't be split out into a dynamic library by design.

[ssokolow]

Poke, poke. Unapproved post.