Announcement

Collapse
No announcement yet.

Rust Developers Move Ahead With Preparing To Upstream More Code Into The Linux Kernel

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • cj.wijtmans
    replied
    Originally posted by jacob View Post

    Well maybe you happen to be lucky. But that is an exception, not a rule. Postgresql won't build on freebsd with clang-14, Mozilla has been complaining how trying to accommodate 3 different C++ compilers for their official Firefox builds (VC++ for Windows, G++ for Linux, Clang++ for MacOS) generates substantial additional work, Chromium prefers to ship its own clang, Linux won't work with anything but gcc (and now clang too, but that was a MAJOR undertaking), a number of packages written in C or C++ in the BSD ports tree still have a hard requirement on GCC, etc.

    The reality is that trying to write code that is compiler independent takes extra effort which is hardly justified. It was another story when there was a much larger number of relevant platforms than today and each had its own proprietary C compiler (or compilers). But for a new programming language that comes with a compiler which is free, that anyone can have, use, port, adapt, fork etc. at will, which already runs on the most important OSes and architectures (although yes, arch support is still a weakness of Rust compared to C), I really don't see the point of demanding multiple subtly different implementations just because reasons. What practical problem would that solve? I don't see any, but I see a number of problems it would introduce.
    The only thing I can think of in terms of "accommodating" is compiler flags. Either that or they are using a lot of C++ extensions that are non-standard. Most likely in order to try and optimize their code. But then you might as well complain that when you use assembly to optimize that you have to accommodate different architectures. Mean while they most likely refuse to use new C++ standards that standardized some perfomance related things like hinting. It's a non-argument because they created the issue themselves in the first place. For example, Attribute specifier sequence(since C++11) - cppreference.com​.​ "Attributes provide the unified standard syntax for implementation-defined language extensions, such as the GNU and IBM language extensions __attribute__((...)), Microsoft extension __declspec(), etc."​

    Originally posted by jacob View Post
    Let's also not forget another factor: writing pure "standards compliant" code was much less of a problem back in the 80s or even the 90s. But user expectations have moved on since then and software complexity has increased by orders of magnitude - and for good, justified reasons, so KISS is not the answer. By the same token some of the then-standards have became basically immaterial (like POSIX). You can still write "hello world" in pure standards compliant C, but if you are developing say LibreOffice, choosing your compiler for the project is as important as choosing the UI framework it will use. After all, if someone wants to rebuild it himself, telling him "you must use g++" is a perfectly valid choice.
    This has to be a joke.
    Last edited by cj.wijtmans; 22 November 2022, 07:01 PM.

    Leave a comment:


  • bug77
    replied
    Originally posted by moltonel View Post

    I disagree here for various reasons: First, some drivers are necessary for booting even a "blind and mute" kernel : hardware buses and sometimes even cpu/memory need to be initialized for the kernel to properly use them. Second, the whole point of a kernel is to provide access to hardware: a kernel without drivers is strictly useless. Thrid, drivers are not the only components that are optional and runtime-loadable: network schedulers, filesystems, virtualization, you name it. Last, 2/3rds of the mainline kernel tree is drivers: go tell driver developers that their work isn't "in the kernel" and see what happens.



    Optional yes (though that will hopefully change someday, when compiler(s) support all the required platforms, and the language has proved itself in kernel developer's eyes). Modules not necessarily, as rust code doesn't have to be compilable as a module (AFAICT, none of the rust code merged in 6.1 can), and almost any code that can be compiled as modules can also be compiled built-in.
    A kernel will also not boot without hardware to run on and power for that hardware. But we're not saying hardware and electricity are in the kernel, are we?
    Anyway, I think we're in agreement, we're just splitting hairs now.

    Leave a comment:


  • moltonel
    replied
    Originally posted by bug77 View Post
    I guess the misunderstanding is simply about what "in the kernel" means. To me, the drivers are not "in the kernel" because the kernel can exist (as in be compiled and boot up fine) without them. It makes them add-ons in my head.
    I disagree here for various reasons: First, some drivers are necessary for booting even a "blind and mute" kernel : hardware buses and sometimes even cpu/memory need to be initialized for the kernel to properly use them. Second, the whole point of a kernel is to provide access to hardware: a kernel without drivers is strictly useless. Thrid, drivers are not the only components that are optional and runtime-loadable: network schedulers, filesystems, virtualization, you name it. Last, 2/3rds of the mainline kernel tree is drivers: go tell driver developers that their work isn't "in the kernel" and see what happens.

    I guess it would more accurate to say that Rust is in optional kernel modules maybe?
    Optional yes (though that will hopefully change someday, when compiler(s) support all the required platforms, and the language has proved itself in kernel developer's eyes). Modules not necessarily, as rust code doesn't have to be compilable as a module (AFAICT, none of the rust code merged in 6.1 can), and almost any code that can be compiled as modules can also be compiled built-in.

    Leave a comment:


  • mdedetrich
    replied
    Originally posted by krzyzowiec View Post

    It's not like trying to add two objects is a serious problem in JavaScript. Either you know about it and know what you should actually do, or you don't know the language at all.
    That has to be the stupidest and dumbest argument, especially considering the creator of Javascript said it was mistake from a language they rushed out in a week due to time pressure.

    This kind of handwaving away retarded design decisions needs to stop, obviously this should never be done but considering how much Javascript is used, this mistake and others has likely wasted collective billions of hours because since Javascript is a dynamically typed language you may not even know that a reference is an object (or not) at runtime and rather than telling you immediately it can just silently run on giving you an even more cryptic and dumb error message later.

    And this isn't even the only case, there are further ones that are even more cryptic and subtle to catch.
    Last edited by mdedetrich; 17 November 2022, 07:25 PM.

    Leave a comment:


  • krzyzowiec
    replied
    Originally posted by mdedetrich View Post

    I think you are confusing dynamic vs static typing with strong vs weak typing. Ruby, similar to Python, is strongly typed which means that if you do something like trying add 2 objects it will fail immediately. Now because its dynamically typed it happens at runtime but unlike Javascript, its not going to silently coerce the object to something else and actually try to add 2 nonsensical things.

    wrt to nil (not null, Ruby doesn't have null) although there are problems with it, they still at least have syntax helpers to help deal with the problem, i.e. https://stackoverflow.com/questions/...uby-on-rails-2
    It's not like trying to add two objects is a serious problem in JavaScript. Either you know about it and know what you should actually do, or you don't know the language at all.

    Leave a comment:


  • NobodyXu
    replied
    Originally posted by bug77 View Post
    I guess the misunderstanding is simply about what "in the kernel" means. To me, the drivers are not "in the kernel" because the kernel can exist (as in be compiled and boot up fine) without them. It makes them add-ons in my head. I guess it would more accurate to say that Rust is in optional kernel modules maybe?
    Yes, Rust currently is opt-in since there're a few CPU archtecture it doesn't support and still being experimental support, but with rustc_codegen_gcc, it will be there.

    But do note that M1's GPU driver is written in Rust, so if you want to build Asahi Linux kernel with GPU support, you might have to turn on the Rust support.

    Originally posted by bug77 View Post
    I have my fingers crossed for Rust proving itself and taking over more critical parts. With memory safety taken care of, I think developers would have a much easier life. Then again, even if Rust proves itself, it's not like all the C code will automagically rewrite itself. Rust will have to coexist with C for a long, long time, so another guess would be that getting admitted into the kernel will depend a lot on how it integrates with C.
    Yes, it will take a long time to migrate.

    Leave a comment:


  • bug77
    replied
    Originally posted by moltonel View Post

    I'm not sure what your current (mis)understanding of the plans are, so I hope this clears up a few things:
    • Driver code is kernel code, you can't claim "Rust will not be in the kernel at all" just because "it's only drivers". The 6.1 kernel is only getting support code with no driver or anything user-facing, but it's still a few thousand lines of Rust
    • Rust code does of course call into existing C apis. But the wrappers are a bit more sophisticated than a straight unsafe binding: they often offer a more idiomatic Rust API and are safe to call. The goal is that most drivers can be written without any unsafe.
    • While Rust can't be used for core parts of the kernel (yet), it's not limited to drivers. There's already a Binder (Android's message bus) implementation, a 9P (network filesystem) implementation, and talks about LSM, parsers, and many others. C code can call into Rust code.
    • Rust probably wouldn't have been added to Linux if it couldn't (at least in theory) be used in every part of the kernel. Expect a flurry of experimentations by the time 6.2 comes out, as people challenge this theory. Only a fraction of this will get proposed for mainline, but it's important to uncover any fundamental language issue early.
    Ok, thanks, that makes sense.

    I guess the misunderstanding is simply about what "in the kernel" means. To me, the drivers are not "in the kernel" because the kernel can exist (as in be compiled and boot up fine) without them. It makes them add-ons in my head. I guess it would more accurate to say that Rust is in optional kernel modules maybe?

    I have my fingers crossed for Rust proving itself and taking over more critical parts. With memory safety taken care of, I think developers would have a much easier life. Then again, even if Rust proves itself, it's not like all the C code will automagically rewrite itself. Rust will have to coexist with C for a long, long time, so another guess would be that getting admitted into the kernel will depend a lot on how it integrates with C.

    Leave a comment:


  • moltonel
    replied
    Originally posted by bug77 View Post
    Can you elaborate a little, please? I was under the impression Rust drivers would use the existing APIs (if wrapped inside unsafe blocks).
    I'm not sure what your current (mis)understanding of the plans are, so I hope this clears up a few things:
    • Driver code is kernel code, you can't claim "Rust will not be in the kernel at all" just because "it's only drivers". The 6.1 kernel is only getting support code with no driver or anything user-facing, but it's still a few thousand lines of Rust
    • Rust code does of course call into existing C apis. But the wrappers are a bit more sophisticated than a straight unsafe binding: they often offer a more idiomatic Rust API and are safe to call. The goal is that most drivers can be written without any unsafe.
    • While Rust can't be used for core parts of the kernel (yet), it's not limited to drivers. There's already a Binder (Android's message bus) implementation, a 9P (network filesystem) implementation, and talks about LSM, parsers, and many others. C code can call into Rust code.
    • Rust probably wouldn't have been added to Linux if it couldn't (at least in theory) be used in every part of the kernel. Expect a flurry of experimentations by the time 6.2 comes out, as people challenge this theory. Only a fraction of this will get proposed for mainline, but it's important to uncover any fundamental language issue early.

    Leave a comment:


  • bug77
    replied
    Originally posted by moltonel View Post

    Huh ? Rust in 6.1 is very limited, but the 6.2 PR has interfaces to network and filesystem in addition to pure driver stuff. The longterm goal is to be able to use Rust almost anywhere in the kernel, even if it remains out of mainline for various reasons. Also, Rust is already a popular language to write eBPF programs in.
    Can you elaborate a little, please? I was under the impression Rust drivers would use the existing APIs (if wrapped inside unsafe blocks).

    Leave a comment:


  • moltonel
    replied
    Originally posted by baryluk View Post
    Which architectures will be supported? If we see for example some PCIe device drivers or file systems in Rust in the future, I hope some niche architectures are not left without support.
    Essentially these, plus any that get added/promoted in future rustc version and minus any that rustforlinux might initially blacklist. The importance of llvm-unsupported platforms is often overblown (you're very unlikely to come across one unless you're a specialized embedded dev), but they do prevent Rust from being used in a mandatory core part of the kernel for now.

    I know there is gcc rust compiler in pretty good shape and will be part of GCC 13 officially (so probably May 2023), which does support quite a lot of platforms
    Gcc-rs is making good progress, but it is very far away from being usable for everyday rust code or kernel code. Don't expect more than a technology preview in gcc13. And remember that each platform requires some work in the frontend too, gcc will not magically support all the same platforms for Rust as for C/C++.

    On the other hand, rustc's gcc backend (rustc_codegen_gcc aka cg_gcc) can already compile most rust code, including kernel code. It's getting close to being a simple to install/use backend for rustc. It's a much more pragmatic answer to the platform support question, with much less controversy.

    not sure what Kernel will use by default.
    If you're targeting a niche arch, you don't really care about the default way to build Linux. The most common way will remain to use Gcc or Clang (much more common than most people think, even before Rust came into the picture) for the C code, and rustc (with its llvm backend) for the Rust code.

    I had various issues with standard rustc compiler, not playing nicely with some tools.

    That describes every language I've ever used, especially C and C++.

    Leave a comment:

Working...
X