Announcement

Collapse
No announcement yet.

New Linux Kernel Patches Begin Plumbing Rust Support Into Bcachefs Driver

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • oleid
    replied
    Originally posted by marios View Post

    You are contradicting your previous post here. You had literally said the latest.
    The latest available via your stable distribution, not the latest you can download and build yourself.

    Leave a comment:


  • marios
    replied
    Originally posted by oleid View Post

    You must mean something different, I guess, because gcc LTO is mainline for years.
    When I said clang is the way to go, I especially mean for developers, end users not that much. With clang you can easily use multi threaded link time optimization that scales very well with the cores. But most importantly, gcc is a lot more memory hungry when it comes to LTO. So memory may be a limiting factor here.
    I was talking about mainline linux, not mainline gcc. While you can compile LTO linux with llvm, you need out of tree patches to do it with gcc like these https://www.phoronix.com/news/GCC-LTO-Linux-2022. For developers, I cannot get what you are saying. I am sure the way to go is, "make sure it runs, with all supported compilers and all potential flag combinations". So you will have to compile it with both llvm and gcc in any case.

    Originally posted by oleid View Post

    Well, yes, if you live on the bleeding edge, the latest llvm version might not get used for rustc right away - they stabilize and test first to prevent regressions. Nevertheless, it is quite easy to pick a combo of clang+rustc that allows cross-language LTO. That's what I wanted to say.

    You are contradicting your previous post here. You had literally said the latest.

    Originally posted by oleid View Post
    I understand your point, but I was wondering if it is really a problem in praxis.
    For some people, me included, having older toolchains just for that reason is really a problem. A small problem, but a problem.

    Leave a comment:


  • oleid
    replied
    Originally posted by marios View Post

    The reasoning in the parenthesis is irrelevant (unless the performance of the output binary is identical among alternatives), LTO sacrifices lots of build performance (which happens once) in order to gain runtime performance (which is expected to be reaped among lots of runs). So always choose the compiler that produces the (even marginally) fastest binary. That being said, currently for Linux, clang/llvm LTO is indeed the way to go, because for some annoyingly stupid reason gcc LTO has not been mainlined.
    You must mean something different, I guess, because gcc LTO is mainline for years.
    When I said clang is the way to go, I especially mean for developers, end users not that much. With clang you can easily use multi threaded link time optimization that scales very well with the cores. But most importantly, gcc is a lot more memory hungry when it comes to LTO. So memory may be a limiting factor here.

    Not true for rustc. A specific version of clang uses the same version of llvm. Actually, the llvm repository includes llvm, clang, lld and other stuff. Rustc however is out of tree and often does not support the latest llvm. It has happened more than once on my Gentoo pc. Things would have been different if rustc was in the llvm tree, but it isn't.
    Well, yes, if you live on the bleeding edge, the latest llvm version might not get used for rustc right away - they stabilize and test first to prevent regressions. Nevertheless, it is quite easy to pick a combo of clang+rustc that allows cross-language LTO. That's what I wanted to say.

    I know that about firefox. Actually my firefox is compiled that way. I never doubted the possibility of doing it in the kernel. What I said is that it restricts the toolchain choice, even excluding versions of the llvm toolchain. See my above point.
    I understand your point, but I was wondering if it is really a problem in praxis.

    Leave a comment:


  • marios
    replied
    Originally posted by oleid View Post
    If you're interested in Link time optimisation, clang is your best choice (due to build performance and memory requirements).
    The reasoning in the parenthesis is irrelevant (unless the performance of the output binary is identical among alternatives), LTO sacrifices lots of build performance (which happens once) in order to gain runtime performance (which is expected to be reaped among lots of runs). So always choose the compiler that produces the (even marginally) fastest binary. That being said, currently for Linux, clang/llvm LTO is indeed the way to go, because for some annoyingly stupid reason gcc LTO has not been mainlined.

    Originally posted by oleid View Post
    If you use the latest clang and latest rustc your distribution provides, they will most likely use the same llvm version.
    Not true for rustc. A specific version of clang uses the same version of llvm. Actually, the llvm repository includes llvm, clang, lld and other stuff. Rustc however is out of tree and often does not support the latest llvm. It has happened more than once on my Gentoo pc. Things would have been different if rustc was in the llvm tree, but it isn't.

    Originally posted by oleid View Post
    You'll be happy to hear that Mozilla uses clang+rustc with cross language lto for Firefox. The same should be possible for the kernel. I didn't try it, yet.
    I know that about firefox. Actually my firefox is compiled that way. I never doubted the possibility of doing it in the kernel. What I said is that it restricts the toolchain choice, even excluding versions of the llvm toolchain. See my above point.

    Leave a comment:


  • Weasel
    replied
    Originally posted by oleid View Post
    By that logic you'd be inconsequential to users as well by requiring a C compiler if people don't code in C, since you could simply use bare assembler
    That's correct for a project that started off in assembly...?

    Linux was always C so it required it since day 1.

    And btw, building C can be quite a nightmare in some cases, not the kernel though because it doesn't have dependencies.

    Leave a comment:


  • oleid
    replied
    Originally posted by marios View Post
    Yes, but rust is not in the list. But using 2 compilers for a single binary is usually a big no for me (especially if the binary is the kernel). It forces to choose between two bad choices in this case. Either use 2 different toolchains, an llvm based for Rust and another of my choice for C (does not like the best idea and might break things, i.e. LTO) or use an llvm toolchain compatible with rustc for both C and Rust which is too restrictive (even the latest llvm might be incompatible with rustc, let alone gcc).
    Just for informational purposes:

    If you're interested in Link time optimisation, clang is your best choice (due to build performance and memory requirements). If you use the latest clang and latest rustc your distribution provides, they will most likely use the same llvm version. You'll be happy to hear that Mozilla uses clang+rustc with cross language lto for Firefox. The same should be possible for the kernel. I didn't try it, yet.

    If you still prefer an gcc based Toolchain, you'll be happy to hear that rustc with gcc backend (via libgccjit) is not far away - Q3 2024 would be my guesstimate. I'm positive it will be available before rust is required for bcachefs. It will also allow lto and afaik also cross language lto.
    Last edited by oleid; 19 February 2024, 07:46 PM.

    Leave a comment:


  • oleid
    replied
    Originally posted by kpedersen View Post

    Wayland was stillborn. It was barely alive before 2015 in the first place
    How can something be stillborn and then barely alive afterwards? That makes no sense.

    It is talked about, but everyone is *still* using X11 programs almost 10 years later...
    Who is everybody? By the end of last year I installed an X-Free Gentoo system to my old laptop with graphical login manager and Wayland compositor. Even without Xwayland. No libX11-anything. It's a possibility.

    You realize we are having this same discussion almost 10 years after 2015 and Wayland has made almost no progress since.
    Hmmmm, it depends on the definition of progress.



    Leave a comment:


  • oleid
    replied
    Originally posted by Weasel View Post
    About it being inconsequential to users? Look at marios' posts. It does affect users in a negative way, even if they don't code a single line of Rust.
    By that logic you'd be inconsequential to users as well by requiring a C compiler if people don't code in C, since you could simply use bare assembler

    Leave a comment:


  • hwertz
    replied
    Rust isn't that bad. It's essentially an ultra-paranoid version of C, you can really can start out with some C code, make some light modifications to it to add the type information etc. that Rust requires, and if your C code doesn't have any security holes it'll compile. It catches switch statements where yo don't cover all cases, potential buffer overflows, potential race conditions and concurrency issues, memory use after free, etc., at compile time throwing a compile time error rather than having potentially insecure code. Since the bcachefs command line tools are already written in Rust, the in-kernel Rust could of course be using some code from there. But I imagine largely it could use the existing C code (modified to meet Rust requirements), and any potential security or concurrency issues Rust finds fixes applied to the Rust code could actually be applied back as fixes to the C code too.

    It doesn't ultimately make sense to have both C and Rust code for this. But it does make sense to have both while the Rust code is under development, it's nice to have 2 options when something is in development rather than being forced into being a beta tester.

    As for other platforms -- I agree, Rust should be available on them. I don't think Rust is exceptionally hard to port, I think the only part that is particularly CPU-specific is the concurrency primitives for threading (i.e. to be able to ensure safe locking/mutex behavior on multi-threaded applications.) (Edit: Well, yeah it also has to generate code for the CPU. I assume like gcc and clang, you have essentially a template you can fill in the instruction set and some characeristics of the CPU to help the compiler.) I mean, they do (per the docs) they don't appear to have an x86 build, but have x86-64, various quite old through modern ARM, MIPS, SPARC, Motorola 68K, a target for Nvidia GPU, RISCV, and PowerPC; oh and WebAssembly. I guess that leaves out like PA-RISC, Alpha, and S390 but really, if it's running on 68K I'm sure it can be brought up and running on those 3.
    Last edited by hwertz; 10 February 2024, 06:13 PM.

    Leave a comment:


  • smitty3268
    replied
    Originally posted by reba View Post
    thought "*.pyc" were compiled python files, my bad.
    They are compiled byte-code, similar to how Java works. And it is possible to compile all the way down to an executable, although that's not typically done.

    I suspect marios would consider Java a compiler, though.

    Leave a comment:

Working...
X