Rust-Written Coreutils Increases GNU Compatibility, Adds NetBSD Support

[rafanelli]

Wanna write an operating system? Use C
Wanna write an android application? Use kotlin
Wanna rewrite other project? Use rust

I only agree with the Kotlin And only if you are not writing "a simple app that needs to work on iOS too" (in that case some other lang/stack might een work better for you.

But only C for kernels in 2023, really? I mean sure it's the safe choice, but in a kernel the library ecosystem does not matter much, and some "safely aspects" may matter a lot. Rust and Zig seem good choices here.

And "other projects" may (1) need to target the browser, which is not Rust's strongest point. Rust also costs a lot to write compared to some GC 'd langs like Kotlin, ReScript, Dart or Go.

[oleid]

I'm only suggesting that they can and occasionally do.

Okay, but that's trivial (in a mathematical sense). Even you own code can, if you use unsafe blocks. That's why you wisely choose your dependencies with the aforementioned tools.

This project also has a number of C dependencies. There's nothing Rust can do to ensure they function soundly.

Unless they represent operating system API, I'm sure they can get replaced as-needed.

[mmstick]

And "other projects" may (1) need to target the browser, which is not Rust's strongest point. Rust also costs a lot to write compared to some GC 'd langs like Kotlin, ReScript, Dart or Go.

Rust's ability to target WASM has always been one of its strengths. Rust was one of the first languages to support WASM as a target. As far as I can tell, it was the first language because it doesn't need to compile a runtime alongside the WASM.

[waxhead]

That's why this coreutils replacement is developed using the test suite of the original project. That helps to get the structural behaviour right.

Yes, that is absolutely true. Test suites are a great thing, but mind you - testsuites do have their limits as well. For example if a weird character never triggered any bugs in original_prog_x then the testsuite may never have checks for weird characters of the third kind in the first place since nobody found it useful. If rewriten_prog_x sometimes gets confused about weird characters and sometimes produce undefined behavior it can be a very hard to catch problem. (just imagine something weird like prog <tab> --help or prog --<tab>help or other "not supposed to happen" things).

[oleid]

Yes, that is absolutely true. Test suites are a great thing, but mind you - testsuites do have their limits as well. For example if a weird character never triggered any bugs in original_prog_x then the testsuite may never have checks for weird characters of the third kind in the first place since nobody found it useful.

True, but in the end we'll have a better test suite that will help both implementations. You never know if the aforementioned error would have happened in the original implementation as well after some refactoring.

If rewriten_prog_x sometimes gets confused about weird characters and sometimes produce undefined behavior it can be a very hard to catch problem.

I don't think this is called undefined behaviour.

(just imagine something weird like prog <tab> --help or prog --<tab>help or other "not supposed to happen" things).

Command line argument parsing is actually well defined. Also, this is nothing you'd implement yourself, there are libraries for this
But I know this is only supposed to be an example.

[darkonix]

The current number and nature of dependencies for this project make me uncomfortable. [Emphasis added for clarity.]
If you feel differently, that's A-OK! There are plenty of forks to go around.

To understand better your point of view, could you give some examples of which dependencies that you feel are problematic? Or is the problem that the number of dependencies exceeds what you feel should be acceptable? I understand some of this will be subjective.

You may be interested in this analysis of dependencies in C & C++ programs which I believe provides some insights on the differences when comparing dependencies with Rust, Python, and other languages that actually manage dependencies: https://wiki.alopex.li/LetsBeRealAboutDependencies

[jstoik]

Could you give some examples of which dependencies that you feel are problematic? Or is the problem that the number of dependencies exceeds what you feel should be acceptable?

My discomfort is solely due to the foundational role that coreutils plays on Linux systems. A bug in "cp" for example isn't just a bug in "cp", but a bug in the unfathomable number of downstream tools and scripts that rely on it. The stakes are about as high as they can be.

In this sort of context, I don't think it is appropriate to trade control for convenience, or to stray from Rust's safety guarantees. (Two different things, but somewhat related.)

Crates like "rayon", for example, are really cool and I use them all the time, but "rayon" has its own priorities. It does wild and magical things to maximize multithreading performance. And that's as it should be. It's what "rayon" is for. That's what makes it such an excellent crate.

But as much as I normally prefer speed, I would much rather something like coreutils prioritize conservativism and safety. If that means a file copy (for example) takes a few seconds longer, so be it. (Not that "cp" does that now, but I would definitely expect a Rust rewrite to do that, since Rust makes it possible.)

Cross-platform compatibility on its own — what uutils is aiming for — isn't enough for me. However objectively ghastly the C language is, fifty years of intense scrutiny sets a very high bar.

You may be interested in this analysis of dependencies in C & C++ programs[…]

You're absolutely right, that nuance has been lost in these discussions. Coreutils has dependencies of its own.

But because of its proximity to the base of the foodchain, its dependencies are pretty much the (suckier) C equivalent of Rust's standard library, stuff like <stdio.h> and <getopt.h>, at least from what I can tell. (I haven't reviewed the entire codebase so there might be exceptions, but the few individual programs I pulled up were pretty minimal compared to their Rust counterparts.)

[darkonix]

My discomfort is solely due to the foundational role that coreutils plays on Linux systems. A bug in "cp" for example isn't just a bug in "cp", but a bug in the unfathomable number of downstream tools and scripts that rely on it. The stakes are about as high as they can be.

In this sort of context, I don't think it is appropriate to trade control for convenience, or to stray from Rust's safety guarantees. (Two different things, but somewhat related.)

Crates like "rayon", for example, are really cool and I use them all the time, but "rayon" has its own priorities. It does wild and magical things to maximize multithreading performance. And that's as it should be. It's what "rayon" is for. That's what makes it such an excellent crate.

But as much as I normally prefer speed, I would much rather something like coreutils prioritize conservativism and safety. If that means a file copy (for example) takes a few seconds longer, so be it. (Not that "cp" does that now, but I would definitely expect a Rust rewrite to do that, since Rust makes it possible.)

Cross-platform compatibility on its own — what uutils is aiming for — isn't enough for me. However objectively ghastly the C language is, fifty years of intense scrutiny sets a very high bar.



You're absolutely right, that nuance has been lost in these discussions. Coreutils has dependencies of its own.

But because of its proximity to the base of the foodchain, its dependencies are pretty much the (suckier) C equivalent of Rust's standard library, stuff like <stdio.h> and <getopt.h>, at least from what I can tell. (I haven't reviewed the entire codebase so there might be exceptions, but the few individual programs I pulled up were pretty minimal compared to their Rust counterparts.)

Thanks, this makes your point of view much more clear. I mostly agree but I think there is space to use those third-parties libraries. Like the rayon example I don't think it is a deal breaker for me but I agree it would be nice to avoid it if possible in this particular project. I'm not sure why it is used, it may be convenience or there may be some more significant need for it. I suppose that as the project matures the developers may consider optimizing dependencies once they have more complete functional completeness.

[anarki2]

BUG: Not GPL... hence not an Gnu coreutils replacement.

Unless you care about freedom and functionality, and not Stallman's wet dreams of owning all code ever written.