Rust-Written Coreutils Increases GNU Compatibility, Adds NetBSD Support

[Developer12]

My thoughts exactly. This project has more than 100 third-party build/runtime dependencies, and Rust's various safety and compatibility guarantees simply do not apply to any of them.

They might all be perfect and memory-safe and well-considered and well-authored now and forever more, but I doubt it.

See https://www.phoronix.com/forums/foru...04#post1415704

[kpedersen]

Maybe from all theses security issues related to memory management.

unsafe
{
  Maybe from all theses security issues related to memory management
}

Solved. Now get back to writing those Rust bindings!

[fitzie]

How do you trust any of the contributors to GNU? how do you trust the kernel you're using? Any upstream can have malicious contributions. By having everyone in the entire rust ecosystem pulling from and keeping an eye on the same package for parsing command line options, you have all the eyeballs on one implementation, rather than 20 or so implementations spread across different projects.

When projects "vendor" libraries by copying code directly into their tree it also becomes impossible to keep track of who has made a copy, and of what version. With shared code repositories you know immediately who's using a library which is invaluable when a bug is discovered and you need to get the word out. In all likelihood people will probably get the bugfix automatically when their build system pulls a new version.

You ALSO solve the problem of everyone writing their own shitty parsers and allocators and red-black trees. Everyone automatically ends up using the best implementation available, rather than one that was written by some dude 7 years ago and forgotten about.

The only reason vendoring libraries or writing them yourself is "boring" is because you end up utterly ignorant of their problems.

you have to choose who you trust knowing that nothings infallible. given the numerous malware attacks in repos. I choose to trust projects that establish a trackrecord, and show care for the role their software is. certainly something that runs as root has a higher bar for me then software that doesn't. I just don't see a compelling reason to abandon coreutils/toybox for software that is a fast moving target just for some memory safety. I don't think vendoring in a bunch of libraries is a good solution, I rather then write the code themselves, just like busybox/toybox/coreutils. This is coreutils, which for me should be maintained by people with a particular knowledge of the systems that it runs upon. if you look at toybox development, you'll see they work very closely with android developers which is one of the major user of toybox. That builds confidence that they Rob has earned the trust of a major distributor and works to build a *boring* core userspace utilities for google and others.

[fitzie]

There are 14 competing languages? Ridiculous! We need to develop one universe language that covers everyone's use cases.

verse looks very interesting. although it's a gc language. i think if they are able to pull of the transactional memory there, it will kick off another round of language innovation, just as rusts ownership/macro system is setting the bar for system languages.

[marlock]

quick question (not rethorical, I'm no expert and have no idea what the answer is, even if I might have a superficial understanding of some of them...):

which of the most common CWEs (types of vulnerabilities) does C happily allow devs to trip over and using Rust might outright eliminate or at least help them trip less often (eg: by circunscribing the issue to unsafe / bindings)?

https://cwe.mitre.org/top25/archive/2023/2023_top25_list.html

other quick question: if their project is still WIP, isn't it a productive approach if the rust coreutils devs use all relevant crates now to get most things in place and later make an effort to minimize dependancies?


and finally: can't a rust over C binding be defined once then used by everyone, thus making it a problem that needs to be solved once instead of a problem each codebase (or even each call to anything) in C needs to implement again and again? isn't that like a huge gain? or did I read too much into what Rust and C advocates have been saying around phoronix comments? and can this binding be made into a crate all by itself, so projects that need it can share the exact same implementation and quickly propagate fixes? (again, absolutely not an expert in any of this... high chance of containing misconceptions and improper logical leaps)

[jstoik]

See https://www.phoronix.com/forums/foru...04#post1415704

I completely agree with you that interdependency, on many levels, is unavoidable. Code can neither be compiled nor executed without the help of myriad, indefinitely-complicated components, all of which require an implicit trust because they're well beyond the scope of what any individual human has any control over.

But there are other levels at which control is possible, and for some projects — like sensitive, low-level system components — I would argue the tradeoffs of reinventing the wheel are not only worth it, but requisite. By taking the RAD road, uutils is losing the single biggest advantage Rust has to offer — memory-safety — leaving it in the same nebulous bug boat as its predecessor, without its predecessor's decades of realworld usage, testing, and battle-hardening.

To be fair, uutils' stated goal is cross-platform compatibility, not safe memory handling; I just don't find the former compelling without the latter.

But who knows. After a few decades of steady development, I may well change my mind.

[jstoik]

which of the most common CWEs (types of vulnerabilities) does C happily allow devs to trip over and using Rust might outright eliminate or at least help them trip less often (eg: by circunscribing the issue to unsafe / bindings)?

Any mistake that can be made in C can be made in Rust too; Rust just forces developers to explicitly opt-in to certain kinds of risky behaviors. It also does a lot to encourage good practices like documentation, unit testing, and code consistency.

Those are all hugely beneficial things, but voluntary.

if their project is still WIP, isn't it a productive approach if the rust coreutils devs use all relevant crates now to get most things in place and later make an effort to minimize dependancies?

Sure, if that's the ultimate plan.

can't a rust over C binding be defined once then used by everyone, thus making it a problem that needs to be solved once instead of a problem each codebase (or even each call to anything) in C needs to implement again and again?

Yes and no.

When compiling a Rust project, the most recent version of each dependency that satisfies the version constraints is pulled down. That could be the latest-and-greatest release, or a release from years and years ago, or maybe even multiple, conflicting releases. (Dependencies might share dependencies but constrain them differently.)

Loose constraints are helpful in that they make updates more or less automatic, but they're harmful for audits and build reproducibility, and might introduce new and different bugs as easily as they might fix them.

[Juke1349]

... can't a rust over C binding be defined once then used by everyone,...

Yes it can, and it is even recommended to publish the "raw" bindings as a "-sys" crate.
"-sys" crates should expose the truly raw, fully unsafe, directly 1-to-1 mapped version of the C library (so basically, like a C-style .h header, but in Rust).
Indeed, they are often generated fully automatically by "bindgen", with perhaps minimal manual polish.

The safe, using all the rust bells and whistles, wrapper around this should be a second crate.

This is so that different opinions on how to make the "best" safe wrapper (which probably involve tradeoffs between performance, completeness, complexity) can still all use the same "header".

Example:
C library "foo" would get a rust mapping called "foo-sys".
Then someone designs a nice, safe wrapper and decides to call it "fubar" (depending on foo-sys).
Someone else thinks they can do it better and creates "bar-ista" (also depending on foo-sys).
Then, ideally, experience shows which works better, and the two developers combine their learned lessons into "bazbaz" (or fubar v2.0, or any other option, because we're flexible now!)

[Almindor]

BSD world will have orgasms with this, as they hate GPL.

I'm quite surprised there aren't massive contributions from BSD world!

What about findutils? No mention of them.

https://github.com/uutils/findutils

I suspect it's coz most BSD folk these days are graybeards with 0 rust experience.

[Developer12]

I completely agree with you that interdependency, on many levels, is unavoidable. Code can neither be compiled nor executed without the help of myriad, indefinitely-complicated components, all of which require an implicit trust because they're well beyond the scope of what any individual human has any control over.

But there are other levels at which control is possible, and for some projects — like sensitive, low-level system components — I would argue the tradeoffs of reinventing the wheel are not only worth it, but requisite. By taking the RAD road, uutils is losing the single biggest advantage Rust has to offer — memory-safety — leaving it in the same nebulous bug boat as its predecessor, without its predecessor's decades of realworld usage, testing, and battle-hardening.

To be fair, uutils' stated goal is cross-platform compatibility, not safe memory handling; I just don't find the former compelling without the latter.

But who knows. After a few decades of steady development, I may well change my mind.

It's not a matter of control, it's a matter of upkeep. When everyone writes their own shitty implementation of command line option parsing, you end up with 20 different implementations which are all buggy, inefficient, and broken in their own way. You also don't have any idea where those implementations are or what problems they have even if problems have been found in some of them.

You WANT this dependancy. It's a significantly better world when there's one implementation of each library, used by everyone, maintained by people who know what they're doing and have the time to do it right.