Uutils 0.0.24 Advances Rust-Written Coreutils Implementation

[jstoik]

using the make file instead of cargo to build it will build the individual executables

BTW: coreutils allows build everything as single binary and set of symliks as well

And down the rabbit hole we go…

Single Binary

 5,664,496 GNU 9.4
12,675,368 Uutils​ 0.0.24

Many Binaries

 28,992,306 GNU 9.4
317,825,182 Uutils​ 0.0.24

I can see why uutils suggests cargo install before make. Haha.

[Quackdoc]

And down the rabbit hole we go…

Single Binary

5,664,496 GNU 9.4
12,675,368 Uutils​ 0.0.24

Many Binaries

28,992,306 GNU 9.4
317,825,182 Uutils​ 0.0.24

I can see why uutils suggests cargo install before make. Haha.

yeah it's pretty chonky, but just asking, did you strip uutils by chance? There are a few things you can do to trim file sizes, stripping is needed since it's not default (unless they changed that), you can also set rust to abort on panic instead of unwind which saves a good amount of space, and then you have things like LTO. and fianlly if you do *really* care, you can always run upx --lzma --best on the final binary which will compress it quite a bit

[kloczek]

And down the rabbit hole we go…

Single Binary

5,664,496 GNU 9.4
12,675,368 Uutils​ 0.0.24

Many Binaries

28,992,306 GNU 9.4
317,825,182 Uutils​ 0.0.24

I can see why uutils suggests cargo install before make. Haha.

So I was right asking about how many times it is bigger
This is why rust/cargo sucks.
First in normal conditions rust projects have HUGE list of dependencies with which all is linked STATICALLY.
Second reason is that all those dependencies are downloaded during the build (many distributions prod build ifras are INTENTIONALLY cut off from access to public network).

[kloczek]

yeah it's pretty chonky, but just asking, did you strip uutils by chance? There are a few things you can do to trim file sizes, stripping is needed since it's not default (unless they changed that), you can also set rust to abort on panic instead of unwind which saves a good amount of space, and then you have things like LTO. and fianlly if you do *really* care, you can always run upx --lzma --best on the final binary which will compress it quite a bit

To compare real binaries size all what is needed is compare `size` command output.
In case of rust generated binaries it will be ALWAYS really bad metric result.
Other aspect is that all that statical linking of all dependencies eats al lot of cpu power on linking stage.

Using upx does not make to much sense because it compresses only VFS binaries size and it adds even more demand on used RAM.
This is why upx on embedded systems is never used.

[Quackdoc]

This is why upx on embedded systems is never used.

I use UPX on loads of my embedded devices because it crunches the size down tons, I actually found UPX because of my embedded needs not inspite of them

[kloczek]

I use UPX on loads of my embedded devices because it crunches the size down tons, I actually found UPX because of my embedded needs not inspite of them

So please check memory usage on those devices.

[ssokolow]

First in normal conditions rust projects have HUGE list of dependencies with which all is linked STATICALLY.

LetsBeRealAboutDependencies

TL;DR: Those dependencies are typically equivalent to code that's either rewritten from scratch in each C project or copied-and-pasted in or part of one giant dependency that you only wind up using part of. For example, stuff like argument parsing, which, in C, you'd probably either rewrite based around the minimal functionality provided by getopt(3) (getopt(3) being minimal enough that I have approximated it in Rust with a single match statement inside a for loop in a project where resistance to supply chain attacks was paramount) or vendor something like https://www.argtable.org/

Second reason is that all those dependencies are downloaded during the build (many distributions prod build ifras are INTENTIONALLY cut off from access to public network).

They're downloaded during the build by default. Cargo provides multiple options for handling things differently, such as cargo fetch; cargo build --offline or cargo vendor, just to name two built-in ones.

Flatpak uses that separated design (download phase without running code, build phase without network access) and there's a tool named flatpak-cargo-generator for generating Flatpak manifests from Cargo.lock files, similar to how there are such tools for other things like Python PyPI dependency lists.

[kloczek]

LetsBeRealAboutDependencies

TL;DR: Those dependencies are typically equivalent to code that's either rewritten from scratch in each C project or copied-and-pasted in or part of one giant dependency that you only wind up using part of. For example, stuff like argument parsing, which, in C, you'd probably either rewrite based around the minimal functionality provided by getopt(3) (getopt(3) being minimal enough that I have approximated it in Rust with a single match statement inside a for loop in a project where resistance to supply chain attacks was paramount) or vendor something like https://www.argtable.org/

getopt() is in LIBC!!!
Yep lack of standard run-time rust library it is this bit which usually starts that looong cargo chain of dependencies.

They're downloaded during the build by default. Cargo provides multiple options for handling things differently, such as cargo fetch; cargo build --offline or cargo vendor, just to name two built-in ones.

IMO this should ALWAYS optional and not default behaviour.

Flatpak uses that separated design (download phase without running code, build phase without network access) and there's a tool named flatpak-cargo-generator for generating Flatpak manifests from Cargo.lock files, similar to how there are such tools for other things like Python PyPI dependency lists.

New PEP517 build procedure do not require pypi access.

Rust fans seems not learned to much from nodejs past bad cases when remove some projects from public network trashed build+90% of all projects.

[ssokolow]

getopt() is in LIBC!!!

Yes, but it's only one match statement away from being Rust's equivalent to argv, std::env::args_os. People still need stuff like --help rendering and a ton of other things that are considered standard these days. That's where standard practice in C is to either rewrite or vendor, while standard practice in Rust is to add a dependency.

I don't know about you, but I prefer code that has more eyes on it and more people testing it to find bugs. (Plus, it gets tiring to have to be responsible for reading HIGs and getting all the niche features you don't use right.)

Yep lack of standard run-time rust library it is this bit which usually starts that looong cargo chain of dependencies.

Better to have "third-party" crates maintained by the Rust team than suffer through The Day The Standard Library Died (C++) or Batteries Included, But They're Leaking (Python). (C++ and Python were specific influences on the approach the Rust designers took.)

IMO this should ALWAYS optional and not default behaviour.

You really don't get it, do you? You're not representative of the average person.

People say, again and again, that the build tooling is one of the reasons they use Rust instead of something designed for convenience over performance like Python or Ruby or even Go. (Go may be better at easy cross-compilation, but Rust doesn't require you to remember URLs for your package names. That's an actual thing people say about them.)

You're responding to "After a day like that, I don't even want to think any more. Let's eat out." with "All the places around here suck compared to home cooking."

New PEP517 build procedure do not require pypi access.

Irrelevant. The purpose of the Flatpak tool is to parse PyPI version information and spit out a list of URLs and SHA256 hashes for Flatpak to retrieve in its download phase and a list of pip commands to run to install the resulting downloaded package archives.

Rust fans seems not learned to much from nodejs past bad cases when remove some projects from public network trashed build+90% of all projects.

1. Crates.io is designed to be an append-only archive. As long as you've committed your Cargo.lock file to your repo, not even cargo yank will break your build. (cargo yank only prevents a version from being added to new Cargo.lock pins) They only remove things from the repo when compelled by law or when they're literally malware. (So, yes, probably best to not rely on Crates.io for retrieving illegal numbers since you're going to be wanting to keep dozens of mirrors of your code when they start sending DMCA takedowns for your project directly.)
2. The cargo vendor command will automate what you have to fiddle around with doing manually in C or C++, while still preserving the information needed to automate re-vendoring a newer version later.
3. Again, most people care more about not making extra work for themselves in the common case (applying updates for security or bug fixes) than agonizing about the dramatic but very rare one. (i.e. the "You're far more likely to get in an auto accident while driving to the airport than in a plane crash, yet people aren't willing to inconvenience themselves to make driving safer" principle.)

[kloczek]

Better to have "third-party" crates maintained by the Rust team than suffer through The Day The Standard Library Died (C++) or Batteries Included, But They're Leaking (Python). (C++ and Python were specific influences on the approach the Rust designers took.)

I do not understand why in this case rust generates DSOs. All should be linked into kernel. In't it?

Irrelevant. The purpose of the Flatpak tool is to parse PyPI version information and spit out a list of URLs and SHA256 hashes for Flatpak to retrieve in its download phase and a list of pip commands to run to install the resulting downloaded package archives.

Try to have look on Solaris where something like proces separation on VFS layer is done without any user space libraries of flatpacks.

Crates.io is designed to be an append-only archive. As long as you've committed your Cargo.lock file to your repo, not even cargo yank will break your build. (cargo yank only prevents a version from being added to new Cargo.lock pins) They only remove things from the repo when compelled by law or when they're literally malware. (So, yes, probably best to not rely on Crates.io for retrieving illegal numbers since you're going to be wanting to keep dozens of mirrors of your code when they start sending DMCA takedowns for your project directly.)

Again .. why not link everything into kernel?