Experimental Rust-Based Coreutils Working Well Enough To Run Debian Basics

[brad0]

I would rather see Rust be used for things such as decoders, parsers, interpreters. Software that deal with reading that data from untrusted sources.
I would like to see Rust in projects like FFmpeg, GStreamer, Poppler, VLC, OpenSSH, OpenSSL, etc.

OpenSSH will *NEVER* happen. Most of the rest are very unlikely as well.

I sure as hell don't. I need portable code, not un-portable code. This is like the new Java, just not quite as un-portable.

[kpedersen]

or you're a dumbass isn't going to have source compatibility issues.

Absolutely. Imagine if you had a dependency that casted pointers into integers or something weird like that. You would be stuck on 32-bit until they fixed their mess. Avoid dependencies and you can avoid this kind of scenario. Endianness and misuse of time variables is also a classic one that developers get wrong in source. Avoid complex libraries to achieve simple tasks and you can avoid their bugs. I.e I want to move a file. Why would I drag in an entire filesystem API library just to do that. Including file stats, diffing, syncing. It is all overkill for the task and it potentially breaks.

I point out 32-bit and ARM because you can predict that the kinds of changes required to upgrade to 64-bit is very likely to occur again as computers evolve. You can learn a lot from history. Don't think that developers living today have nailed all types of portability problems!

Trolling? Hell no. There is a massive community of reducing dependencies. There are even projects creating "single header file" i.e encoders.

Don't resort to "he disagrees with me so he must be trolling". That's a little childish

[phoenk]

Assuming I was going to replace a small dependency with something massive like Java did make me chuckle

Small dependencies don't get replaced with bigger ones. They get replaced with bespoke code tailored to solving the problem. They get removed, not replaced.

Consider for example in C++ rather than dragging in some string splitting library, I would simply use string::substr and split it based on some char, adding it into a std::vector. Probably about ~8 lines of code.

Sure, it would possibly be less "complete" than some library dedicated to splitting strings. It wouldn't generate some json output. It also wouldn't generate an image of the result. It doesn't need to. It would however be many less lines of code, so easy to audit and write unit tests for.

Dependency removed.

This is a silly take. Most dependencies do something more complicated than "string splitting". We use dependencies because it is much easier than essentially rewriting the dependency, even if we only need a subset of the functionality. Avoiding dependencies strictly for the sake of avoiding dependencies is going to massively increase development and maintenance cost. As a python dev, I frequently use libraries such as flask, sqlalchemy, and pyyaml. Should I write my own web framework, ORM, and yaml parser when perfectly good ones already exist that meet my needs? It would be a LOT more than 8 lines.

One of the most prominent shifts in development in the last 20 years has been towards better dependency management. Tools like maven, pip, cargo, npm, etc are all widely popular because they integrate well with their languages and make it easy to share code with other people. This is beneficial for everyone. as other people can easily reuse your code, and in turn you might get bug fixes and improvements back. Arguably, reducing this friction has been a key driver in why open source has taken off, eliminating the legal barriers to code reuse while package managers have eliminated the technical barriers. To be against dependencies for the sake of being against dependencies is to ignore the past 20 years of technical progress, and arguably, the very community which this website covers.

[Delgarde]

I don't see much advantages of porting small utilities like coreutils to Rust because these utilities are many small files and with Rust the file size will grow. Also these utilities probably are not security sensitive.

While you're not wrong, do note the part in the article that said "Ledru wrote more than 100 patches to get the code along this far and used it as an exercise to get more involved with Rust." This isn't about being useful, it's mostly about porting simple and familiar utilities as a learning exercise...

[kpedersen]

As a python dev, I frequently use libraries such as flask, sqlalchemy, and pyyaml. Should I write my own web framework, ORM, and yaml parser when perfectly good ones already exist that meet my needs? It would be a LOT more than 8 lines..

Not necessarily. Likewise I would use SDL because it would not be a good use of time to write windowing systems for Win32, X11, Wayland, Android, etc...
However it is useful to be responsible over it. I can scan through a number of projects (especially python) and can see a number of less than useful dependencies. Not all python projects are bad. i.e the Waitress production wsgi server using nothing but the standard python library shows that it can be done!

A classic example I see is dragging in the whole of boost filesystem just to get the size of a file. Boost filesystem drags in ASIO, and a number of others. Disappointingly crow does this (https://crowcpp.org/) which considerably reduces its usefulness as a "microframework" IMO.

Have a look through some NPM or PIP dependency files on a couple of projects and you will see developers drag in large (or many small) dependencies for the most trivial of things.

[Michael_S]

This is a silly take. Most dependencies do something more complicated than "string splitting". We use dependencies because it is much easier than essentially rewriting the dependency, even if we only need a subset of the functionality. Avoiding dependencies strictly for the sake of avoiding dependencies is going to massively increase development and maintenance cost. As a python dev, I frequently use libraries such as flask, sqlalchemy, and pyyaml. Should I write my own web framework, ORM, and yaml parser when perfectly good ones already exist that meet my needs? It would be a LOT more than 8 lines.

One of the most prominent shifts in development in the last 20 years has been towards better dependency management. Tools like maven, pip, cargo, npm, etc are all widely popular because they integrate well with their languages and make it easy to share code with other people. This is beneficial for everyone. as other people can easily reuse your code, and in turn you might get bug fixes and improvements back. Arguably, reducing this friction has been a key driver in why open source has taken off, eliminating the legal barriers to code reuse while package managers have eliminated the technical barriers. To be against dependencies for the sake of being against dependencies is to ignore the past 20 years of technical progress, and arguably, the very community which this website covers.

To side with kpedersen, I think you have to distinguish between useful dependencies and ones that introduce more maintenance overhead and code bloat than value.

NPM for Node is the classic example of this. I have a Node app at work that serves a REST API backed by a Postgres database with a few unit tests. 2500 lines of our code, including the tests. It doesn't even have a security layer, we have an entirely separate application layered over the Node app that handles authentication and authorization. That 2500 line project has 260 dependencies totalling 1.2 million lines of code.

That's not "better dependency management", that's a disaster. Every time any one of those dependencies is revved for a security fix, we have to audit our project to determine if the dependency is exposed and when necessary upgrade and repackage the entire application. I spend more time checking NPM package vulnerability disclosures than I do writing the app.

Java with Gradle or Maven is almost as bad. We have a series of web applications that are ~80MB when packaged with about 12MB of our code and the rest is over 50 dependencies. Some dependencies are critical, like Apache POI for handling Excel file formats or Spring Security for our authentication and authorization, or the libraries we're using for YAML or XML. But for dozens of dependencies we're pulling in 250k-5MB libraries so we can use < 2k lines of code. It's almost as bad as NPM, we spend a staggering amount of time auditing for security vulnerabilities and determining the risk.

On the bright side, JavaScript (ECMAScript) and Java have both received a lot of useful features over the past few years. If you start a brand new project, you often don't need as many dependencies as you did in 2013 because JavaScript has class/let/const/async/await (and IE is dead) and Java has lambdas/streams/multiline strings/default interface method implementations. (Edit: but as soon as you pull in some existing library that was written before those features were added, it's likely to pull in other old libraries that implement workarounds for the missing features.)

[oleid]

Consider for example in C++ rather than dragging in some string splitting library, I would simply use string::substr and split it based on some char, adding it into a std::vector. Probably about ~8 lines of code.

Funny thing that you mention string splitting. String handling is - due to the c++ standard library knowing nuthing about encodings - a major pain point. Splitting utf8 strings might break horribly.

[moltonel]

I don't see much advantages of porting small utilities like coreutils to Rust because these utilities are many small files and with Rust the file size will grow.

On my 64bit Linux machine, the Rust coreutils binary takes 9.5Mb (wilth all unix features on) while the GNU coreutils binaries take 6.7Mb. Bigger sure, but not really a deterrent.

Also these utilities probably are not security sensitive.

Huh ? They are used throughout the system and perform critical functionality, a coreutils bug can enable a system compromise. GNU coreutils have a good track record all things considered (https://www.cvedetails.com/product/5...l?vendor_id=72) but I'd feel safer with a Rust version, once it has matured.

I would rather see Rust be used for things such as

This is not an either/or situation, time spent on coreutils isn't taken away from time spent on your favorite topic. And if you want to influence what Rust is used for, go make a contribution.

decoders, parsers, interpreters. Software that deal with reading that data from untrusted sources. I would like to see Rust in projects like FFmpeg, GStreamer, Poppler, VLC, OpenSSH, OpenSSL, etc.

Rejoice: there are more and more Rust codecs used in ffmpeg and gstreamer (and a slew of other projects), rustls is a great alternative to openssl, there are ssh and pdf crates out there... Rust is gaining ground in a lot of places.

[moltonel]

OpenSSH will *NEVER* happen. Most of the rest are very unlikely as well.

I sure as hell don't. I need portable code, not un-portable code. This is like the new Java, just not quite as un-portable.

OpenSSH certainly won't happen due to political reasons, not technical ones. But that doesn't mean that a Rust SSH client/server won't rise to prominence someday, just like Rustls is an OpenSSL replacement (one of many) that you can expect to see more and more. Ffmpeg and gstreamer (and therefore VLC) already pull in some Rust codecs on my system. Don't know much about poppler, but I wouldn't be surprised to see it follow librsvg's path someday (similar project and community).

My rust code is much much more likely to compile and run flawlessly on a different platform at the first try than any of the C or C++ code I've ever written. In that sense, Rust is more portable than C/C++. I sympathize with platforms that haven't managed to bring in the LLVM ecosystem, but supporting these niche/dying platforms is not worth the trouble of going back from Rust to C/C++.

[increasechief]

I think that these utilities projects are very interesting, I'm not clear on the connection to Debian, I can use cargo to install hundreds of utilities which work natively on any Debian architecture I may want to use.