Announcement

Collapse
No announcement yet.

Uutils 0.0.23 Implements More GNU Coreutils Functionality In Rust

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • #11
    Originally posted by erniv2 View Post
    So it passes 65% of the tests, can we start talking if we have about 90% pass rate?
    A lot of the failures are for very pedantic things, which don't actually break stuff in real-world scenarios. For example I doubt that cat failing with error 40 instead of error 62 when encountering a symlink loop (fixed in this release) is something that any script actually relied on. Uutils was already compatible enough over two years ago to boot debian and build its most popular packages. Don't be dismayed by this 65% pass rate, it's actually close to 100% for everyday use.

    Comment


    • #12
      Originally posted by moltonel View Post

      This doesn't make much sense, busybox and coreutils have very different goals and compromises, the same project couldn't replace both at the same time.

      There was a busybox rust fork project, but it's currently abandoned (probably because it was transpiled from C, making it initially of dubious value and a PITA to cleanup). If you want a rust shell (but not a minimalist one) you could try nu (lots of builtins, like busybox, and has recently started integrating uutils code, but feels more like MS powershell than a POSIX shell) or fish (a more traditional but ergonomic shell, which is about halfway in its C++ to Rust migration).
      Im not sure why it couldn't Is there any particular reason why not?

      Comment


      • #13
        busybox is for very small systems with a static lightweight libc implementation. Rust is on the other hand fully fledged.

        Comment


        • #14
          Originally posted by ClosedSource View Post
          busybox is for very small systems with a static lightweight libc implementation. Rust is on the other hand fully fledged.
          i've already shown that as it is, uutils can adequately fit within very small size constraints thanks to the help of upx, so Im not sure Im understanding the point here. ofc im sure uutils will expand over time, but it's not like rust exactly does a good job of size optimization when building gnu libc statically. musl helps (but there are bugs surround it atm, I seem to hit this a lot https://github.com/rust-lang/libc/pull/3213)

          Comment


          • #15
            Originally posted by V1tol View Post

            I wonder if something like eyra could help there. I am still wondering why Rust stdlib decided to use libc instead of implementing everything they need using syscalls (like Go does).
            Because they need to do it that way for platforms like the BSDs where libc is the syscall API stability boundary and, if you're going to do that anyway, then the return-on-investment to special-casing Linux, the one major platform with stable syscall numbers, is much lower.

            (Heck, on OpenBSD, last I checked, Go was relying on a binary header flag that's analogous to "opt me out of NX-bit/ED-bit/Hardware DEP", but instead turns off "kill any process that attempts to make a syscall without going through libc because only malicious code would jump straight to unstable internal APIs".)

            Go 1.12 had to walk back the decision to bypass libSystem.dylib on Darwin because macOS updates kept breaking Go binaries, and, IIRC, they never bypassed ntdll.dll on Windows for the same reasons.

            Outside of Linux, platforms develop the kernel and libc in the same repo as two halves of the same codebase which just happens to straddle the kernelspace/userspace boundary, and system calls are an enum referenced by both halves where they don't care about keeping the numeric representations stable.

            Comment


            • #16
              Originally posted by ClosedSource View Post
              busybox is for very small systems with a static lightweight libc implementation. Rust is on the other hand fully fledged.
              Huh? Rust is for what you use it for. From micro controllers to large servers.
              This particular rust project, however, certainly isn't located close to the micro controller end of the spectrum.

              Comment


              • #17
                Originally posted by Quackdoc View Post
                i've already shown that as it is, uutils can adequately fit within very small size constraints thanks to the help of upx, so Im not sure Im understanding the point here. ofc im sure uutils will expand over time, but it's not like rust exactly does a good job of size optimization when building gnu libc statically. musl helps (but there are bugs surround it atm, I seem to hit this a lot https://github.com/rust-lang/libc/pull/3213)
                I work with systems with flash sizes between 8 to 32 MiB and I have to put a whole Linux distribution there. You really start to count every single KiB. The upx argument is actually a bit odd, because I can use upx with busybox and again it is much smaller. You won't get Rust binaries down to the size of C binaries, because C is basically Assembly in "readable" and Rust brings a bigger runtime to fight all the insecure issues. Also a big issue is that most Rust developers pull in all these online packages and you never know if malicious players are involved. Look at the cpan, pypi and npm stuff, they are flooded by mal- and adware. If you replace some of these tools by a Rust version that pulls in basically thousands of these external libs, you made it worse. From a security perspective this is a nightmare. Don't get me wrong, I'm a C/C++ developer for 30+ years now (working a lot with SAS/C and egcs in the beginning) and I actually prefer Rust, but I also saw a lot of insane shit and that Rust-pull-in-thousands-of-external-libs thing is a very bad idea. I saw Python projects where people pulled in external libs to get code which is 3-5 lines long. And now I see the same nonsense in Rust projects. Seriously, what is this?

                Originally posted by ssokolow View Post
                Outside of Linux, platforms develop the kernel and libc in the same repo as two halves of the same codebase which just happens to straddle the kernelspace/userspace boundary, and system calls are an enum referenced by both halves where they don't care about keeping the numeric representations stable.
                ‚Äč
                The reason for this is that people are reluctant to understand that Linux is the kernel and some kernel tools and nothing more. Everything else are GNU and/or third party tools.

                /Rant over

                Comment


                • #18
                  Originally posted by erniv2 View Post
                  So it passes 65% of the tests, can we start talking if we have about 90% pass rate?
                  But I like to see these progress blurbs and the comments.

                  Comment


                  • #19
                    Originally posted by Akiko View Post
                    I work with systems with flash sizes between 8 to 32 MiB and I have to put a whole Linux distribution there. You really start to count every single KiB. The upx argument is actually a bit odd, because I can use upx with busybox and again it is much smaller.
                    Why is it odd? I don't need it to absolutely demolish busybox in every regard, I need it to fit my needs.
                    You won't get Rust binaries down to the size of C binaries, because C is basically Assembly in "readable" and Rust brings a bigger runtime to fight all the insecure issues.
                    I've never understood this, C really isn't that special in this regard, with rust when you use no_std and no_main it can be almost as small as C code in my experience when you use libc. for instance when doing panic abort, building
                    Code:
                    RUSTFLAGS='-C target-feature=+crt-static -C opt-level=z -C panic=abort -C strip=symbols' cargo build --target=x86_64-unknown-linux-musl --release
                    results in the hello world binary being 477k, a far cry away from 1Mb, and that's before doing no_std shenanigans. (yes using gnu hurts here a lot). after doing the no_std stuff, the binary is 20k or so. not as small as musl granted, but small enough that I don't care.

                    Also a big issue is that most Rust developers pull in all these online packages and you never know if malicious players are involved. Look at the cpan, pypi and npm stuff, they are flooded by mal- and adware. If you replace some of these tools by a Rust version that pulls in basically thousands of these external libs, you made it worse. From a security perspective this is a nightmare. Don't get me wrong, I'm a C/C++ developer for 30+ years now (working a lot with SAS/C and egcs in the beginning) and I actually prefer Rust, but I also saw a lot of insane shit and that Rust-pull-in-thousands-of-external-libs thing is a very bad idea. I saw Python projects where people pulled in external libs to get code which is 3-5 lines long. And now I see the same nonsense in Rust projects. Seriously, what is this?
                    What issue is this? C/C++ code bases have the same issues just hidden by a package manager. Rust isn't special in this regard, Keep in mind that in rust, a single library can have multiple crates. not only that, rust makes it insanely easy to audit the code you need. Some of the worst cases i've actually worked with only wind up having a cargo tree.

                    Lets do some really quick napkin math with uutils for instance, cargo tree --prefix none | wc -l shows 625 "deps" However running cargo tree --prefix none | awk '{print $1}' | sort | uniq -u | wc -l brings it right down to 177. adding sed '/uu/d' brings it down to 106, adding awk -F- and -F_ to remove obvious subcrates of a single library brings it down to 71 (-1 for the actual coreutils entry) likely unique crates.

                    of those 71 crates, it makes it fairly easy to "audit" them, I certainly don't need to audit things like rust crate or rand or time etc. I really don't see this issue where rust devs are using crates with one or two lines, im sure some exist, don't get me wrong, it's just not an issue I see for apps I look into.

                    Comment


                    • #20
                      Originally posted by oleid View Post

                      Huh? Rust is for what you use it for. From micro controllers to large servers.
                      This particular rust project, however, certainly isn't located close to the micro controller end of the spectrum.
                      Rust requires LLVM meaning C++ meaning an architecture that has a C++ compiler. Your car keys probably have a C compiler.
                      You're absolutely right in that this is irrelevant because you can cross compile but the reason why minimal implementations of libc exist is because of what I just explained.
                      Last edited by ClosedSource; 17 November 2023, 04:10 AM. Reason: grammar

                      Comment

                      Working...
                      X