Rust-Based, Memory-Safe PNG Decoders "Vastly Outperform" C-Based PNG Libraries

[bacteriamanicure]

It depends on who you ask. There is a list of criteria I have before I would use it, but I do not want to make this about me. One thing I believe to be common practise is to up a version number to 1.0, or even 1.1, once code is considered stable. And this is not the case yet.

Rust PNG has a version number of v0.17.15.
Zune PNG has a version number of 0.4.10.
Wuffs PNG has a version number of 0.4.0-alpha9.

The claim of being production-ready seems like an unfortunate choice of words when I look at those version numbers.

Assuming semver that's a really good point. Even if it's all "done" actually guaranteeing stability is a pretty important step

[ssokolow]

Lets talk again in a few years when someone has to maintain this ...

image.png

That's what you find difficult when C has things like the Clockwise/Spiral Rule?

It's a trivial function that just layers on a few basic syntaxes C has no equivalent for:

• It uses async and await to write a fragment of a state machine struct using function syntax.
• It has the lifetime annotations needed to allow zero-copy deserialization incorporating the input string without risking dangling pointers or leaking the implementation of the function into the API you want to keep stable. (the 'a things, which show up inside the generic parameter lists because, on an abstract level, lifetimes are type-system generics too.)
• It will do early-return, similar to throwing an exception but as part of the return type signature, if reqwest::get() or response.json() report failure. (the ? operator. Also, you can tell by the :: vs . member-access operators that reqwest::get is an associated function (i.e. class method) while response.json will receive self as its first argument.)
• It returns a tagged union of either the requested value or an error (the Result<T, E>, which is a de facto standard provided by the standard library).
• If it returns an error, it will be an exclusive RAII pointer to the heap which (Box) which does dynamic/vtable dispatch (dyn) to some unspecified concrete type which implements the Error interface. (Another de facto standard provided by the standard library.)
• The type that will be returned on success is generic and may be any type T which implements the Deserialize interface. (Part of the Serde library for serializing and deserializing, which is a de facto standard that's outside the standard library.)
• response.json() is defined with an overloaded return type and the variant which takes type T will be called (The ::<>, which is known as the turbofish operator and agreed by Rust's developers to be its ugliest construct... but a necessary evil for using familiar-to-C++-developers <> for generic type parameters instead of something like [].)

In Plain English, the first four lines are "Define an async function fetch, which is generic over lifetime a and type T. It takes one argument, url, which is a reference (non-nullable pointer) to a string or piece of a string (str) which will remain valid for lifetime a. It returns a Result where the Ok variant is of type T and the Err variant is an RAII smart pointer to something which implements the Error interface. Only types which can implement the Deserialize interface when any references are bounded to lifetime a are valid choices for type T."

Once you know the syntax, I think the most confusing part for a newcomer of the whole thing is where the heck T's concrete value actually comes from so it knows which variant of response.json to call and the answer is that it's set by Rust's type inference working backwards from what you use fetch's return value for at the call site. Basically, you're looking at that ::<> on response.json from the other side... though the function calling fetch probably won't need ::<> unless it too is just passing T back up to a parent without constraining it to be able to do things with it.

(Seriously. This is an API which takes a URL and returns a struct deserialized from the resultant JSON with which struct to deserialize to chosen by the type inference information at the call site and, if there isn't enough inference information to unambiguously pick one, you'll have to either use ::<> to specify it explicitly or specify a type on the let for the variable you're assigning it to.)

The most confusing part for a more experienced programmer is probably why the person who wrote it wanted to bind the lifetime of the returned struct to 'a when I don't see any path for a reference to url to get injected into the returned struct T. (But, thanks to Rust, if there is one and I'm just too low on caffeine to see it, removing the 'a will result in a compile-time error instead of a dangling pointer.)

You tend to see this kind of verbosity more in the guts of the libraries you depend on than in code you yourself write, because it generally only crops up when you're trying to expose a "just accept anything and do what I mean efficiently" interface where you need to explain how things like type inference should flow from the code calling you to the code you're calling. (Basically, same situation as the horror that is the guts of the C++ STL, but more readable.)

...and no, I don't know what this code is from and I've never used it before. I just recognize the types from the standard library and the interfaces from Serde, I know that reqwest is an HTTP client library (not that it was necessary information), and this kind of "do what I mean" API design is commonplace, inspired by how Rust's standard library interfaces enable things like let my_foo: Foo = bar.into();

[Daktyl198]

It depends on who you ask. There is a list of criteria I have before I would use it, but I do not want to make this about me. One thing I believe to be common practise is to up a version number to 1.0, or even 1.1, once code is considered stable. And this is not the case yet.

Rust PNG has a version number of v0.17.15.
Zune PNG has a version number of 0.4.10.
Wuffs PNG has a version number of 0.4.0-alpha9.

The claim of being production-ready seems like an unfortunate choice of words when I look at those version numbers.

SemVer hasn't been the standard for version numbers in a long time. IIRC it started from software projects being stable, but wanting to reserve the right to make breaking changes without people bitching so they refused to ever version their software 1.0. Eventually that just became normal. For example, WUFFS has been in production use in Google Chrome for a while now, despite only having a version number of 0.4.

It's unfortunate, but it is what it is.

[ssokolow]

Assuming semver that's a really good point. Even if it's all "done" actually guaranteeing stability is a pretty important step

Rust isn't the only place where production-ready things don't want to do a semver-break just to apply a v1.0 number... to the point where someone made https://0ver.org/ as a joke.

[sdack]

For example, WUFFS has been in production use in Google Chrome for a while now, despite only having a version number of 0.4.

Here a quote by the developers of Wuffs PNG:

Version 0.3 (April 2023) is the latest stable version. Stable means that its API won't change any further, but being a "version 0.x" means that:

• It will not have long term support.
• Newer versions make no promises about compatibility.

The compiler undoubtedly has bugs. Assertion checking needs more rigor, especially around side effects and aliasing, and being sufficiently well specified to allow alternative implementations. Lots of detail needs work, but the broad brushstrokes are there.

I leave it to others to decide whether this qualifies as production-ready. Personally, I do not think for something being "used in production" means the same as "production-ready", or else the worst code can qualify as such if it only gets used.

[ssokolow]

Rust is supposed to be like c++ but with training wheels for imbeciles. Competent c++ devs tend not to like it because it's not actually in any meaningful way better than c ++ for competent devs. But getting to that level takes a lot of time which makes rust attractive for lazy scrubs and fresh faced devs.

In my experience, competent C++ devs in real-world situations like it because it means that the less competent contributors can bang their heads against rustc instead of wasting the competent devs' time auditing patches with avoidable mistakes.

The whole point of Rust is to make it possible to extend that "automate away the drudgework instead of doing it by hand" part of the UNIX philosophy to as many parts of programming as possible.

[ssokolow]

Yes, and don't get me started on the crazy null-terminated C-strings and the unsafe string handling functions...

*nod* Rust using Pascal-style counted strings and arrays instead of null-terminated ones makes it much easier and cleaner to write and maintain parsers which do less copying because the stock, standard string/array constructs let you reference a piece of something bigger as easily as referencing the whole thing.

[ssokolow]

Can you see why people prefer Python?
image.png

As someone with 20+ years of Python experience, I'm migrating my projects to Rust because I'm tired of having to reinvent Rust's verbosity even more verbosely (and less reliably) in Python unit tests to push back against "It works! Don't f**k with it!" when the writing ends and the maintaining begins.

[ssokolow]

There was a time programmers made extensive use of assembler and didn't rely on a compiler to auto vectorize a thing, they hand optimized their code.

Yes, and as Ahoy touched on in RollerCoaster Tycoon was the last of its kind., that era was killed by superscalar CPUs.

The knowledge burden for optimizing for modern CPUs has just grown too high to be practical to expect people who aren't compiler developers to know it in addition to whatever their specialty is.

In a sense, you're lamenting the loss of the era before the printing press when human knowledge was small enough that a single nobleman could have a passing competence in everything. Specialization is an inevitability as technology advances and human brains and lifespans don't.

[ssokolow]

(llvm? As far as I know rustc uses llvm as a compiler backend)

Yes. LLVM is rustc's primary backend, though people are working, at various stages of completion, on a Cranelift backend to provide a more Go-like trade-off between compile time and performance for debug builds, a GCC backend for ISA targets or codebases where LLVM is unsuitable, and a .NET CLR backend for incorporating Rust modules into .NET codebases without need for per-platform cross-compilation support.