Rav1e 0.5 Beta Released For This Rust-Written AV1 Encoder

[arQon]

modern C++ (snip)

Jep, I'd agree with everything in that paragraph.

> C ... In theory, it is perfectly and totally replaceable by Rust

I'm not sure it is yet. Every time you see Rust used for anything other than - hrm, not sure how to phrase this: "trivial" is the wrong word, since you can certainly write VERY meaningful programs in Rust. "userspace-y", or maybe "application-y", is about as close as I can get to expressing it - you find all the "heavy lifting" or some other critical core piece is actually calling out to C. That may be predominantly for legacy reasons, but given how enthusiastic most Rust fans are about the language it's a little odd.

As you say, no matter how "good" Rust gets, chances are it'll never be a good fit for some projects, but I think it's also fair to say that those are likely to not only be highly-specialized, but also implicitly "small enough" that a lot of the problems with C simply vanish just because the codebase is so small.

> Rust (snip) change the game about threading and memory safety.

Do you have any idea how many times I've heard that exact same claim made for any of eleventy-thousand other languages over the years?
In the time it's taken those languages to go from The Great White Hope to something even their developers have forgotten ever existed, I've written servers that process millions of trades/searches/messages/etc per day, running on OSes doing even more work, without ever hitting race conditions, or "memory safety" errors, or leaks - and so have plenty of other people. Anything that makes that easier is WELCOME, certainly, but we don't NEED it - because we've already proven that we don't.
And honestly, it's really not even that hard: it just requires competent design up front, and competent implementation of a fairly tiny amount of "core" code, and after that it's not really much different to any other development. Even your junior staff can do meaningful work without constantly running into problems that are over their heads, and the only problems you ever run into are the result of people not even understanding the *concept* of multi-threading: something that Rust won't help with at all.

I'm not saying Rust's improvements aren't highly desirable: I'm absolutely in favor of anything that "enables" developers. But we've been here before, over and over and over again, and the bottom line is that *you cannot turn a mediocre developer into a good one just by improving their tools*. This is the lie that gets repeated every time. It has never come true yet, and it never will. All you're doing, at best, is enabling bad developers to write bad multi-threaded systems instead of bad single-threaded ones. Whoop-de-do. The end result will belong in the same dumpster either way.

> C++ ... does not solve the root problem, something Rust instead does by design.

No, it doesn't. Rust makes it difficult - potentially impossible - to cause a certain subset of *technical* failures when working with threads and memory. It doesn't do anything to help with the logic errors that represent a far larger percentage of the bugs in real systems.

> Plus Rust has most of the modern fancy things about OOP and functional programming that makes fashion people happy nowadays.


I think the only appropriate response to that is "f**k fashion", and a suggestion that people like that should never be allowed anywhere near a compiler in the first place... :P

> I would go for C and Rust; C++ seems like a dead end in the midterm

FWIW, I think you're exactly wrong, but we'll see how it goes. Rust has until the hype jumps to the next language that "fixes everything" to establish an actual base. The kernel is potentially enough to stop it dying out completely, but it'll still be a tiny fraction of the size of the C++ community even with that much help.

> (C++21 removes the pointers, something that should have been done years ago

No it doesn't, and no it shouldn't. For all that I remember just how hard it was for me to truly grok pointers (and that was coming from an assembler background!), to put it bluntly, anyone who can't cope with pointers absolutely does not belong on a development team, period. OTOH, anyone still using *raw* pointers in 2021 also absolutely does not belong on a development team for anything except legacy C-only projects that are willfully choosing to work with substandard tools. Which, unfortunately, is about 95% of FOSS projects...

C++'s pointer semantics, as of 11, *also* make it literally impossible to have memory-management issues unless you explicitly opt into them. I do get very tired of seeing bloggers etc (not you ) writing shitty code and then blaming the language because they didn't bother to learn it. You can't fix that sort of Stupid even if you make them program in BASIC instead. :P

[blackshard]

arQon
Nice bread for thoughts, as a first though I would opinion that it is wise for the C programmer to move to Rust instead of C++, but this requires some thinking about.

I approached modern C++ two or three years ago (I'm a medium-skilled C programmer), and I find it very error prone and low productivity language until you reach high skills and master the depths of the language. There are dozen of things, rules and keywords to remember and pitfalls are everywhere. It looks to me that the modern paradigm grown over some ancient fundamentals that needed a refresh and the refresh could not be done because there is the need for backward compatibility.

C language is an evergreen instead: it is medium-level imperative language that you want to use when you're constrained by time, space and memory requirements. You have full control, but you also need to take care of everything by yourself. I don't see anything wrong with this, the language itself is, by design, close to hardware than any other language except assembly, so there is nothing wrong with that, since the purpose is clear. In theory, it is perfectly and totally replaceable by Rust, but would you choose Rust for a single-thread program that runs on a microcontroller without an operating system that has to interact with a half-dozen sensors and nothing more?

Rust (which I never had the pleasure to program with yet) is something different that introduces some programming paradigms that change the game about threading and memory safety. These two issues are not address by C (because it is not its task, it's medium-level language, closer to hardware) but neither by C++. C++ yet includes a lot of safety threading things in its standard library, but does not solve the root problem, something Rust instead does by design. Plus Rust has most of the modern fancy things about OOP and functional programming that makes fashion people happy nowadays.

Moral: I would go for C and Rust; C++ seems like a dead end in the midterm (C++21 removes the pointers, something that should have been done years ago, maybe C++ should have born without pointers...)

[arQon]

I sort-of agree with most of this: C vs Rust is pretty much doomed to be a losing battle over time. It will take at least a decade, but it'll happen. Templates / parametric polymorphism are simply too valuable to keep passing up, and C's requirement that you spend half your time copypasting / including the same set of of void* workarounds into every project instead of actually doing the important things will kill it in the end, regardless of who the killer is.

C++ vs Rust though is a very different story. Rust is still missing some critical capabilities, regardless of how good it is at the things it CAN do. Linus is both right and very wrong about C++: right in that a single incompetent can cause massive harm by using C++ badly, but very wrong that the kernel shouldn't be using it. The way out is to define a subset of "acceptable C++" and move to that, and it is absolutely a failure on Linus's part that the kernel didn't do that *at least* 10 years ago, and probably more. The stronger type-checking alone, plus const, plus inline variable declaration, plus STL-like templating for standard containers etc, would have *massively* improved the quality of the kernel source (and probably removed several million lines of copypasta from it).

Rust will never kill C++: it would take at least decades, and by then something else will have long since appeared that has new fanboys shouting about it.
(Which is probably Rust's biggest problem right now: getting enough developers to have any momentum left when that happens, and the "ooh, NEW SHINY!" peels off to write blogs praising that instead and calling out Rust as "inadequate, flawed, etc etc", the same way they did with Go before it).

So the question now is really "where will the last generation of C developers go"? To the "easy" language, i.e. Rust, where they'll have to completely relearn syntax (which TBH isn't anything like the hurdle some people make it out to be) and lose the ability to do certain things (thus requiring them to work in Rust AND C still), or to the "hard" language, where they keep all the familiarity and all the power/ability?
For the kernel, that decision's already been made. I think it's the wrong one, but I'm not Linus. For everything else, it's anybody's guess: it'll mostly just come down to which one a project's leads favor, which is going to be overwhelmingly C++ for at least the next few years, but may change over time if Rust ever gets any real-world traction.

[mdedetrich]

True. But they could do the same in C, however, it seems to be a PITA.
I think, however, glibc is quite good at handling lots of small allocations. They usually reserve a memory reservoir per thread and hand out memory from that.

Indeed this is perfectly possible in C but of course much more error prone compared to Rust. Most games actually use region based allocators rather than malloc/free since malloc/free is not cheap and if your application has a point in time where you can free everything in context (i.e. the end of a level in a game) then the user experience is much better.

[oleid]

Although I share your opinion about potentially more optimized assembly code
However I read and compared Rust vs C code and it seems to me not very fair to use malloc in C and external bumpalo library in Rust, which is just an arena allocator (i.e. just a big allocated buffer with custom malloc/free which operates within the range of this buffer instead of system call to OS)

True. But they could do the same in C, however, it seems to be a PITA.
I think, however, glibc is quite good at handling lots of small allocations. They usually reserve a memory reservoir per thread and hand out memory from that.

[mdedetrich]

Although I share your opinion about potentially more optimized assembly code
However I read and compared Rust vs C code and it seems to me not very fair to use malloc in C and external bumpalo library in Rust, which is just an arena allocator (i.e. just a big allocated buffer with custom malloc/free which operates within the range of this buffer instead of system call to OS)

This is a fair criticism although it is also fair to point out using arena's in Rust is much safer than C

There are traits of rust (pun intended) which can make it provide far better assembler compared to C in like for like scenarios, an obvious one being that Rust has proper support for parametric polymorphism, i.e. in Rust (like C++) you can write a generic sort function thats optimized for the type. C cannot do this without having to manually write the sort for every type you want to work with, if you want to write a generic sort you need to use void* functions which is going to provide worse binary.

This is actually a very simple case, there are far more cases where Rust can compile better binaries because the compiler can prove more about the program compared to C, unfortunately due to bugs with noalias mode in LLVM (ironically historic dodgy C support) this has been disabled and they only re-enabled in March this year (with Rust developers joking how long it will last, i.e. see https://github.com/rust-lang/rust/is...ment-803880176)

[RedEyed]

It depends. Rust can be fairly low-level if it needs to be. Yet, high-level has its advantages as well. The compiler can, ideally, better see your intention and thus provide a more optimized assembly.


https://benchmarksgame-team.pages.de...ust-clang.html

Although I share your opinion about potentially more optimized assembly code
However I read and compared Rust vs C code and it seems to me not very fair to use malloc in C and external bumpalo library in Rust, which is just an arena allocator (i.e. just a big allocated buffer with custom malloc/free which operates within the range of this buffer instead of system call to OS)

[mdedetrich]

C does not handle concurrency in any way, does it?

Well yes and thats the problem.

[oleid]

Maybe it is the opposite: C is lower level than Rust, at most Rust has to catch up (at most).

It depends. Rust can be fairly low-level if it needs to be. Yet, high-level has its advantages as well. The compiler can, ideally, better see your intention and thus provide a more optimized assembly.

https://benchmarksgame-team.pages.debian.net/benchmarksgame/fastest/rust-clang.html

[Quackdoc]

aomenc is the best encoder to use. the quality gains over svt-av1 is a large amount