The DAV1D AV1 Video Decoder Has Become Very Fast

[tildearrow]

But is Dav1d written in Rust?

But why do you think Rust is the only language capable of "memory safety" and that "C in general is evil"?

And if it were written in Rust, would you accept assembly optimizations or are they evil to you as well?

[uid313]

But why do you think Rust is the only language capable of "memory safety" and that "C in general is evil"?

Because my personal experience with C was very unproductive, tedious, burdensome, and painful. So many cryptic unhelpful compiler errors and warnings, so many segfault crashes.
I am a terrible C programmer, and I realize there are other much better programmers than me.

I don't think C is inherently evil, I just don't personally like it. I don't think it is a bad language, and for some use cases it might be a very good language, example for heavy processing and calculation, rendering, maybe a game engine.

But for certain things, where I need to parse third-party internet-delivered content, such as music or videos, which is done by interpreters, decoders or parsers, then I think security is very important, which is why I think for such things a more robust and type safe language (such Rust) is preferred (over C).

I never said that Rust is the only language capable of memory safety, but my understanding is that it is safer than C due to a better and safer type system.
But my understanding is that Ada is supposed to be rather type safe too, hence its use in industrial and military application.
Also TypeScript and Kotlin have better null handling than C.

And if it were written in Rust, would you accept assembly optimizations or are they evil to you as well?

I am not much for Assembly, it is so unreadable and difficult to understand. Also it is not portable. Even among people who do understand it, the people who really do understand it are very few and rare, hence often compilers tend to compile machine code that is faster than hand-optimized assembly code.

But like it or not, at the end of the day there might be significant performance benefits to writing assembly code in order to take advantage of ISA-specific instructions. In those use cases it should be used.

If you can use hardware acceleration, then that it is great.

[shmerl]

But why do you think Rust is the only language capable of "memory safety" and that "C in general is evil"?

And if it were written in Rust, would you accept assembly optimizations or are they evil to you as well?

Cost of memory safety in C is prohibitively high in comparison with Rust. I mean developer effort cost. That's the whole point.

See a recent post about it, by a C developer: https://dpc.pw/still-in-love-with-rust

[Creak]

Because my personal experience with C was very unproductive, tedious, burdensome, and painful. So many cryptic unhelpful compiler errors and warnings, so many segfault crashes.
I am a terrible C programmer, and I realize there are other much better programmers than me.

I don't think C is inherently evil, I just don't personally like it. I don't think it is a bad language, and for some use cases it might be a very good language, example for heavy processing and calculation, rendering, maybe a game engine.

But for certain things, where I need to parse third-party internet-delivered content, such as music or videos, which is done by interpreters, decoders or parsers, then I think security is very important, which is why I think for such things a more robust and type safe language (such Rust) is preferred (over C).

I never said that Rust is the only language capable of memory safety, but my understanding is that it is safer than C due to a better and safer type system.
But my understanding is that Ada is supposed to be rather type safe too, hence its use in industrial and military application.
Also TypeScript and Kotlin have better null handling than C.

As a C++ dev for 12 years now, I agree.
Writing C/C++ code that is safe and multithreaded is indeed doable, but it's a huge pain in the ass: lots of ropes to hang yourself with.
I also believe other (more modern) code languages can do better, and Rust is well placed in this race.
Even C#, thanks to all the tweaks made by Unity, can be a better language while being at least as fast (take a look at what's beside the ECS tech). I'm talking compiler errors when writing code that can lead to race conditions!! (which a C++ programmer wouldn't even dream of)

I am not much for Assembly, it is so unreadable and difficult to understand. Also it is not portable. Even among people who do understand it, the people who really do understand it are very few and rare, hence often compilers tend to compile machine code that is faster than hand-optimized assembly code.

But like it or not, at the end of the day there might be significant performance benefits to writing assembly code in order to take advantage of ISA-specific instructions. In those use cases it should be used.

If you can use hardware acceleration, then that it is great.

Although it's written "assembly" in the VLC blog post, I'm fairly sure it's not assembly, but rather SIMD intrinsics. It's way more portable and maintable than assembly and it allows to take advantage of (modern) CPU features such as AVX-256 that, for instance, can treat 8 floats at the same time on one thread.

[log0]

Although it's written "assembly" in the VLC blog post, I'm fairly sure it's not assembly, but rather SIMD intrinsics. It's way more portable and maintable than assembly and it allows to take advantage of (modern) CPU features such as AVX-256 that, for instance, can treat 8 floats at the same time on one thread.

Nope. It is actual assembly. Intrinsics aren't that great, as the compiler can still mess things up underneath.

[microcode]

If the encoder can encode 10FPS per thread but can only use 1 thread, and improving that to 12FPS takes the same amount of work at making the encoder use up to 4 threads, Google will chose to improve the FPS per thread instead of increasing the total number of threads it can use. So both Google and home users see a 20% improvement, whilst it the thread count had been improved Google would have seen no improvement and home users would have seen a 300% improvement.

I think that it's likely YouTube will use hardware or at least FPGA encoders in the long run. The home users, they expect, will generally use hardware encoders as well. As for multithread, it's possible (at a pretty big reduction in coding efficiency) to encode the video in tiles, which can work on completely separate threads.

[microcode]

Because my personal experience with C was very unproductive, tedious, burdensome, and painful. So many cryptic unhelpful compiler errors and warnings, so many segfault crashes.
I am a terrible C programmer, and I realize there are other much better programmers than me.

I don't think C is inherently evil, I just don't personally like it. I don't think it is a bad language, and for some use cases it might be a very good language, example for heavy processing and calculation, rendering, maybe a game engine.

But for certain things, where I need to parse third-party internet-delivered content, such as music or videos, which is done by interpreters, decoders or parsers, then I think security is very important, which is why I think for such things a more robust and type safe language (such Rust) is preferred (over C).

I never said that Rust is the only language capable of memory safety, but my understanding is that it is safer than C due to a better and safer type system.
But my understanding is that Ada is supposed to be rather type safe too, hence its use in industrial and military application.
Also TypeScript and Kotlin have better null handling than C.


I am not much for Assembly, it is so unreadable and difficult to understand. Also it is not portable. Even among people who do understand it, the people who really do understand it are very few and rare, hence often compilers tend to compile machine code that is faster than hand-optimized assembly code.

But like it or not, at the end of the day there might be significant performance benefits to writing assembly code in order to take advantage of ISA-specific instructions. In those use cases it should be used.

If you can use hardware acceleration, then that it is great.

Basically all of the core duties of any reasonably efficient AV1 decoder are inherently memory unsafe, as far as I can tell. If you end up with an equally questionable mess of hot, unsafe loops, it seems like it would be better to write this mess in a language which is more familiar, has more tools, has more implementations, has more practitioners, is available on more platforms, and can be more readily embedded in a greater number of programs.

[Creak]

Nope. It is actual assembly. Intrinsics aren't that great, as the compiler can still mess things up underneath.

Oh my god you're right! O_O
There's 50KB of ASM code in one file: https://code.videolan.org/videolan/d...x86/x86inc.asm

I truly believe it's a bad thing to do that. Game engines are using intrinsics without any problem and they get the most out of every MHz of each CPU cores. I can understand that this way is faster to execute in the end, but it is so unmaintainable and unportable
The cost/benefit ratio must not be very good.

Well I guess it's good for them that I'm not part of their team then!

[GreenReaper]

libaom has been incorporating avx2 and [s]sse(2-4) versions of crucial functions, as well as increasing the threading capability. It will be interesting to see where things end up six months down the line when we will be seeing more AV1 out there.