Rav1e 0.4 Released For Faster Rust AV1 Encoding - But Still Is Quite Slow

[aspen]

Did you know that significant part of the changes from version to version in librsvg is devoted only to follow rust language definition?

"Did you know that significant part of the changes from version to version in librsvg is devoted only to follow rust language definition?"
Not sure what you mean by this but every major version in the changelog has actual fixes and changes, not just "following Rust's changes" if that's what you mean

Generally speaking I'm not against rust ideas because rust approach is really nice promise of solving some problems on top of evolving hardware which will be possessing more and more CPU cores even in case of commodity laptop cases.
Issue only is that in reality this is still only promise ..

There's already decent tooling in Rust for taking advantage of things like multi-threading, the biggest example being rayon.

Simple Linus will say "go on /dev/tree" if you will be asking to integrate cargo based build tooling into current make based framework.

I believe Linux in-tree modules would just directly use rustc from within makefiles, and while said modules would be optional, they'd be compiled by default if the compiling system has rustc available. The fact that Linus is actually discussing the idea is a decent enough sign.

Ultimately, all cargo does is download dependencies and call rustc. Is it different from cmake and meson? Yes. Cargo is closer to something like npm than it is to cmake or meson (albeit cargo does have better support than npm for build-time stuff, via build.rs, which is often used to compile C/C++ dependencies when running cargo build)

But still, no reason you can't just call rustc from a Makefile. It's not some extremely esoteric interface, I've used rustc directly myself while experimenting with things before.

[Quackdoc]

Actually, tiles is the least fancy parallelism (all encoders support it), and the most important reason to enable it is for decoding performance: Decoders have tile threads too!

Indeed, I see a lot of people saying libaom encoder is slow, and badly optimized for parallel encoding, while true to some extent, projects like av1an actually make the encoder really fast.

I like neav1e as it has sane defaults, and properly chunks videos for parallel encoding, and on my 2600 is the most preformant option (Using libaom, giving ffmpeg 2 threads per job and just making many jobs by Choping the video up), for me its even faster than svt-av1

[bug77]

Bl*dy hell .. but why it is written in rust?

Because the developers decided to go that route.

It is fundamental design issue using tooling which forces to use own build automation (cargo) and still is very unstable! (from design perspective)

Apparently not. But I would love to hear why cargo is supposed to be such a flawed approach.

First of all I'm not English native speaker. If you are trying to make here "argument" that is your problem.
Almost none from above is about how stable rust is. About that I've been talking.
Really I'm not interested about Phoronix forum "morphology" as well.
Please stick to the subject and move away from persons.

Did you know that significant part of the changes from version to version in librsvg is devoted only to follow rust language definition?
Mesa, kernel and other rust part of the rust activity are still on POC level.
This is why still things like accepting rust code into main kernel Linus kernel code will definitely not happen in at least 2-3 years.
Mesa and rust still is only proposal. (look on mesa git tree).

Generally speaking I'm not against rust ideas because rust approach is really nice promise of solving some problems on top of evolving hardware which will be possessing more and more CPU cores even in case of commodity laptop cases.
Issue only is that in reality this is still only promise ..

IMO fist king of "sine qua non" condition which needs to be fulfilled to open real doors for rust is on tooling layer.
Cargo is to alien. It must be easier way to compile rust code using even make that using meson and cmake. Meson seems is making progress on that area but still few things needs to be finished.
Simple Linus will say "go on /dev/tree" if you will be asking to integrate cargo based build tooling into current make based framework.

I'm not sure whether you realize how silly it is to see people telling other people how unfit Rust is for anything, while the number of projects using Rust grows all the time. And Rust has been the most loved programming language since it was born.

[andreano]

You don't need fancy parallelism in your encoder if you can do chunk encoding : D

Actually, tiles is the least fancy parallelism (all encoders support it), and the most important reason to enable it is for decoding performance: Decoders have tile threads too!

[kloczek]

Your post is barely coherent. If you want to be an elitist nitwit, go to HN instead. Actually no, Phoronix is perfect for elitist nitwits, it's a good containment forum. Keeps them away from everywhere else.

I'm tired of idiots saying that everything should be C or C++ instead. Rust has it's clear advantages and disadvantages, and so does C/C++, most things have both pros and cons, but ultimately, whoever is willing to put in the work to maintain a project gets to decide how they do so, and unless you want to maintain their software for them, you don't get a say in it.

I've looked through many posts relating to Rust - whether it's relating to Rav1e, bzip2, Mesa, Linux kernel modules, whatever, someone like you always bursts in and tries to play chess like a pigeon.

I am not interested in what career you have or however many years you've been toiling away in said career. I just want it to be very well known that you kinda don't get to bash the methods being used for anything unless you're fully willing to do it whichever way you feel is "right", but most people who complain on online forums are the kind of people who would never actually put in the effort to do it "right".

Also, once again, I can't really refute whatever you're going on about unless you can rewrite it in coherent english, because nobody has any clue what you're saying, to put it lightly.

First of all I'm not English native speaker. If you are trying to make here "argument" that is your problem.
Almost none from above is about how stable rust is. About that I've been talking.
Really I'm not interested about Phoronix forum "morphology" as well.
Please stick to the subject and move away from persons.

Did you know that significant part of the changes from version to version in librsvg is devoted only to follow rust language definition?
Mesa, kernel and other rust part of the rust activity are still on POC level.
This is why still things like accepting rust code into main kernel Linus kernel code will definitely not happen in at least 2-3 years.
Mesa and rust still is only proposal. (look on mesa git tree).

Generally speaking I'm not against rust ideas because rust approach is really nice promise of solving some problems on top of evolving hardware which will be possessing more and more CPU cores even in case of commodity laptop cases.
Issue only is that in reality this is still only promise ..

IMO fist king of "sine qua non" condition which needs to be fulfilled to open real doors for rust is on tooling layer.
Cargo is to alien. It must be easier way to compile rust code using even make that using meson and cmake. Meson seems is making progress on that area but still few things needs to be finished.
Simple Linus will say "go on /dev/tree" if you will be asking to integrate cargo based build tooling into current make based framework.

[Quackdoc]

Somebody needs to enlighten the world and write the first (non-outdated) word about parallelism in AV1 on Wikipedia.

You don't need fancy parallelism in your encoder if you can do chunk encoding : D

[andreano]

aspen
1. rav1e has no threading other than tile-threading IIRC.
2. aomenc-av1 has tile threading, row threading(row-mt), and more recently, temporal threading, but no frame parallel threading.
3. SVT-AV1 has tile threading, row-mt, and frame parallel threading.

---

For tile threading, there's a very good article about it by rav1e devs:

https://blog.rom1v.com/2019/04/implementing-tile-encoding-in-rav1e/

For block based row threading, you have this from ittiam:

https://www.ittiam.com/vp9-video-encoder-faster-turnaround/#

And this for a more in depth explication about a similar method to row-threading and frame parallelism:
https://x265.readthedocs.io/en/stable/threading.html

Somebody needs to enlighten the world and write the first (non-outdated) word about parallelism in AV1 on Wikipedia.

[andreano]

There is some truth to this: The state of AV1

… cpu-used=1 by default and no one knows what tiling is …

[BlueSwordM]

aspen For tile threading, there's a very good article about it by rav1e devs:

https://blog.rom1v.com/2019/04/implementing-tile-encoding-in-rav1e/

For block based row threading, you have this from ittiam:

https://www.ittiam.com/vp9-video-encoder-faster-turnaround/#

And this for a more in depth explication about a similar method to row-threading and frame parallelism:

https://x265.readthedocs.io/en/stable/threading.html

[aspen]

When looking at encoder speed, you always need to look at 2 factors:
- Single threaded encoding.
- Multi-threaded encoding.

In regards to the 1st point, all the current AV1 encoders(rav1e, aomenc-av1, SVT-AV1) are actually quite similar in terms of speed.

The difference comes to multi-threaded encoding:
1. rav1e has no threading other than tile-threading IIRC.
2. aomenc-av1 has tile threading, row threading(row-mt), and more recently, temporal threading, but no frame parallel threading.
3. SVT-AV1 has tile threading, row-mt, and frame parallel threading, which is why SVT-AV1 is overall the fastest in terms on single encoder testing, but aomenc-av1 has improved in this regard, which is nice. That is true mostly for the higher speeds(4-8). Once you get down to lower speeds, the encoder becomes less threaded and quite a bit slower.

That's basically a wide overview of the situation.

Huh, is there anywhere I can read up on the different types of threaded encoding? I'm curious about all this ngl