Rav1e 0.5 Brings More Speed-Ups For This Rust AV1 Encoder

[dado023]

I am doing two pass and tiling.

What is --cpu-used parameter you are using?

[aorth]

I have given up hope for AV1. I have a blu-ray rip being re-encoded using AOM on a Ryzen 7 1700X I gave the encoder 12 threads. It has been running for over a month and still isn't done. That is at a point where I still have to lose details like fly away hairs etc. I appreciate the effort they put into creating it but AV1 is for mega companies like Google and Netflix not people. You are far better off mining crypto and building a data center to hold your movies that you are trying to re-encode to AV1.

I agree it seems to be something wrong with your setup or your threading. The default speed in aomenc is 0 if I recall correctly, which is abysmally slow and not meant for anyone to actually use. For the record, I get very good results and speed with ffmpeg/libaom:

ffmpeg -hide_banner -y -i file.mp4 -c:v libaom-av1 -b:v 0 -crf 40 \
              -aq-mode 0 -c:a libopus -b:a 16k \
              -sc_threshold 0 -cpu-used 4 \
              -tiles 4x4 -row-mt 1 \
              -auto-alt-ref 1 -lag-in-frames 25 \
              -g 300 -threads 12 \
              -f webm file.webm 2>/dev/null

The above should easily saturate 12 threads due to the 4x4 tiles, and quality should not suffer if your content is 4K, otherwise maybe try 2x2 with 6 threads. In all my testing I've found cpu-used 4 to be the sweet spot, and cpu-used 2 to be ~5x slower. You will also probably need a lower CRF if you have Bluray content.

[Toggleton]

and be sure that you use an up to date aom "Debian finally updating its aom packages from 1.0.0 (3.2.0 now in Debian Unstable) a few days ago. aom 1.0 is awful slow compared to newer releases.

[Michael_S]

My experience, when I digitised my library years ago, was that h265 wasn't worth the compression time anyway (using SW encoders, because GPU ones suck hard). While certainly better in terms of filesize to quality (vs h264), it was massively slower to encode, and only barely usable at 1080p on the devices that don't have HW decode, all to save ~1/3 the disk space in an era where hard drives are ~$100 per 4TB.

To put it simply, h265 is only "worth" it if all your devices support it AND you aren't doing the encoding yourself, i.e. you're just torrenting. Given how little value h266 adds, in return for yet another tenfold increase in encoding time, using that at ANY point in time over the next decade is simply stupid. And the same goes for AV1, except even more so, since almost nothing does or ever will support HW decode of that.

Thanks for the great post.

Even at $100 per 4TB, there is the issue of backups. I use H.265 to get my entire collection to 2.2 TB, so I can fit a complete copy onto four different disks. That seems paranoid, but the rip-and-reencode process took over a year and I'll be furious if I lost everything.

My wife and I have been collecting DVDs and Blu Rays for over twenty years. Our 800 films (most DVD) are encoded at what I believe is a quality level good enough that people can't see a difference. I know I can't. For some old SpongeBob episodes, my ripped and reencoded versions looked awful, so I put the DVD in - and it looked just as terrible, so I had done nothing wrong and just faithfully re-encoded garbage from the company.

One thing to note is that I think most newer gadgets have H.265 hardware decode. I haven't run into any device that had problems with my ripped videos in more than four years. I had problems with Android and some audio codecs a few years ago, but never H.265.

I'm sure you have, but that's simply the result of inept users and mediocre tools. If used with even the smallest amount of competence, h265 is unquestionably ~30% "better" than h264 for any given bitrate/quality target. What I found when looking around a few years ago was "experts" who've come up with their own "super good" custom settings for 30+ parameters that they don't understand AT ALL, and unsurprisingly just produce far worse results on both metrics than simply using a crf target does.

Agreed. I ran my own H.264 vs. H.265 comparisons using equivalent CRF values (I think H.264 and H.265 have different CRF scales). The H.265 took three or four times as long to encode, but always gave equal image quality for less disk space.

[shoky]

I agree it seems to be something wrong with your setup or your threading. The default speed in aomenc is 0 if I recall correctly, which is abysmally slow and not meant for anyone to actually use. For the record, I get very good results and speed with ffmpeg/libaom:

ffmpeg -hide_banner -y -i file.mp4 -c:v libaom-av1 -b:v 0 -crf 40 \
-aq-mode 0 -c:a libopus -b:a 16k \
-sc_threshold 0 -cpu-used 4 \
-tiles 4x4 -row-mt 1 \
-auto-alt-ref 1 -lag-in-frames 25 \
-g 300 -threads 12 \
-f webm file.webm 2>/dev/null

The above should easily saturate 12 threads due to the 4x4 tiles, and quality should not suffer if your content is 4K, otherwise maybe try 2x2 with 6 threads. In all my testing I've found cpu-used 4 to be the sweet spot, and cpu-used 2 to be ~5x slower. You will also probably need a lower CRF if you have Bluray content.

I just registered to help you with this issue.
I found that using ffmpeg that comes with Kubuntu is utterly s**** slow. I only get like 0.01 fps on Ryzen 5900 with 64GB of RAM. I switched to Snap version and speed increased to 1 fps. So I tested ffmpeg on Windows and noticed that my I7 7700HQ on 3.4 GHz encodes 26 FPS.

So now, I'm using this to encode some tutorial files with :

wine ffmpeg.exe -y -hide_banner -i "$file" -c:v libaom-av1 -strict experimental -crf 48 -cpu-used 5 -row-mt true -threads 12 -c:a libopus -b:a 24k -c:s copy out.mkv

With this I get ~0.75X encoding speed per video, so I usually encode 4-6 videos in parallels (ffpmeg usually uses 3-4 threads per video).

Also note that mkv gives gives smaller files than mp4 (less overhead).

I noticed that using tiles 2x2 (or 1x4) and -sc_threshold 0 -g 300 produces way bigger files without noticeable speedup (at least with my testing).

Also, I would really like to know how anyone gets 70 FPS ?

[aorth]

I noticed that using tiles 2x2 (or 1x4) and -sc_threshold 0 -g 300 produces way bigger files without noticeable speedup (at least with my testing).

I was under the impression that the rule of thumb for ''-g'' was to use ten times the FPS of the input source (ie 240 or 300 for 24 and 30 FPS content).

And yes, it seems there is a tradeoff with using tiles to enable more threading. From what I've read on r/AV1 the more tiles you have the more "edges" (literally surface area around the tiles) you have, which decreases quality and perhaps increases file size. There are many people who prefer to move threading completely out of the encoder, using tools like av1an to do massively parallel chunked encoding on each frame. They may be right, but I think av1an's CLI is finicky and prefer to stick with ffmpeg and aim for a target VMAF of 90 or 95 depending on my content.

[Toggleton]

I just registered to help you with this issue.
I found that using ffmpeg that comes with Kubuntu is utterly s**** slow. I only get like 0.01 fps on Ryzen 5900 with 64GB of RAM. I switched to Snap version and speed increased to 1 fps. So I tested ffmpeg on Windows and noticed that my I7 7700HQ on 3.4 GHz encodes 26 FPS.

So now, I'm using this to encode some tutorial files with :



With this I get ~0.75X encoding speed per video, so I usually encode 4-6 videos in parallels (ffpmeg usually uses 3-4 threads per video).

Also note that mkv gives gives smaller files than mp4 (less overhead).

I noticed that using tiles 2x2 (or 1x4) and -sc_threshold 0 -g 300 produces way bigger files without noticeable speedup (at least with my testing).

Also, I would really like to know how anyone gets 70 FPS ?

I guess your problem was that debian and ubuntu did not update libaom and so it is using the slow version from 2019(the 1.0 was not optimized at all. Not sure what aom version the ffmpeg snap is using

Its a shame that People have a bad opinion of AV1 cause debian/Ubuntu are shipping outdated Software). I guess a better way than running ffmpeg exe over wine you could use a static compiled ffmpeg like https://johnvansickle.com/ffmpeg/
and run it like ./ffmpeg-git-20211029-amd64-static/ffmpeg -y ........

or compile ffmpeg yourself(https://trac.ffmpeg.org/wiki/CompilationGuide/Ubuntu)

at least there is now a bit movement to update it https://bugs.launchpad.net/ubuntu/+s...m/+bug/1947695

from the FFmpeg wiki - Note: Users of libaom older than version 2.0.0 will need to add -strict experimental (or the alias -strict -2).
If it does not run without -strict experimental it is very likely 1.0 libaom

[shoky]

I guess your problem was that debian and ubuntu did not update libaom and so it is using the slow version from 2019(the 1.0 was not optimized at all. Not sure what aom version the ffmpeg snap is using

Its a shame that People have a bad opinion of AV1 cause debian/Ubuntu are shipping outdated Software). I guess a better way than running ffmpeg exe over wine you could use a static compiled ffmpeg like https://johnvansickle.com/ffmpeg/
and run it like ./ffmpeg-git-20211029-amd64-static/ffmpeg -y ........

or compile ffmpeg yourself(https://trac.ffmpeg.org/wiki/CompilationGuide/Ubuntu)

at least there is now a bit movement to update it https://bugs.launchpad.net/ubuntu/+s...m/+bug/1947695

from the FFmpeg wiki - Note: Users of libaom older than version 2.0.0 will need to add -strict experimental (or the alias -strict -2).
If it does not run without -strict experimental it is very likely 1.0 libaom

Oh, I don't have bad opinion about AV1, quite the opposite, I really like it. It gives way better quality (especially on tutorials with code which is way more readable than x265).

And this ffmpeg version from https://johnvansickle.com/ffmpeg/ encodes at 1.35x-1.85x speed vs 0.65x-0.85x ffmpeg.exe version 2021-10-28-git-e84c83ef98-full_build from www.gyan.dev.
Now, encoding 4 videos in parallel uses ~50% of all cores on 4.5 GHz Ryzen 5900.

So, thanks for the link Toggleton.

[andreano]

Actually there are very few countries that recognize software patents

Sweet dreams. HEVC Advance alone has patents in 231 countries (including expired patents):

https://accessadvance.com/wp-content/uploads/2021/06/2021.06-Website-Patent-List.xlsx

The thing is that there is no such thing as a pure software patent, because that's just stupid when you can patent the full "method and apparatus" – the algorighm and the computer together.

[MadeUpName]

Sweet dreams. HEVC Advance alone has patents in 231 countries (including expired patents):

https://accessadvance.com/wp-content/uploads/2021/06/2021.06-Website-Patent-List.xlsx

The thing is that there is no such thing as a pure software patent, because that's just stupid when you can patent the full "method and apparatus" – the algorighm and the computer together.

Even the brain dead US courts finally threw their hands in the air and said enough to the millions of patent applications that do X that we have been doing for a thousand years but with the novel idea of doing say math on a computer. Hey give me a patent so I can act like an ass hole. You actually have to invent some thing new in order to get a patent. A general purpose commuter in and of it's self is hardly new. Now some one could create an ASIC to get the job done but if I was using an ASIC I wouldn't be trying to make it work purely in software.

That list is meaningless. Sure companies making hardware devices have issues but that isn't what I am doing. Just because some troll says they have a patent and it affects me doesn't make it true until a court in the relevant jurisdiction says it is and the courts here have been abundantly clear software is non-patentable along with a whole lot of other American get rich quick schemes. I suggest you go read MPEG-LAs web site they know the days of av patent trolling are over. You also forget that the patent trolls have put limits on who has to pay. I believe that number starts at an income of a million dollars a year.