libvpx 1.4.0 Brings Faster VP9 Encode/Decode

[Gusar]

I'm not sure why you expect x265 to mature over time due to having better technology, but don't extend the same logic to VP9 since it also has better tech than h264.

I didn't explicitly mention vp9, that's true. But I did mention vp8 was crap years ago and now it's not, so you could infer that I expect all encoders to get better with time, including vp9. Especially since the vpx devs have started looking into psy optimizations and adaptive quantization.

Though I tested two of the aq modes available (variance and complexity), and they produced a blurrier result. This is probably why this guide explicitly turns aq off in the "best settings" example (it's off by default anyway, at least in the vpxenc utility). But with time, this could become something. Adaptive quantization was a big quality booster for x264.

[smitty3268]

I didn't explicitly mention vp9, that's true.

Sorry, i misread one of your earlier posts.

[Gusar]

I decided to repeat the encoding test, using newer versions of all the encoders:

vp8/9 v1.4.0-907-g91feec1
x264 core:148 r2579 73ae2d1
x265 1.7+382-7c83f7755422
xvid 1.3.4
theora git commit 12f20c7

Same clip, chapter 3 of Serenity, but with an added twist - an SD encode (720x432 anamorphic, DVD source), an HD encode (1280x544, Blu-ray source, downscaled to 720p), and a full HD encode (1920x816, Blu-ray source). If those resolutions seem off, well, the movie is in 2.35 format, meaning it has black bars, the resolutions are the result of cropping those away.


Encoder settings: Compared to last time, I gave vp9 more lookahead (lag-in-frames option), and I increased x265's psy-rd to 0.6 from the default 0.3 in an attempt to have it preserve more detail (I think this one backfired, it resulted in visible distortion in some places). Then, for the HD encodes, I increased x265's merange, to accommodate the higher resolution.

SD: http://pastebin.ca/3083067
HD: http://pastebin.ca/3083068
FHD: http://pastebin.ca/3083069


Encoder speed: VP9 encoding took even longer this time, but that's due to the higher lookahead mentioned above. The encoder hasn't gotten slower since the 1.4.0 release. Hasn't gotten faster either. Hopefully this will change in the near future, though it doesn't seem to be Google's priority.

SD: http://pastebin.ca/3083072
HD: http://pastebin.ca/3083074
FHD: http://pastebin.ca/3083077


And the most important part - visual results. A separate gallery for each resolution, three sets of pictures in each gallery, in each set the order is original, theora, vp8, vp9, x264, x265, xvid.

SD: http://www.imagebam.com/gallery/pilw...b0sm5fjfn0hgb6
HD: http://www.imagebam.com/gallery/nnl3...1zubibs2h12msp
FHD: http://www.imagebam.com/gallery/wxa3...bave5vklo3lnyh

I expect more from x265. It's beaten by x264 in all three encodes and that's with x264 being significantly faster. Xvid is very fast, but can't compete, it's blocky as hell. Theora is neither fast (because it's single-threaded) nor good, loses too much detail. VP8 is sufficiently fast, but not quite there quality wise. VP9 would be ok I guess, if it didn't take so effin' long to do its job.

x264 is simply out of this world. How it manages to produce the best picture, while being so fast at it, is... insane.


I would have liked to add Thor to the test, but it doesn't seem to have proper rate control yet. Also, I have a feeling it's really, really slow. We'll see what the future brings.

[Gusar]

I went and did low-bitrate encodes. 400 instead of 1050 kbit/s for SD, 800 instead of 2000 kbit/s for HD.

Visual results:
SD: http://www.imagebam.com/gallery/665j...4awtuil4u4pvew
HD: http://www.imagebam.com/gallery/sgja...1hqba3ss6tecl7

Shouldn't be hard to figure out which of those are vp9 and which are x264 . And I have to note that the vp9 encodes look even worse in motion, in particular the scene where Mal is walking off the bridge is dowright horrible in vp9.

So no matter what the bitrate, vp9 can't keep up. The cause of this is both the implementation (libvpx) and the tech vp9 has at its disposal. When it comes to tech, adaptive quantization (which vp8 didn't have at all) is less flexible than with h264. Then there's multiple reference frames (vp9 does have some form of it, but it's very limited) and b-frames, which aren't available at all. This has a reason that shouldn't come as a surprise - patents. But whatever the reason, their lack hurts vp9 quite a bit. And concerning implementation, well, the libvpx devs don't spend nearly enough time on rate control and psychovisual enhancements. Those two areas in particular is where x264 destroys all competition.

Fun fact 1, relating to tech: Cisco's Thor *does* have b-frames as well as multiple reference frames. If this makes it into NetVC, watch out Google!

Fun fact 2: Since a few days, libvpx-git contains vp10. I kid you not. Of course it doesn't offer anything new at this moment, but it's there.


To top it all off, I made a very special encode, purely for the lulz - cinepak!! Grab the cinepak encode here. Video party like it's 1992

[SystemCrasher]

B-frames are like double-edged sword. While they _can_ improve bitrate to quality ratio in various cases, they could literally kill picture quality with fire in another cases. So I would rather call 'em very questionable compression technique, to say the least. Not to mention decoder becomes more complicated and tends to have much worse issues if some frame has been corrupt, etc.

In some scenes, attempts to use B-frames tends to cause very noticeable and very specific attitude/artifacts, where you can literally see each I-frame occurence with naked eye. Once there was enough B-frames in sequence, it could get seriously "distorted" and differs from "original" frame quite a lot (each difference is small enough, though). I-frame suddenly corrects this difference. This causes very annoying and very noticeable picture quality "jumps" on each I-frame, especially in low-bitrate encodings. So you can see picture "resets" accumulated errors on I-frame, causing very specific visual effect. You can't see this attidude on still pictures, but its annoying on movies. And virtually all MPEG-based codecs expose this nasty attitude, here and there. Especially if one who encoded sequence thinks B-frames are cool and uses them a lot. OTOH I never seen anything like this in any of VPx codecs. It seems "P-frames-only" sequences do not expose such attitude (or much less sensitive to this issue).

And as for encoding examples by Gusar:
1) I do not get what is the point of encoding this strange movie. It seems to contain quite synthetic, computer generated graphics. While general purpose codec supposed to work well in all occasions, it's still quite specific case.
2) The only fair way to compare codecs is to grab uncompressed sequence, which was NEVER COMPRESSED BEFORE. So it does not contains compression artifacts of "previous" codec. Do you, Gusar, got this movie in RAW, uncompressed format? In original version? Without pre-existing compression artifacts?

Why it's only fair to compare on uncompressed "master copies"? Imagine you have crappy 128-kbit MP3. When master, original RAW audio (say, WAV) has been compressed, MP3 has thrown away some data to reduce bandwidth. Then you decompress it back to WAV. But it still sounds like a crap since data were thrown away and there is no way to get it back - that's why it called "lossy" compression, after all. So its worth of nothing it's "uncompressed". It's already "broken" and no way to fix it. Then you compress it with, say, ogg vorbis. Vorbis throws away different parts of data, adding extra compression artifacts. Since its algo different, it would make different decisions. Even more compression artifacts would appear. On other hand, if you recompress to MP3, MP3 compressor could make decisions, similar to "first" codec instance. So, does it means MP3 is better than Vorbis? Because recompressed MP3 could even sound better than Vorbis version does (Vorbis would have more artifacts). Nope! It means WRONG APPROACH!

Correct idea to compare codec A vs codec B is: grab master RAW data, which were NEVER COMPRESSED BEFORE. Apply codec A and codec B to these data. Compare compressed versions of data for A and B. Using pre-compressed streams to compare codecs is not fair and usually puts codecs similar to one used in source in advantage.

And btw: where to get RAW, uncompressed sequences? Legal and even free of charge? Of course, xiph.org for the rescue - they have plenty uncompressed sequences to chew on. There is catch though: uncompressed movies are HUGE. It would take some time to download. And messing with RAW video data is quite demanding for HDD bandwidth and space. But it's the only fair way to compare various codecs.

[Gusar]

B-frames are like double-edged sword. While they _can_ improve bitrate to quality ratio in various cases, they could literally kill picture quality with fire in another cases. So I would rather call 'em very questionable compression technique, to say the least. Not to mention decoder becomes more complicated and tends to have much worse issues if some frame has been corrupt, etc.

Picture quality regarding b-frames is about having a good encoder implementation. x264 overhauled their b-frame placement algorithm at least once if not more, so this isn't an issue. They're not questionable at all, x264 encodes will consist 70-75% of b-frames, because they're simply that valuable.

Yeah, it makes decoders more complex, but that's not an issue at all in practice. For example, Blu-ray specifies a strict pyramidal b-frame hierarchy (exactly to reduce complexity of hardware decoders), but actual hardware decoders out there don't care about that, they can handle without problems even more complex non-pyramid hierarchies. In fact, x264 is non-strict by default (but there's a --bluray-compat switch if you want strict Blu-ray compliance).

In some scenes, attempts to use B-frames tends to cause very noticeable and very specific attitude/artifacts, where you can literally see each I-frame occurence with naked eye.

Only with bad encoders, bad rate control causes it. With mpeg2, this was mitigated by using openGOP (open group of pictures). OpenGOP can be used with H.264 as well, but in practice isn't needed, at least not with x264.

OTOH I never seen anything like this in any of VPx codecs. It seems "P-frames-only" sequences do not expose such attitude (or much less sensitive to this issue).

Oh, I-frame pulsing was observed in the VPx family as well, it isn't immune to it, no codec is. It's not caused by available frame types. The cause is, like I said, bad rate control. And libvpx used to be bad at exactly rate control. But encoder optimizations make the problem go away. There's no I-frame pulsing in any of the encodes I did. Ok, not completely true, the Cinepak encode has it


Anyway, seems to me you're trying to argue that b-frames are bad because it supposedly increased I-frame pulsing with some encoders. So, you blame the tech rather than implementations. Not a good argument as I've explained above. For example, a typical DVD GOP is IBBPBBPBBPBBI, which means 75% of it are b-frames!
Similar percentage to what x264 produces, though x264 has dynamic frame placement rather that a strict structure - in my test, there can be up to 5 consecutive b-frames, you can see what the encoder decided in the stats:

x264 [info]: consecutive B-frames:  3.8%  6.2% 17.8% 13.6% 10.8% 47.7%

B-frames are used so much because the compression efficiency gain is simply so high. So VP9 lacking this tech is noticeably hurting it.

I do not get what is the point of encoding this strange movie. It seems to contain quite synthetic, computer generated graphics. While general purpose codec supposed to work well in all occasions, it's still quite specific case.

There is nothing "synthetic" about the clip. There's CGI at the beginning, but then it's people walking through halls and talking. I did mention a characteristic of the clip, it's quite dark, but you yourself said a codec is supposed to work well in all occasions, which *includes dark ones*. In fact, such a clip can very well expose issues that an _actually synthetic_ video (Big Buck Bunny, often used in encoder comparisons) won't.

The only fair way to compare codecs is to grab uncompressed sequence, which was NEVER COMPRESSED BEFORE. So it does not contains compression artifacts of "previous" codec.

I addressed this already. But I don't have a problem addressing it again, just to further cement the point:
1) I'm doing a real-life encode, something people will be doing in, well, real life - backing up their DVDs and Blu-rays. That's my test, I backed up a DVD and a Blu-ray. Same movie in both cases, but still. People will far more likely do that, than have access to raw video.
2) The bitrate on DVDs and Blu-rays is insanely high for their respective resolutions. So there aren't any particular codec artifacts in the video.

Why it's only fair to compare on uncompressed "master copies"? Imagine you have crappy 128-kbit MP3. [...]

DVDs and Blu-rays *aren't* a crappy 128kbps MP3, they're more like 512kbps MP3 - still lossy, but not significantly so, MP3 reaches transparency way before 512kbps.

So your entire point, while valid, doesn't apply here. You point would apply if I downloaded a rip from the internet and re-encoded that, rather than use the actual DVD and Blu-ray.

Using pre-compressed streams to compare codecs is not fair and usually puts codecs similar to one used in source in advantage.

Let's say that this applies (it doesn't in my test as I've explained above). Well, the Serenity Blu-ray is VC1, so neither x264 nor libvpx had an advantage. Not to mention all codecs in my test are DCT-based with block-based motion compensation, a point I also already made in previous posts, so they're all "similar".

And btw: where to get RAW, uncompressed sequences? Legal and even free of charge? Of course, xiph.org for the rescue - they have plenty uncompressed sequences to chew on. There is catch though: uncompressed movies are HUGE. It would take some time to download. And messing with RAW video data is quite demanding for HDD bandwidth and space. But it's the only fair way to compare various codecs.

People have used those sequences for encoder tests, the "park_joy" clip is particularly popular. You know the conclusions? Basically same as in my test. In fact, "park_joy" is so hardcore, the differences between encoders are even bigger than in my test.


Anyway, if you still think my test is invalid, you, or anyone else for that matter, are welcome to do your own test. Say which clip you used, provide encoder commandlines and encoder ouput, post sample pictures from the encode. A complete test, just like I did. Then, be prepared to defend your choice of clip and your encoder settings, again just like I have had to.