Google Starts Pushing Out VP10 Open-Source Code Into Libvpx

[89c51]

VP whatever# are useless as long as there is no HW acceleration available. The rest don't matter much

[Gusar]

Maybe troll is not the right word. Nay sayers? People who seem to be against VP* for some reason or other?

Facts simply aren't in VPx's favor, what can I say. I don't have a stake in this game, I'm arguing solely from the technical perspective.

What is your conclusion by the way? If my H264 file is 1Gb, what VP9 size do I need for the same quality? 2Gb, 3Gb? This is genuine interest, not trolling - I have seen lots of "better than" comparisons, but no actual metrics for "how much bandwidth do I save/lose for the same quality?".

I don't like to think in those terms. Two reasons for that:

First, it's way too dependent on the video material. There's a big difference whether you're encoding Big Buck Bunny, Terminator 1 (grainy as hell) or Terminator 2 (almost too clean).
I mention the Terminator movies because I encoded just those last week. I always use x264 with its constant rate factor mode, '--crf 20'. Now, you can't directly compare because the T1 picture is 16/9, while T2 is 2.35/1, but it still should give you a perspective - T1 came out at 4.2 Mbit/s, while T2 came out at 2.5 Mbit/s !! Does that mean that VP9 encodes would need to be 5.6 Mbit/s and 3.3 Mbit/s (30% more for both, made-up number just to explain the point, I have no clue what the actual percentage is)? No. Because VP9 deals with grain completely differently from x264, so I say T1 would need a bigger increase than T2.

Second reason, marketing. Every time a new codec is announced, the marketing is full of "XY percent bitrate reduction for same quality!!!!!!1111oneoneeleven!". Well, these claims are always bullshit. *Always*. So I wouldn't like to make same kind of claims that then won't hold true in reality. I trust my eyes. And my eyes tell me, x264 '--crf 20' produces excellent quality at a sensible bitrate. Then I check what bitrate x264 needed to achieve that rate factor and then do 2pass encodes to that bitrate for all encoders in the test, and see how much better (usually worse) they perform compared to x264.

x264 is simply the golden standard other codecs have to beat. Well, except when it comes to Cinepak, that one is awesome all on its own

There isn't much to refute in your testing, other than a minor quibble that BluRays are encoded with H264 to start with

Except when they're not . The Serenity Blu-ray I used in my test is *not* H264, it's VC1. Terminator 2 is, much to my surprise, also VC1.

which will unfairly benefit H264 encoders.

I addressed this is that other thread. To quote myself: "DVDs are high-bitrate (for their resolution) and as such don't exhibit heavy codec artifacts, so you can't claim codec A won over codec B because it could better deal with mpeg2-style artifacts. All encoders in the comparison stand on their merits and their merits alone."
Yeah, I argued DVDs there, but it's even more true for Blu-ray. Blu-rays have insane bitrate for their resolution, so this kind of codec favoritism argument doesn't hold.

[OneTimeShot]

I must confess, when I last encoded in VP9, it was slow, and I was disappointed with the result. I can imagine that x264 is currently the best encoder overall.

Is VP9 at least the best "free codec" available in your tests?

FWIW - I realise how biased I am, but this is a Linux forum

[Gusar]

I must confess, when I last encoded in VP9, it was slow, and I was disappointed with the result.

The result would be acceptable at least, if it didn't take so long to get to the result. Of course these results are far from expected, after all VP9 is supposed to be better than h264.

Is VP9 at least the best "free codec" available in your tests?

Going purely by quality, yes. However, from a quality/performance perspective, I would rank VP8 higher. It's lower quality compared to VP9, but not that much, and it's damn fast. Like really fast.

I haven't tried Thor yet. I haven't done so because it doesn't have proper rate control yet, just constant quantizer mode, and I suspect it's really slow. But what the heck, let's give it a go. I mean, I played with Cinepak over the weekend (behold its power here ), so why not Thor. Will report back how it goes.


Edit: BTW, I'm not sure what relevance it has that this is a Linux forum. Sure, some might favor a royalty-free solution. But on the other hand, x264 is open source, and most of us have a hardware h264 decoder in our machines anyway.

Edit2: Can't get Thor to compile. There's bugs open about it at github, so testing will wait until that's sorted out.

[SystemCrasher]

Why even bother with VP10? Why not just start backing Daala or Thor, considering they're doing the same exact thing as VP10 is (in the same generation) and both are farther ahead.
Is Google just suffering from a need to control everything it uses?

Well, at least Google proven they can make high-quality codec. VP9 can beat crap out of H.264 in terms of bitrate to quality, yet it does not comes with insane licensing terms like H.265. And playable by many browsers around here and now.

On other hand, the only codec from Cisco that appeared to the day is H.264 in Firefox. It really bad quality codec. It is slow like a hell, losing about 2 times to gstreamer codec in Ubuntu. FAIL! To make things even worse, if it can't keep up with realtime, it unable to drop frames to catch up in sane ways, like other sane codecs do (gstreamer, ffmpeg, etc). So if your system got just 95% of CPU power needed to play certain scene in real time, Cisco's H.264 in Firefox would give you a horrible slide show, literally measured in some frames per minute, not frames per second. On other hand, ffmpeg or gstreamer would just drop about 5% of frames or reduce postprocessing and you'll have hard time even noticing all this time-saving magic at all. Maybe slightly more jumpy picture in few intense scenes, but you only know about it if you monitor CPU load and know what to look for. Not to mention that due to much better code optimization it would happen much less frequently, to begin with.

So does someone thinks Cisco can create decent codec? Are you guys from Cisco's marketing dep't, don't you? Get lost with your marketing bullshit. Nobody needs just replacement of Google with Cisco. And as for standards, H.264 is "standard". Yet its proven to be patent-encumbered and developed behind closed doors. So it is not anyhow better than VP9. It actually worse: licensing terms of H.26x are awful, you do not get free hardware decoder, you do not get decent reference implementation, but you're asked to pay royalties. For some abstract specs. LOL, it sucks and it's very nice there is Google who is going to change that.

As for Daala - it looks interesting since it uses really different technique to encode videos compared to existing codecs. It can prove to be powerful approach, possibly beating others. But it would take YEARS to make it practical, where it can compete other codecs and handle most use cases. On other hand VP9 is ready. It works here and now. It gives nice bitrate to quality ratio. Thor on other hand neither beats VP9 in bitrate to quality ratio, nor it inclined on new approaches like Daala. So what's the point about Thor? Jsut to try to replace Google with Cisco? Cisco isn't opensource friendly company at all. Google is much better in this regard. And mumbling about standards looks lame: there're H.264 and H.265 - they're kinda "standard". Yet they have awful licensing terms. So, go to hell, IEEE. If you're only going to pad business interests of few corps, who the hell needs of you except these few corps?! Competitoin should appear. That's what Google did. Something to think on, eh?

[renkin]

For those of you bashing VP9 quality, have a look at the low bitrate tests on Doom9. A guy by the handle Zerowalker decided to encode some game footage at low bitrate. It was followed up by a more comprehensive test by BadFrame

€‹(zerowalker test)
€‹(BadFrame test)

?€‹Sadly, BadFrame used imageshack to host his findings. But you can tell by context that VP9 won that fight. And this was not even a month since freezing the bitstream. The encoder was not optimized.

[profoundWHALE]

Here's an up-to-date test done on that same forum

[liam]

Checkout mozilla's arewecompressedyet.com
In particular https://arewecompressedyet.com/?x264...-08-18-7927316

At low bitrates x265 is the best, x264 the worst, with daala in between. VP9 doesn't even exist below about 0.015bpp, but at that level, and above, it peforms at least as well as x265 until about 0.03bpp where it starts to level off and slip beneath x265. All of the codecs have their closest point of approach to one another at about 0.05bpp, and after that things get a bit weird.
At about 0.09bpp daala and x264 start performing better than x265, and at about 0.012 daala starts just pulling away from the others altogether.
BTW, I'm mostly concerned with fastssim. IIRC, in the other testing domains vp9 peforms nearly identically to x265.

[Gusar]

For those of you bashing VP9 quality, have a look at the low bitrate tests on Doom9. A guy by the handle Zerowalker decided to encode some game footage at low bitrate. It was followed up by a more comprehensive test by BadFrame

??(zerowalker test)
??(BadFrame test)

?Sadly, BadFrame used imageshack to host his findings. But you can tell by context that VP9 won that fight. And this was not even a month since freezing the bitstream. The encoder was not optimized.

Less than 1000k for FullHD isn't "low bitrate", it's nonsensically uberlow bitrate. It's unfortunate that the pics are gone, but I have a very good idea how they looked like - horrible, both x264 and libvpx. No one will actually do real-life encodes like that. Even Youtube (whose bitrates are generally too low) uses more than that for 720p, let alone for FullHD. And there's a comment saying that the above-1000k libvpx encodes look "polished", so you can hardly say that it "won that fight".

What I did here is "low bitrate", 400k for SD and 800k for 720p. Neither x264 nor libvpx does well at such bitrates. libvpx creates mush, x264 is sharper but distorts (this could be tweaked by lowering psy-rd, and it's quite possible the distortion isn't as jarring during motion).

Checkout mozilla's arewecompressedyet.com

There's just one problem with arewecompressedyet.com ... Metrics do not correlate to visual quality. They use FastSSIM, which is the "best" at doing so, but even FastSSIM does *not* tell you how the picture will look like. The x264 '--tune ssim' option disables all psy optimizations! When it's exactly psy optimizations that give x264 the edge (that, and rate control).

Then there's the too often used PSNR metric. That one is completely off! The x264 '--tune psnr' option disables not only psy optimizations but also adaptive quantization.

As an example, this is theora-1.1.1 which optimizes for PSNR: http://www.imagebam.com/image/4ad797430028907]
And this is theora-git which optimizes from SSIM: http://www.imagebam.com/image/699072430028923]

(hmm, the first pic appears too dark in Firefox. download both pics and look at them in an image viewer)

The first pic is total mush, the second one is at least passable. But even optimizing for SSIM does not optimize for visual quality! That's why I said in my previous post, I trust my eyes. Do not look at graphs to make conclusions about encoders. You won't be watching graphs, you'll be watching videos.

[SystemCrasher]

At low bitrates x265 is the best, x264 the worst, with daala in between. VP9 doesn't even exist below about 0.015bpp, but at that level, and above, it peforms at least as well as x265 until about 0.03bpp where it starts to level off and slip beneath x265.

I guess Google targets more or less realistic use cases like internet video hosting, and puts a lot of effort optimizing it. Which is logical for codec targeted to browsers, to say the least. Very low bitrates are inevitably doomed to have poor image quality, especially at intense scenes, and nobody in sane mind targets them. So whole "bits per pixel" metric is kinda synthetic. I.e. where it takes into account, say, motion compensation efficiency of certain codec? Does it, at all? Then, very high bit rates would be no use in Internet either. So I guess Google puts less efforts in optimizing cases. Looking on commit messages I can see they have test cases they're measuring here and there and it mostly resembles how transcoding done on video hosings. Which is probably main use case for network-oriented codecs anyway.

So, in practice, VP9 performs very well if you target some common Internet usage profiles. Say, 480P at just 500kbps average? No prob, it would do. And you'll have hard time finding compression artifacts (there're some, but it would take quite specific scenes to actually see them). Not bad for just 50KBytes a second, isn't it? And even crappiest ADSL would work fine for such bitrate. As well as 3G, etc.

At about 0.09bpp daala and x264 start performing better than x265, and at about 0.012 daala starts just pulling away from the others altogether.
BTW, I'm mostly concerned with fastssim. IIRC, in the other testing domains vp9 peforms nearly identically to x265.

There is some issue: these metrics are purely synthetic. They do not measure how much certain compression technique artifacts are annoying for users.

For example, on low bitrates (e.g. 480P@500kbps target), x264 would make small text nearly unreadable, x265 would do the same, though it would look a bit better overall. But it would still look like a crap when compared to VP9. So while formal metrics could be comparable, one would have issue reading small text compressed by x26x as long as it targets low bitrates. So, does anyone thinks it's cool to watch movie and then .... get crappy, unreadable credits text?