Google Starts Pushing Out VP10 Open-Source Code Into Libvpx

[Gusar]

VP9 can beat crap out of H.264 in terms of bitrate to quality

Sorry, but no, it can't. Look at my encoder test. Unless you do a test yourself like I did, provide encoder commandlines and sample pics, and this test shows the opposite of mine, your claim has no merit.

On other hand, the only codec from Cisco that appeared to the day is H.264 in Firefox. It really bad quality codec.

It is a baseline profile only codec intended for use with WebRTC. In fact, that's the only place it's used! It is not used for HTML5 video (because that's usually main or high profile), it's not meant to be used for that. It's only purpose is to provide interoperability for communications.

So does someone thinks Cisco can create decent codec? Are you guys from Cisco's marketing dep't, don't you? Get lost with your marketing bullshit.

I have encoder tests, you have ad-hominems. You really think that's the proper way to make an argument?

And as for standards, H.264 is "standard". Yet its proven to be patent-encumbered and developed behind closed doors.

Developed behind closed doors? Quite the opposite. Michael Niedermayer put it best at his blog:

About open, ITU H.264 was developed on a public mailing list namely JVT-experts, using public FTP containing software, spec drafts, proposed changes, test results, meeting protocols and god knows what else. Anyone could read in near realtime what was discussed, proposed,changed and why and any expert that wanted to contribute iam pretty darn sure would have been taken serious. google webm is On2 last video codec, that to the best of my knowledge was developed behind closed doors by On2 and On2 people where on the jvt-expert list.

So yeah, VPx are the ones developed behind closed doors, first at On2, now at Google. The only difference is, Google has a public repository of their code. But it's still a wholly internal project. Whereas the ITU standards have open participation, as Michael so nicely described.

Thor on other hand neither beats VP9 in bitrate to quality ratio, nor it inclined on new approaches like Daala.

Did you actually test Thor? I haven't yet because it wouldn't compile, but there's a Windows binary, so I'll boot into Win and test it there. But anyway, unless you have tested it, you have no way of knowing how it performs against VP9.

That said, I don't expect much from Thor right now, but that's because it's not optimized at all yet, it's far from a finished product (no rate control yet, probably no visual optimizations of any kind). It's just a pure tech demonstration right now, not something to be used by end users. While VP9 is or at least should be optimized by now and is meant for end users. Thor does have tech available that VP9 doesn't though, like b-frames and multiple reference frames, so tech-wise, it quite likely has the edge over VP9.

So what's the point about Thor? Jsut to try to replace Google with Cisco?

No, the point of Thor is to create the next *actually* open standard, by working together with Mozilla and Xiph (and anyone else who wishes to join, including Google!) as part of the NetVC working group at IETF. NetVC will not be Thor, but it will be something that contains Thor tech, in addition to Daala tech.

Seems to me you have some sort of agenda against Cisco, instead of being interested in properly discussing codec tech.

[Gusar]

To further highlight why you should not look at metric graphs, I did three x264 encodes, identical settings except for the tuning. Metric results:

--tune psnr

x264 [info]: SSIM Mean Y:0.9854481 (18.371db)
x264 [info]: PSNR Mean Y:45.846 U:47.758 V:48.598 Avg:46.398 Global:46.113

--tune ssim

x264 [info]: SSIM Mean Y:0.9865392 (18.709db)
x264 [info]: PSNR Mean Y:45.717 U:47.443 V:48.194 Avg:46.213 Global:45.781

--tune film

x264 [info]: SSIM Mean Y:0.9834771 (17.819db)
x264 [info]: PSNR Mean Y:44.661 U:47.620 V:48.366 Avg:45.438 Global:45.052

This clearly shows that tuning for a metric will give the highest score at that metric, while tuning for visual quality will give really low scores. Now, when we look at visual quality...

--tune psnr: http://www.imagebam.com/image/ed9630430055610
--tune ssim: http://www.imagebam.com/image/4befb8430055614
--tune film: http://www.imagebam.com/image/446a99430055627

The whole gallery with another set of pics: http://www.imagebam.com/gallery/rmpx...htdqy1qkso45pw

Both metric-tuned encodes look like crap (it's even worse in motion), only the film tuning produces something watchable. That's the power of x264's psychovisual optimizations at work.


@SystemCrasher: Can you give example encodes that exhibit the behavior with text you describe? Because from my experience, it'll be exactly the opposite - vp9 will blur out the text to the point of not being readable, while x264 will preserve it well. Unless x264 would distort the text too much because of lack of bitrate.

Besides, credits are usually big text, not small. And it needs very little bitrate to encode. For my own encodes, I use a zone for credits and instruct the encoder to significantly lower bitrate in the zone, so that credits will be at about 300kbps, while the movie itself will be 2500-4000kbps depending on video complexity. The credits still look perfectly fine at 300kbps.

Also, 500 kbit/s for 480p video is at the lowest edge of sensible. Yeah Youtube's 480p is 500-1000kbps, but Youtube uses, as I said, generally too low bitrates. Most Youtube videos are pretty bad, regardless of codec. Perhaps that's why some are ok with VP9, they prefer its blurriness to x264's distortion. In other words, they prefer one kind of mush to another kind of mush. But it's mush, either way.

[liam]

The problem with any system other than metrics like psnr/ssim/fastssim/etc is, obviously, repeatability and measurement. As some have said, the problem with such metrics is that they can give a false impression of a codecs' performance...which is why tests include many, many different types of scenes (in addition to highlighting weakpoints in a codec, obviously).
IOW, it's the worst system except for anything else

[Gusar]

IOW, it's the worst system except for anything else

That's not true. It's a terrible system, period. And there *are* better ones - your eyes!

For example, this paper about x264's macroblock-tree rate control describes that a strength factor was arbitrarily derived from experimentation. And that's not the only such "magic" constant in the x264 source. These constants were derived not from looking at metric graphs, but from... looking at actual videos!

It's why the VPx family of codecs sucked completely when they were still closed source codecs made by On2. Because the On2 folks did look exclusively at metric graphs. Apparently there was a divide at On2 between the PSNR group and the visual quality group, and the PSNR group won.

Even now VP8/9 lack psychovisual optimizations, Google doesn't spend nearly enough time in this area. There are some beginnings of adaptive quantization in VP9, but no work has been done since the initial commits, and Google's own guide instructs to turn it off.

Compare that to x264, where a lot of time was spent on AQ, including overhauling it completely - there's a huge "VAQ 2.0 alpha testing" thread at doom9.org where users were testing various methods on their videos to figure out what works best. And that is why x264 kicks so much ass.

Even x265 can't reach it yet, even though it does have cutree rate control (there's no macroblocks in H265, but there are "coding units") and adaptive quantization and several options to tweak psychovisual enhancements. However, there's a "Comparisons of x265 and x264" as well as a "Suggestion for x265's --tune film" thread at doom9.org, where people are trying to work out these many psy options and figure out how to tune the other settings. Metrics aren't involved at all in these two threads, just people using their eyes.

[liam]

You missed my point. The reason why metrics like fastssim/psnr/whatever are better than metrics that rely on a person is repeatability. Yes, they don't work in all circumstances, as you've shown with your clip, but thats why you use many, many clip types. On average you can get an idea of how they perform, especially relative to itself over time.
By relying on your eyes you get far too much disagreement, and no real hope of repeatability and measurement which are really important when developing a codec. Now, I'm certainly not saying that eye-based/subjective metrics aren't important.

[Gusar]

On average you can get an idea of how they perform

No, you don't. The metrics will screw up on *every* clip. That is the point. Don't believe me? Pick a clip, do three encodes just like I did, same settings except for the tune parameter.

By relying on your eyes you get far too much disagreement, and no real hope of repeatability and measurement which are really important when developing a codec.

The quality of x264 proves you wrong. x264 is so good exactly because the devs *did* rely on their eyes and the eyes of the testers at doom9.org.

It's because of thinking like yours that VPx isn't competitive, Google is still too hung up on metrics. There is no "--tune=film" in libvpx, just "--tune=[psnr|ssim]"