Firefox Is Seeing Work On Wayland VA-API Video Acceleration

[Guest]

Thats not the whole answer. Sure Mozilla gets most of it's money currently from google, but they're trying to generate other income, by for example starting to offer kind of VPN services. And Google and Mozilla aren't controlled by the same people, so they might cut their relation when they're in need. So Google might partially control Mozilla currently, but not to absolute degree.

And Google has full control over the video codec used by Youtube.

[Hibbelharry]

And Google has full control over the video codec used by Youtube.

Context?

Sure, googles does have control over the video codec, since youtube is a google company. But Youtube is switching to mostly open codecs and royalty free codecs like AV1, so whats the matter here?

Mostly no hardware accelerates AV1 today, but many other codecs started just the same, it took a while, and then there was decoding acceleration in hardware. There is no reason that acceleration can't be offered via VA-API, so everything is going into the right direction currently.

[Hibbelharry]

You know that browser got sold to a chinese search engine or something right? So they're not going to be much different than Google, especially with using Blink as the engine under the hood.

It's even worse. It's that company that already killed startcom (starttls) and other companies and services. Nowadays opera is mostly financed by offering predatory loans. Opera isn't a browser company anymore, it's a loan company owning a browser as a sidekick. And on a global scale, they're loosing browser marketshare rapidly.

[utopik]

Chromium should take notice, although some distributions ship their build with VA-API patches already. Hardware video acceleration is a big deal, especially on laptops.

Event with this patch, va-api acceleration can't work on wayland unfortunately.

[andre30correia]

Wayland lets you avoid copying pixels around and works on the concept of shared buffers. Firefox can set the hardware decoder to use a shared buffer that's also shared with the compositor.

This isn't really possible under X, Firefox would have to start copying pixels everywhere which is terrible for performance and battery life reasons.

and this works with chromium... same old story, Ubuntu don't go with Wayland because nvidia non free drivers and gaming but someone remembers to put work in something only for wayland

[CommunityMember]

Thats not the whole answer. Sure Mozilla gets most of it's money currently from google, but they're trying to generate other income, by for example starting to offer kind of VPN services. And Google and Mozilla aren't controlled by the same people, so they might cut their relation when they're in need. So Google might partially control Mozilla currently, but not to absolute degree.

As always, it is complicated. Mozilla's income (from Google) depends on Google receiving ad revenue from Firefox searches. Mozilla has to walk a fine line between maintaining the ad revenue for Google in order to maintain its revenue stream. If Mozilla goes too far Google ad revenue from Firefox would likely drop, and then Mozilla income would drop, and then Mozilla would need to fire even more staff to survive (perhaps stop paying their CEO $2M/yr?). So, finding alternative income streams for Mozilla is necessary, but creating services to replace $400M/yr is not going to be easy, and so Mozilla will continue to make compromises in order to not be an ex-Mozilla.

[Apokalypz]

I have a few related questions and observations:

1. Without DMA-buf, it was said that there's no way to implement vaapi? I thought the xserver used Xv as a way to target a window with a video (and scale/convert colorpace as needed). It's not zero copy, but it still works and has worked for years (despite needing to copy every frame in ram to it's destination). Though its useful for efficiency, I don't see how DMA-buf is |absolutely| necessary for hardware video accel.

2. I don't think Mozilla's experimental servo project is supposed to be "the next Firefox" they use it as a test bed for new features and if those features work (like webrender, quantum, etc) it gets ported to Firefox and further maintained in the Firefox tree. At least, that's my understanding.

3. I personally don't like Google's chrome being the only other browser (and the browser that most users use.) It gives Google way too much power over the future of the internet. But at the same time, if you look at websites as using an API that your browser implements to show websites or perform cloud services, then it kinda makes sense to use software from the same company that has the most influence in what this API looks like. If that makes any sense. Plus, you can always fork chromium and make any changes you want. Personally, I usually run the underdog (yes, I have a history of using MSN.com...then bing.com for my searches... some very dark days those were. But I got help I needed and was shown the way of the Google.) I simply use Firefox because I don't like Google's hands being in everything from the browser to the server (in some cases). That felt like more of a rant than a comment or question, so I'll leave it here.

[timrichardson]

A mainstream browser with upstreamed wayland and hardware decode is a big wayland milestone. Chromium has good decode support at least for intel (I've been using it for more than a year without a single problem) and wayland support is apparently not too far away, but Chromium it seems will never upstream the linux hardware decoding.

I thought firefox was a long way from hardware decode because I thought it depended on webrender migration being finished. I hope it works with kwin.

[Guest]

Context?

Sure, googles does have control over the video codec, since youtube is a google company. But Youtube is switching to mostly open codecs and royalty free codecs like AV1, so whats the matter here?

Mostly no hardware accelerates AV1 today, but many other codecs started just the same, it took a while, and then there was decoding acceleration in hardware. There is no reason that acceleration can't be offered via VA-API, so everything is going into the right direction currently.

The only direction we all can take. PERIOD

[Gusar]

1. Without DMA-buf, it was said that there's no way to implement vaapi? I thought the xserver used Xv as a way to target a window with a video (and scale/convert colorpace as needed). It's not zero copy, but it still works and has worked for years (despite needing to copy every frame in ram to it's destination). Though its useful for efficiency, I don't see how DMA-buf is |absolutely| necessary for hardware video accel.

Xv is an ancient interface for hardware that hasn't existed for a long time now. On all remotely modern GPUs, Xv is emulated using shaders. Xv has several shortcomings. One, it only handles rendering, not decoding. Two, it's impossible to integrate Xv with anything, it's meant to display the video directly on the screen, so it's incompatible with compositors and it's not possible to have Xv video as part of a webpage. Three, everything that you want to display on top of the video (like subtitles or the player interface) needs to be rendered in video resolution, blended with the video in software and then it'll get scaled by Xv, which looks horrible. There's a few more things, like I don't think Xv can handle 10bit video. Bottom line, no one should be using Xv nowadays, not even on crappy Intel hardware like GMA950.

VAAPI is absolutely the way to go. You don't strictly need dma-buf for it, but then you're limited to displaying the video directly on the screen. If you want to integrate the video into a webpage, you need to import the hardware decoded surface into OpenGL or Vulkan. Heck, even video players like mpv and VLC use this approach, because it gives you much better control over how the video will be rendered (mpv has the direct display option as well, but there's no benefit to use it). And that's where dma-buf comes into play, it allows zero-copy transfer from the decoder hardware to an OpenGL or Vulkan context. Then, Firefox has been working on Webrender, a new way to render webpages using OpenGL. So it all comes together - you have the hardware decoded video in an OpenGL context, the webpage is in an OpenGL context, so blending them together isn't hard anymore, and it all happens in hardware.

It's nice to see work being done to bring hardware decode to Firefox. I just hope it won't stay Wayland-only. There's absolutely no reason for that, the same vaExportSurfaceHandle+EGLImage approach works just as well in X as it does in Wayland.