Some Additional Chrome vs. Firefox Benchmarks With WebRender, 67 Beta / 68 Alpha

[ssokolow]

And sadly it's not just performance, it's the gigantic mistake Mozilla made when they dropped support for most all existing plugins. It was an incredible strategic error, and to compound it the reasoning was so Firefox could be just as fast or faster than Chrome.

No, it was because having all their internal implementation details exposed as "plugin API" was hamstringing their ability to refactor the codebase in any way.

The legacy API's greatest strength and weakness was that it worked by allowing extensions to monkey-patch just about anything at runtime.

[muncrief]

Ouch.

Having started with Mosaic, and fighting the advance of Netscape Navigator and beyond, it's disappointing to see how far behind the major open source browser challenger is.

And sadly it's not just performance, it's the gigantic mistake Mozilla made when they dropped support for most all existing plugins. It was an incredible strategic error, and to compound it the reasoning was so Firefox could be just as fast or faster than Chrome.

So Firefox would still have had a huge problem even if the performance had become competitive, because many people dropped Firefox the moment their plugins didn't work, and wouldn't have been there to see it anyway.

[Anvil]

It'd be interesting to see how these two compare on windows and if the benchmarks are similar there. FF doesn't seem to care much about linux.

true that, more users on Windows than Linux so Firefox Devs tend not to care to much about it, But WR isnt as wonderful as it was made out to be.

[xiando]

Michael, I propose doing further browser test on a more equal ground.

You can go to the special url chrome://gpu in chrome and chromium and view what GPU acceleration, if any, is enabled. The default appears to be to have most disabled. The special url chrome//flags allows you to change "override software rendering list" and "gpu rasterization", "out of process rasterization" and "zero-copy rasterizer" and "vis display compositor" to enabled.

In firefox you go to about:config and you can set layers.acceleration.force-enabled and layers.omtp.enabled and gfx.webrender.enabled and gfx.canvas.azure.accelerated to true (I believe you have to create the last one as a new bool). Then you can go to about:support and see that you have opengl compositing, hw_compositing and webrenderer.

My humble opinion is that it's "fair" to either test both Firefox and Chrome with a blank profile and no settings added (the default experience) OR test change settings to performance-tune both. That way we get an apples to apples comparison.

On a totally different but related note: I would also love to see actual GPU RAM values for the tests and a test or two which just aims to measure GPU RAM use (open a few tabs of heavily intensive JS which draws stuff). radeontop is useful on AMD GPUs. The reason I bring this up is that modern web browsers GPU RAM use is insane, specially and very noticeably so in Firefox. I sometimes see 5-6 GB of GPU RAM use by browsers. That's just crazy. Also, the leaks are a bit crazy. Leave some site which pulls a websocket and renders the results as a graph or something like that open for a few hours and Firefox can easily go from say 2 to 8 GB of GPU RAM and then it runs out of GPU RAM and crashes. This is a problem with both Chromium and Firefox and it's much worse in Firefox - which also happens to be way slower in just about everything.

[smitty3268]

It seems WebRender is an almost irrelevant PR project from Mozilla, otherwise where are the benchmarks proving otherwise? Not only doesn't it outperform Chrome, it's mostly far behind.

WebRender is about speeding up RENDER performance, and a lot of these tests aren't even rendering anything.

In fact, that inspired me to look up what they are actually testing:

ARES-6 - javascript only.

Octane - javascript only.

WebXPRT - "six HTML5- and JavaScript-based scenarios created to mirror the tasks you do every day: Photo Enhancement, Organize Album Using AI, Stock Option Pricing, Encrypt Notes and OCR Scan, Sales Graphs, and Online Homework" - probably a more balanced/real world test than most of the others? Looks like it's doing some DOM, JS, and CSS/drawing stuff.

Basemark - WebGL (which i don't think WebRender would be expected to change)

Motionmark - Animation test stressing CSS, SVG, canvas drawing, etc. This is one I'd expect a big boost from WebRender, and indeed the tests show there is a decent sized boost.

Speedometer - tests the speed of simulated user interactions through frameworks such as react, jquery, etc. as well as direct js DOM calls. As a test of DOM, JS, and perhaps a small amount of css, i'd expect this test to be more balanced and real world than many of the others.

[Venemo]

Michael Would be interesting to see a couple more free / open source browsers lined up in the same benchmark. What come to mind are Epiphany (aka. Gnome Web), Falkon, Midori, rekonq... maybe there are more. But would be interesting to see where those alternatives stand in the performance game.

[amehaye]

Is there a build of Firefox with V8?

It would be interesting to run the benchmarks against such a build, so that any JavaScript performance difference can be accounted for.

[treba]

Well many of the benchmarks are AFAIK mostly about JS then about rendering speed, so no differences where expected there, just sad to see MozJS/SpiderMonkey being beaten so hard
Concerning Webrender, it's nice to see the MotionMark getting much faster, especially considering the big difference between 67 and 68 (which could still improve until it lands)!

[Hibbelharry]

It seems WebRender is an almost irrelevant PR project from Mozilla, otherwise where are the benchmarks proving otherwise? Not only doesn't it outperform Chrome, it's mostly far behind.

I think the major goal of WebRender up to now is still to produce correct results. When they generally get correct results things will be optimised and accelerated. It's still to early to judge.

[equeim]

It seems WebRender is an almost irrelevant PR project from Mozilla, otherwise where are the benchmarks proving otherwise? Not only doesn't it outperform Chrome, it's mostly far behind.

We need test results for Firefox on Windows with NVIDIA graphics to determine the actualperformance impact of WebRender, since this is the configuration Mozilla develops WebRender for.