Some Additional Chrome vs. Firefox Benchmarks With WebRender, 67 Beta / 68 Alpha

[kozman]

If you are such a big fan stop being ignorant. LLVM is faster than gcc or VS, in many cases with FireFox as it enables optimizations that the other two can't do yet. LLVM also has superior MSVC compatiblity compared to GCC. See here for explination https://glandium.org/blog/?p=3888

Also WebRender isn't fully utilized under Linux yet, as using the GPU for acceleration isn't enabled by default see here https://bugzilla.mozilla.org/show_bug.cgi?id=1491303 , But the fact is that FireFox has not had acceleration on Linux in eons, so the fact that it is being worked on now is encouraging.

I'm not a programmer, but may of the benchmarks I've seen done on Phoronix over the years comparing GCC to LLV/Clang don't bear out what you say. Sometimes GCC trounces Clang in a *specific* bench, sometimes the other way around. They each seem to have their own + and -. The folks in the GCC camp, I am told, are rather stubborn in accepting contributions to the compiler which may greatly increase perf but will slow compilation (or some other related function) as a byproduct. I don't know how LLVM land is in terms of that.

As a regular bug reporter to Mozilla, yes, I have been following the Webrenderer stuff and it is good that Linux is getting work in that area done but it is lacking compared to Windows-land due to limitations in Linux. Just have to give it some time to mature.

[treba]

Also WebRender isn't fully utilized under Linux yet, as using the GPU for acceleration isn't enabled by default

It is. If you were to run Webrender on software it would be terribly slow. The old 'hardware acceleration' via 'layers.acceleration.force-enabled' is replaced by Webrender, which is why you don't need to enable both to have Webrender running. Overall, we now have 'Basic', 'OpenGL' and 'Webrender' options.

[cb88]

Man, this is not good. I'm a big fan of Mozilla but, man.....<sigh>. I would like to see results on Windows but I would like to point you guys to http://www.speed-battle.com. Look at the 2019 results for Windows 7 6.1. See all those test results between 64, 65 and 66 in the many thousands? That's me running a simple non-stessing bench test on an old Optiplex 7020. 66 has actually *dropped* in performance compared to 65. I don't know how much Rust is helping FF's cause but WTF were they thinking when they switched to LLVM from VS? It's just a turd. I mean, they *just* switched to VS a few years ago from GCC. Pick a friggin' compiler and stick with it!

FF 66 was build against an early LLVM 8.0 RC (I think) but seeing as how 8.0 final was just recently released after 66 went final, I doubt it would make any more perf improvement. I agree FF doesn't get as much love as in Windows but it has made a lot of strides since 65 but I sometimes feel like they're just chasing shadows or something.

If you are such a big fan stop being ignorant. LLVM is faster than gcc or VS, in many cases with FireFox as it enables optimizations that the other two can't do yet. LLVM also has superior MSVC compatiblity compared to GCC. See here for explination https://glandium.org/blog/?p=3888

Also WebRender isn't fully utilized under Linux yet, as using the GPU for acceleration isn't enabled by default see here https://bugzilla.mozilla.org/show_bug.cgi?id=1491303 , But the fact is that FireFox has not had acceleration on Linux in eons, so the fact that it is being worked on now is encouraging.

[Cape]

Let's wait and see what excuses Google haters will come up with this time around...

From a technical standpoint, chrome is a wonderful piece of engineering. And it's got plenty room for improvement as well.

And you bet it does! It's the most used browser on the planet by far, developed from the most rich and successful company of the modern era, who has 98,000 - well payed employee in CA alone!

I wouldn't expect nothing less from this gigantic pile of financial/political admixture!

[g3wcm2V8uqwR]

I haven't checked if it's applicable here but the actual rendering (and not js-execution) bound benchmarks could suffer from the same issue Firefox has with WebGL on Linux: copying back every frame before compositing.

Are you referring to the fact that Chromium gets more FPS because it skips (rendering) frames when it detects latency? as per https://bugzilla.mozilla.org/show_bug.cgi?id=611596#c26

[juno]

I haven't checked if it's applicable here but the actual rendering (and not js-execution) bound benchmarks could suffer from the same issue Firefox has with WebGL on Linux: copying back every frame before compositing.

[Michael]

Michael

Honestly I've never heard of this benchmark.

Why haven't you checked the performance in JetStream (ex SunSpider), Octane, and Kraken? What about the Windows versions of these web browsers?

I've also found these tests:

http://www.kevs3d.co.uk/dev/canvasmark/ (for me Chrome is twice as fast as Firefox in this test).
https://testdrive-archive.azurewebsi...ance/fishbowl/ (test with 2000 fish)
https://testdrive-archive.azurewebsi...ce/chalkboard/ (this one is painfully slow in Firefox)

https://web.basemark.com/

http://webvizbench.com/Default.htm?skipdetect (doesn't properly work on Firefox for Linux - no video, still Chrome is thrice as fast)

Basemark is included as are others. Are you referring to 'Selenium'? It's simply a WebDriver for allowing the automation of the browser tests.. As you can see from the graphs, Basemark, Octane, and others are what is being tested.

Jetstream now added, the URL wasn't working a few days ago. https://openbenchmarking.org/test/system/selenium

And for those not familiar with Selenium: https://www.seleniumhq.org/ after learning its interfaces, liking it quite a lot for automating all the browser benchmarks from Python.

[kozman]

Man, this is not good. I'm a big fan of Mozilla but, man.....<sigh>. I would like to see results on Windows but I would like to point you guys to http://www.speed-battle.com. Look at the 2019 results for Windows 7 6.1. See all those test results between 64, 65 and 66 in the many thousands? That's me running a simple non-stessing bench test on an old Optiplex 7020. 66 has actually *dropped* in performance compared to 65. I don't know how much Rust is helping FF's cause but WTF were they thinking when they switched to LLVM from VS? It's just a turd. I mean, they *just* switched to VS a few years ago from GCC. Pick a friggin' compiler and stick with it!

FF 66 was build against an early LLVM 8.0 RC (I think) but seeing as how 8.0 final was just recently released after 66 went final, I doubt it would make any more perf improvement. I agree FF doesn't get as much love as in Windows but it has made a lot of strides since 65 but I sometimes feel like they're just chasing shadows or something.

[Michael_S]

We need test results for Firefox on Windows with NVIDIA graphics to determine the actualperformance impact of WebRender, since this is the configuration Mozilla develops WebRender for.

There must be other Windows-specific engineering work done by Mozilla for Firefox. These benchmark numbers are a few months out of date, but on my hardware the MotionMark (Animation testing) benchmark on Chrome had a more than 400% advantage on Linux regardless of Webrender. On Windows on the same benchmark, Chrome had only (for some odd definition of 'only') a 70% advantage.

I don't have Nvidia graphics.

On the other hand, the Speedometer benchmark probably comes closer to a real world comparison of speed that a end user would notice. On that one Chrome has a 20% advantage on Windows but Firefox had a 10% lead on Linux.

[bug77]

Ouch.

Having started with Mosaic, and fighting the advance of Netscape Navigator and beyond, it's disappointing to see how far behind the major open source browser challenger is.

And sadly it's not just performance, it's the gigantic mistake Mozilla made when they dropped support for most all existing plugins. It was an incredible strategic error, and to compound it the reasoning was so Firefox could be just as fast or faster than Chrome.

So Firefox would still have had a huge problem even if the performance had become competitive, because many people dropped Firefox the moment their plugins didn't work, and wouldn't have been there to see it anyway.

I've had Netscape and later Seamonkey as my main browser till Seamonkey gave up the ghost. Now I'm using Firefox by default. I'm stubborn enough to use it on my phone and tablet where it's painfully slow (I believe not even OMTC is enabled on Android).
And yet, Firefox is dead. Not because of what it is, but because newcomers don't even bother to give this browser a chance. They almost always go for the fastest browser which has been Chrome for what is seems like an eternity now. And in the odd chance someone values functionality more, Vivaldi and others still have Firefox beat.
I kinda like Rust, but I haven't done anything worth noting with it yet.