Benchmarks Show Firefox 57 Quantum Doing Well, But Chrome Largely Winning

[pq1930562]

Michael Also, why did you test with kernel 4.10 and not with kernel 4.13? And why did you test with Mesa 17.0 and not with Mesa 17.2?

You are praising performance improvements with latest kernel and Mesa versions but then you don't use them when testing and also use an old Chrome version?

No offense, but this is a bit confusing.

[pq1930562]

EDIT: it looks like FF56 (last version to support traditional FF extensions) will also be an ESR release, so security updates should be available for it.

Source?

[pq1930562]

According to this CSS Benchmark, Chrome is basically no competition at all for Firefox 57:

Firefox (6.5 seconds): https://streamable.com/bbrwf
Chrome (16 seconds): https://streamable.com/9n7ds

Wrong:

Firefox 55.0.2 (64-bit): 11 seconds
Firefox 58.0a1 2017-09-28 (64-bit): 11 seconds
Chromium 61.0.3163.79 (64-bit): 4.5 seconds



Tested on Core i7-6700K using integrated graphics running KDE neon (based on Ubuntu 16.04.3 with kernel 4.10 and Mesa 17.0.7 and xf86-video-intel 2.99.917+git20170309).

[leipero]

Are you talking to the right person? I just showed firefox did worse because despite being faster than chrome a few times, it was slower almost every other time by fairly large margins.

No, we don't need to disagree to have replies, I was just continuing what you wrote.

pq1930562 That test (CSS) is way too random... to be useful at all, on Chromium it goes anywhere from few seconds, to 1,5 seconds on 2nd, 3rd etc. run. It is consistent on Firefox tho, and yeah, slower.

[pq1930562]

pq1930562 That test (CSS) is way too random... to be useful at all, on Chromium it goes anywhere from few seconds, to 1,5 seconds on 2nd, 3rd etc. run. It is consistent on Firefox tho, and yeah, slower.

I've noticed that as well, but it's not quite correct what you're saying:

1. Consistence is the same between Firefox and Chrome/Chromium. Firefox is not more consistent than Chrome/Chromium.
2. The inconsistency is not triggered by subsequent runs, instead it is triggered by the "Generate a new random maze" button. As long as you do not press this button, it is largely consistent. And you can always refresh the page (F5) to go back to a good state.

[mmstick]

Wrong:

Firefox 55.0.2 (64-bit): 11 seconds
Firefox 58.0a1 2017-09-28 (64-bit): 11 seconds
Chromium 61.0.3163.79 (64-bit): 4.5 seconds



Tested on Core i7-6700K using integrated graphics running KDE neon (based on Ubuntu 16.04.3 with kernel 4.10 and Mesa 17.0.7 and xf86-video-intel 2.99.917+git20170309).

Have you tried enabling WebRender? https://mozillagfx.wordpress.com/201...-newsletter-5/

[pq1930562]

Have you tried enabling WebRender? https://mozillagfx.wordpress.com/201...-newsletter-5/

Thanks for the link. No, didn't have it enabled. I enabled WebRender now for Firefox 58 (55 is only running with layers.acceleration.force-enabled=true):

Firefox 55.0.2 (64-bit): 11 seconds
Firefox 58.0a1 2017-09-30 (64-bit): 5.4 seconds
Chromium 61.0.3163.79 (64-bit): 4.5 seconds

Still Chromium is about 1 second faster .

Anyway, am I correct in assuming that Michael also did not have WebRender enabled?

So, Michael did not have WebRender enabled for Firefox. He used Firefox 56//57 instead of Firefox 58. He used Chrome 60 instead of Chrome 61. He used Chrome instead of Chromium. He used kernel 4.10 instead of kernel 4.13. He used Mesa 17.0 instead of Mesa 17.2 or Mesa 17.3. And he used xf86-video-modesetting instead of xf86-video-intel, even though he knows that it is slower for 2D.



Michael How about re-doing the benchmark ?

[leipero]

pq1930562 Maybe his test were done before and he published it few days/weeks latter? When you run website you ahve to have some schedule.

Also on the topic of WebRender, I didn't know that was an option? Why Mozilla do not enable this by default, I will test it with it, is it possible on v55 or do I have to run v57 or even v58? mmstick

[mmstick]

pq1930562 leipero Phoronix benchmarks rarely ever go outside of the default options, so it's likely that he had hardware acceleration and WebRender disabled. WebRender isn't enabled by default because it's still a WIP. It can cause all manner of glitches and crashes on some websites. It's also far from being feature-complete, as only portions of WebRender are hooked up to Gecko at the moment, and there's a lot of redundant work being processed between Gecko and WebRender that is causing significant rendering times. So the current performance is just a glimpse of what's to come. It may be ready by 58, but I think it's likely that WebRender will take at least by 59 or 60 before it's ready to be enabled by default.

[leipero]

mmstick After some testing i see why theya re disabled by default, gfx.webrender.enabled actually hurts performance on my hardware, and some benchmarks can't be finished (Firefox v57 crashes), I could notice it instantly while scrolling pages became laggy and stuttery.