GCC 4.9 Compiler Optimization Benchmarks For Faster Binaries

[Michael]

A much better test would be compiling ... Mozilla Firefox with given optimization levels and then measuring its performance in various JS benchmarks.

I'm not aware of any automation tools for LibreOffice to test its performance, but LibreOffice tests would be nice as well.

The problem with these artificial benchmarks is that the average Joe rarely if never runs them.

The average Joe meanwhile spends a lot of time browsing the web and creating spreadsheets.

If there were such automated application tests, I'd be happy to run them, but alas there isn't.

[MWisBest]

If there were such automated application tests, I'd be happy to run them, but alas there isn't.

And even if there was, the amount of time it takes to compile Firefox is already ridiculous enough!

[oleid]

Michael, could you please include -mtune=generic to get a glimpse how fast distribution provided binaries are? That would be quite interesting!

[hubicka]

If there were such automated application tests, I'd be happy to run them, but alas there isn't.

Firefox has quite few ways to benchmark it. Dromaeo, telemetry, talos, sunspider... I plan to get some more data, so far I only measured dromaeo with and without LTO/FDO http://hubicka.blogspot.ca/2014/04/d...-feedback.html

I also built libreoffice with and without LTO. W/o LTO the directory with all libraries is 430MB (GCC 4.9), with clang's February mainline checkout LTO it is 350 and with GCC 4.9 it is 303MB. I will discuss with LO folks ways to benchmark it. So far I only got suggestion to run the unit tests. Will do that as time allows.

[Michael]

Firefox has quite few ways to benchmark it. Dromaeo, telemetry, talos, sunspider... I plan to get some more data, so far I only measured dromaeo with and without LTO/FDO http://hubicka.blogspot.ca/2014/04/d...-feedback.html

But with any of those ways is it automated from launching Firefox to closing it and gathering the results? I've heard of a few ways to automate actions within the browser but unfortunately not a way for it to be fully automated where it could be launched from a script and the result returned upon completion.

I also built libreoffice with and without LTO. W/o LTO the directory with all libraries is 430MB (GCC 4.9), with clang's February mainline checkout LTO it is 350 and with GCC 4.9 it is 303MB. I will discuss with LO folks ways to benchmark it. So far I only got suggestion to run the unit tests. Will do that as time allows.

So far haven't been aware of any real ways to fully automate LO. Back in the OpenOffice days there was reportedly some automated benchmarks via http://www.oooninja.com/2009/03/mult...chmark-30.html and that author I had contacted said he would release the scripts eventually or sell them.

[chinoto]

[...] The average Joe meanwhile spends a lot of time browsing the web and creating spreadsheets.

There isn't much benefit to more speed in a browser. YouTube works fine, what more do you need out of a browser . I've read that complex spreadsheets can get slow, but I doubt people do that very often.
I would think video editing and 3d rendering programs (ie games) would be among the more noticeable ones for average users, but LTO might not even do anything for them.

[toyotabedzrock]

But with any of those ways is it automated from launching Firefox to closing it and gathering the results? I've heard of a few ways to automate actions within the browser but unfortunately not a way for it to be fully automated where it could be launched from a script and the result returned upon completion.



So far haven't been aware of any real ways to fully automate LO. Back in the OpenOffice days there was reportedly some automated benchmarks via http://www.oooninja.com/2009/03/mult...chmark-30.html and that author I had contacted said he would release the scripts eventually or sell them.

Use Selenium - http://docs.seleniumhq.org/
There is also http://dalekjs.com/ but I'm less familiar with it.

Also I'm curious if gcc can use TSX to optimize a binary?

[gnzlbg]

These things are extremely useful Michael (I hope you are sending bugreports for the regressions with O3 being slower than O2).

Would it be possible to benchmark -Ofast -march=native -flto ?

Since -Ofast enables -O3 + -ffast-math I would expect it to be the most aggresive "easy to activate" optimization level.

[curaga]

There isn't much benefit to more speed in a browser. YouTube works fine, what more do you need out of a browser

Fast startup, fast responsiveness, low resource usage. We aren't at the "what more do you need" point until _every_ action takes at most 10ms, including startup from cold cache on a hdd.

[hubicka]

But with any of those ways is it automated from launching Firefox to closing it and gathering the results? I've heard of a few ways to automate actions within the browser but unfortunately not a way for it to be fully automated where it could be launched from a script and the result returned upon completion.

TALOS seems to do that (I am playing with it now). Dromaeo indeed needs one mouse click but there is probably way how to script that.



So far haven't been aware of any real ways to fully automate LO. Back in the OpenOffice days there was reportedly some automated benchmarks via http://www.oooninja.com/2009/03/mult...chmark-30.html and that author I had contacted said he would release the scripts eventually or sell them.

Except for the unit tests me neither, it is on my TODO.