Mesa Developers Discuss LTO'ing + PGO'ing Builds For Greater Performance

[Soul_keeper]

I think to do this properly would require a directory full of sample programs that could be compiled/run to generate profile data.
I nice collection that get good code coverage for the most common use cases. You can combine the profile data from multiple runs of multiple projects relatively easily.
This would be ideal.

In theory you could profile mesa for a single game/app that you want to run super fast. The "basket" approach would serve most people.

[tildearrow]

Typo?...

while collecting samples to analyze hot/code code-paths, etc,

[dcbaker]

oibaf, CochainComplex, with meson you really want to use the builtin instead of passing -flto manually, `meson builddir -Db_lto=true`. That way if the build system wants to make decisions based on lto you can, and its portable to other compilers.

[soulsource]

As pointed out in the thread on mesa-dev, a wide collection of representative profiles would be great to have, and it should be automated. As the Phoronix Test Suite does automated testing, could it be possible to generate these profiles from automated benchmark runs and integrate it into the build process for release builds?!

I'm wondering if one could use recorded API traces for that purpose. I've got absolutely no idea about the legal situation, but maybe the community could then even provide recordings of commercial software, so that the profiles can be generated for real-life cases without the need for a license?

[QwertyChouskie]

To work around legal issues, API traces of software like SuperTuxKart/Xonotic/0AD/open benchmarks/etc. could be used.

[nuetzel]

As pointed out in the thread on mesa-dev, a wide collection of representative profiles would be great to have, and it should be automated. As the Phoronix Test Suite does automated testing, could it be possible to generate these profiles from automated benchmark runs and integrate it into the build process for release builds?!

That was my idea. @Michael?

I'm under the impression that we needn't 'all' my used progs.
But I have to test with one more OpenGL (DiRT Rally) and 2 (+) Vulkan games (SotTB + F1 2017).

What we have to know next is if we could use a collection of training data for multiple CPUs. @Honza?

[nuetzel]

Is there a possibility to find your buildflags? I want to play aroud with it on ClearLinux .

p.s. nuetzel is that you?

Yep, Dieter here.

If 'you' need my 'instructions' or more numbers go, here:

https://lists.freedesktop.org/archives/mesa-dev/2020-February/224096.html

[pal666]

the hard thing about pgo is profile has to be representative of target load. i.e. ideally it should run same game under same videocard model

[Tvashtar]

It'd be nice if there was a central repo containing the PGO profiles to save folk generating them themselves

I've not used PGO before on Gentoo, what's actually produced and how is it then reused by the build?

If it's a directory would you mind dumping it somewhere? I'm curious to know whether profiles can be partially reused between versions of apps. i.e. does it say a particular function is used a lot and another less so. So as long as the function names are the same the profile will work? Or is it a lot more complicated than that

Anyway I'm babbling

You do two compilations, after first, mesa will spit bunch of *.gcda file in specified directory when ran and closed, usually I do some benches of gpu acceled software I care about.
Then .gcda is used by second compilation. It can be semi-automated under Gentoo [ e.g. paths ]
It might be more complicated since profiles break between mesa versions and my usual is the bleeding edge

[nuetzel]

the hard thing about pgo is profile has to be representative of target load. i.e. ideally it should run same game under same videocard model

I do not think this is right.
We optimize the (CPU) setup code.