Announcement

Collapse
No announcement yet.

Intergrating profiling of Mesa into PTS using OProfiler

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • Intergrating profiling of Mesa into PTS using OProfiler

    Does the idea of intergrating profiling of Mesa into PTS using OProfiler sound plasable? It seems if it was possible to automate profiling with something like PTS that the job of finding and improving slow code paths in Mesa could be made a whole lot easier. For more on my idea see: http://www.itsqueeze.com/2014/05/thr...project-ideas/

  • #2
    Originally posted by tarceri View Post
    Does the idea of intergrating profiling of Mesa into PTS using OProfiler sound plasable? It seems if it was possible to automate profiling with something like PTS that the job of finding and improving slow code paths in Mesa could be made a whole lot easier. For more on my idea see: http://www.itsqueeze.com/2014/05/thr...project-ideas/
    I don't have much experience with oprofile but from what I know and my experience with utilizing other metrics during PTS testing, yes, it would be fairly easy to implement; almost within a few dozen lines of code could quite possibly be all that's needed.
    Michael Larabel
    https://www.michaellarabel.com/

    Comment


    • #3
      Originally posted by Michael View Post
      I don't have much experience with oprofile but from what I know and my experience with utilizing other metrics during PTS testing, yes, it would be fairly easy to implement; almost within a few dozen lines of code could quite possibly be all that's needed.
      Great this is good to know. Are you able to point out where in the code you would think changes would need to be added? I'm just looking for a starting point as it stands I'm not really sure how it all fits toghether so I'm kind of search for a needle in a hay stack. It would save time if you could point me in the right direction.

      Comment


      • #4
        Originally posted by tarceri View Post
        Great this is good to know. Are you able to point out where in the code you would think changes would need to be added? I'm just looking for a starting point as it stands I'm not really sure how it all fits toghether so I'm kind of search for a needle in a hay stack. It would save time if you could point me in the right direction.
        phoronix-test-suite/pts-core/modules/

        If from there you look at the timed_screenshot module that is very simple for simply showing processes to run concurrently during the actual testing process by taking a screenshot at a defined interval. There's a few other trivial modules in that directory that show for how to read the current value of the test being run, etc. Most likely you would want to create some oprofile PTS module in that directory.
        Michael Larabel
        https://www.michaellarabel.com/

        Comment


        • #5
          Originally posted by Michael View Post
          phoronix-test-suite/pts-core/modules/

          If from there you look at the timed_screenshot module that is very simple for simply showing processes to run concurrently during the actual testing process by taking a screenshot at a defined interval. There's a few other trivial modules in that directory that show for how to read the current value of the test being run, etc. Most likely you would want to create some oprofile PTS module in that directory.
          Nice. Thanks

          Comment


          • #6
            Originally posted by Michael View Post
            phoronix-test-suite/pts-core/modules/

            If from there you look at the timed_screenshot module that is very simple for simply showing processes to run concurrently during the actual testing process by taking a screenshot at a defined interval. There's a few other trivial modules in that directory that show for how to read the current value of the test being run, etc. Most likely you would want to create some oprofile PTS module in that directory.
            Ok, I've got something profiling in system-wide mode but it would be better if I could retrive the pid of the application being tested so that I can just profile that application. I will need to make a change to pts_test_execution.php to get the pid (havent worked out all the details of how to do this yet), and maybe add a new function to the module interface to be called right after the test is started (some kind of thread will be needed I'm guessing) in order for the module to kick off profiling using the retrived pid.

            Comment


            • #7
              Originally posted by tarceri View Post
              Ok, I've got something profiling in system-wide mode but it would be better if I could retrive the pid of the application being tested so that I can just profile that application. I will need to make a change to pts_test_execution.php to get the pid (havent worked out all the details of how to do this yet), and maybe add a new function to the module interface to be called right after the test is started (some kind of thread will be needed I'm guessing) in order for the module to kick off profiling using the retrived pid.
              hmmm okay, there's a few functions in PHP for being able to obtain the PID of a process. In pts_test_execution the shell_exec() might need to be reworked to use proc_open() instead and from there you can use proc_get_status() and from there should be able to read the PID as one of the elements in the returned array.

              Proper threading in PHP is a bit of a bitch and not always available; php5-pcntl is generally required and is shipped by default on most distributions but not all. It might be better to check for threading support within the module and then just have it wait from the existing API that calls right before test execution until it finds the PID placed within some file or something. If you want to fork into a separate thread, I have a template for that if you call pts_module:ts_fork_function('FUNCTION') where that function should be called asynchronously or pts_module:ts_timed_function(FUNCTION, TIME) where TIME is to keep calling that function from a separate thread every TIME seconds.

              Hope that might be of some help.
              Michael Larabel
              https://www.michaellarabel.com/

              Comment


              • #8
                Originally posted by Michael View Post
                hmmm okay, there's a few functions in PHP for being able to obtain the PID of a process. In pts_test_execution the shell_exec() might need to be reworked to use proc_open() instead and from there you can use proc_get_status() and from there should be able to read the PID as one of the elements in the returned array.
                Yeah I stumbled across proc_open() while googling. I have updated pts_client.php to use proc_open() rather than shell_exec() and have been able to retrieve the pid and call a module function from there.
                I've pushed my current work to github its still probably needs a bit more work (fyi I'm no php programmer) as I've only really tested with nexuiz so far and the way the pid is retrived seems a bit flaky. Currently the oprofile module simply triggers a single test run and profiles the entire run but I hope to (if I can find time) do some fancy stuff with the oprifle output and also try run multiple profile bursts on parts of the test were cpu peaks.
                If you are interested in what I've done so far the github branch is here: https://github.com/tarceri/phoronix-.../tree/oprofile

                Edit: When I say "its still probably needs a bit more work" I'm talking about the pts_client change. The module is a bit of a mess and needs quite a bit of work still

                Originally posted by Michael View Post
                Proper threading in PHP is a bit of a bitch and not always available; php5-pcntl is generally required and is shipped by default on most distributions but not all. It might be better to check for threading support within the module and then just have it wait from the existing API that calls right before test execution until it finds the PID placed within some file or something. If you want to fork into a separate thread, I have a template for that if you call pts_module:ts_fork_function('FUNCTION') where that function should be called asynchronously or pts_module:ts_timed_function(FUNCTION, TIME) where TIME is to keep calling that function from a separate thread every TIME seconds.

                Hope that might be of some help.
                Thanks for the tips I will probably create a modified version of pts_timed_function() to try profile cpu peaks.
                Last edited by tarceri; 18 May 2014, 12:43 AM.

                Comment


                • #9
                  Instead of trying to be hacky and timing the cpu peaks, the accurate solution would be to use oprofile's capability to record multiple events, and then present the functions ordered by "unhalted clocks in highest power state", for example.

                  Oprofile does not currently expose the power state as an event. You may be able to use some other event as a "high power state" indicator, or consider adding p-state event support to oprofile.

                  Comment


                  • #10
                    Originally posted by curaga View Post
                    Instead of trying to be hacky and timing the cpu peaks, the accurate solution would be to use oprofile's capability to record multiple events, and then present the functions ordered by "unhalted clocks in highest power state", for example.

                    Oprofile does not currently expose the power state as an event. You may be able to use some other event as a "high power state" indicator, or consider adding p-state event support to oprofile.
                    Thanks for pointing this out. I'll have to look into this further, to be honest I'm new to using oprofile so I'm not fully across all its features yet.

                    Comment

                    Working...
                    X