Announcement

Collapse
No announcement yet.

Custom Benchmark Phoromatic

Collapse
X
  • Filter
  • Time
  • Show
Clear All
new posts

  • Custom Benchmark Phoromatic

    I'm very excited about integrating local tests with Phoromatic in the 7.8 release. I've started messing around with it, and it works great for my needs! As expected, I can create a custom local test profile and run it on a client server if I add it to a custom suite. However, when running a benchmark from Phoromatic that includes a custom test, it repeats over and over once the test completes. It immediately runs the test again, printing 'Test started' and 'Installed' each time. It only stops if I go on the Phoromatic interface and click End Ticket. I do not have this problem if the benchmark includes only pts and system tests. Should I be running custom tests another way? Is there a tag I've neglected in the xml configuration files? Thanks.

  • #2
    Originally posted by aurnes View Post
    I'm very excited about integrating local tests with Phoromatic in the 7.8 release. I've started messing around with it, and it works great for my needs! As expected, I can create a custom local test profile and run it on a client server if I add it to a custom suite. However, when running a benchmark from Phoromatic that includes a custom test, it repeats over and over once the test completes. It immediately runs the test again, printing 'Test started' and 'Installed' each time. It only stops if I go on the Phoromatic interface and click End Ticket. I do not have this problem if the benchmark includes only pts and system tests. Should I be running custom tests another way? Is there a tag I've neglected in the xml configuration files? Thanks.
    Hmmm haven't encountered this problem at all. Is your test working fine from the CLI in terms of returning a test result? Unfortunately it's hard to debug without seeing the phoromatic.log or phoronix-test-suite.log or the test profile itself. Would you be able to post any of those?
    Michael Larabel
    http://www.michaellarabel.com/

    Comment


    • #3
      Sure, here's part of phoromatic.log. I'm just running a benchmark called Pop from a previous thread. The only test is local/pop. As you can see, it says "V3QI8 - needs to benchmark ticket for Pop" multiple times; that's each time it tries to run the benchmark. Normally (for a benchmark with pts or system tests, not local), it only does all that once, and it only runs once.

      Code:
      [Feb 21 21:43:46 2018] V3QI8 - needs to benchmark ticket for Pop
      [Wed Feb 21 21:43:46 2018] 192.168.6.25:34932 [200]: /phoromatic.php
      [Wed Feb 21 21:43:47 2018] 192.168.6.25:34938 [200]: /phoromatic.php
      [Wed Feb 21 21:43:47 2018] 192.168.6.25:34940 [200]: /server.php
      [Wed Feb 21 21:43:47 2018] 192.168.6.25:34942 [200]: /openbenchmarking-cache.php?is_test&repo=&test=835402de76d0a3da71ee1a1360fec82b88ba6b51
      [Wed Feb 21 21:43:47 2018] 192.168.6.25:34944 [200]: /openbenchmarking-cache.php?is_test&repo=&test=835402de76d0a3da71ee1a1360fec82b88ba6b51
      [Wed Feb 21 21:43:47 2018] 192.168.6.25:34946 [200]: /phoromatic.php
      [Wed Feb 21 21:43:49 2018] 192.168.6.25:34948 [200]: /phoromatic.php
      [Wed Feb 21 21:43:54 2018] 192.168.6.25:34950 [200]: /phoromatic.php
      [Wed Feb 21 21:43:56 2018] 192.168.6.25:34952 [200]: /phoromatic.php
      [Wed Feb 21 21:44:01 2018] 192.168.6.25:34954 [200]: /phoromatic.php
      [Wed Feb 21 21:44:01 2018] 192.168.6.25:34956 [200]: /phoromatic.php
      [Wed Feb 21 21:44:01 2018] 192.168.6.25:34958 [200]: /phoromatic.php
      [Feb 21 21:45:02 2018] V3QI8 - needs to benchmark ticket for Pop
      [Wed Feb 21 21:45:02 2018] 192.168.6.25:34960 [200]: /phoromatic.php
      [Wed Feb 21 21:45:03 2018] 192.168.6.25:34964 [200]: /phoromatic.php
      [Wed Feb 21 21:45:10 2018] 192.168.6.25:34968 [200]: /phoromatic.php
      [Wed Feb 21 21:45:10 2018] 192.168.6.25:34970 [200]: /openbenchmarking-cache.php?is_test&repo=&test=4d81279ce06b466f4054238ca7c05d0bfd008dc3
      [Wed Feb 21 21:45:10 2018] 192.168.6.25:34972 [200]: /openbenchmarking-cache.php?is_test&repo=&test=4d81279ce06b466f4054238ca7c05d0bfd008dc3
      [Wed Feb 21 21:45:10 2018] 192.168.6.25:34974 [200]: /phoromatic.php
      [Wed Feb 21 21:45:11 2018] 192.168.6.25:34976 [200]: /phoromatic.php
      [Wed Feb 21 21:45:13 2018] 192.168.6.25:34978 [200]: /phoromatic.php
      [Wed Feb 21 21:45:14 2018] 192.168.6.25:34980 [200]: /phoromatic.php
      [Wed Feb 21 21:45:14 2018] 192.168.6.25:34982 [200]: /phoromatic.php
      [Wed Feb 21 21:45:20 2018] 192.168.6.25:34984 [200]: /phoromatic.php

      Comment


      • #4
        Are you seeing any test results uploaded for that test? Do you happen to have the output from what shows on the PTS client system? Thanks.
        Michael Larabel
        http://www.michaellarabel.com/

        Comment


        • #5
          This information is from the system page on the Phoromatic interface:
          • The client log says
            "[Feb 23 13:47:52 2018] Running Benchmarks For: PopTest
            [Feb 23 13:47:52 2018] Tests In Run Queue: local/pop
            [Feb 23 13:47:53 2018] Running: local/pop
            [Feb 23 13:47:56 2018] Benchmarks Completed For: PopTest"
            repeatedly.
          • It logs the error "The test run did not produce a result" every time.

          On the client server, it says
          "Phoronix Test Suite v7.8.0

          Installed: local/pop
          local/pop: Test Started
          Installed: local/pop
          local/pop: Test Started"

          immediately and repeatedly. If I uninstall the test, it briefly installs it from the host server before resuming this behavior. Also, as I mentioned previously, it does not stop unless I end the ticket.

          Comment


          • #6
            After some pondering, I now see that the intended functionality is to run custom tests that reside on the Phoromatic host server from the command line on the client server. This works fine. Do you have any idea why it misbehaves when the command originates from the Phoromatic host server itself? Or do you have any plans for implementing running custom tests THROUGH the Phoromatic web interface in the future? Ideally, I'd like to run custom tests on several client systems at once. Thanks, sorry for the confusion.

            Comment


            • #7
              Originally posted by aurnes View Post
              After some pondering, I now see that the intended functionality is to run custom tests that reside on the Phoromatic host server from the command line on the client server. This works fine. Do you have any idea why it misbehaves when the command originates from the Phoromatic host server itself? Or do you have any plans for implementing running custom tests THROUGH the Phoromatic web interface in the future? Ideally, I'd like to run custom tests on several client systems at once. Thanks, sorry for the confusion.
              Is there any chance I could take a look at the test profile? Can email it if desired, as I still haven't been able to reproduce this issue on my end.

              You should be able to select the custom test from the web interface and use it with any test schedule. Is it not showing up for you? If so if you could take a screenshot or so where it's not showing up in a given menu or area that would be useful. Thanks.
              Michael Larabel
              http://www.michaellarabel.com/

              Comment


              • #8
                Oh? It might be worth mentioning that I have never witnessed a local test on a dropdown menu or anything on the web interface. I have to create a benchmark, add a pts or system test to it, go on the host server, and edit its XML file to have the name of the local test instead. As I mentioned, it, uh, may or may not work. Also the custom tests I've tried are mostly based off of things I've seen around the forums, but it's possible I've committed some fatal error. Do you have any simple example custom tests that I could look at?

                Comment

                Working...
                X