Announcement

Collapse
No announcement yet.

How to create a new test?

Collapse
X
  • Filter
  • Time
  • Show
Clear All
new posts

  • How to create a new test?

    Hi all,

    I like to create a test for software not covered so far.
    Unfortunately when clicking Create a Suite at http://openbenchmarking.org/create_suite
    I am getting only:
    Coming Soon
    This feature is currently disabled.


    Is there a way to do without?
    Or does anyone know how soon 'soon' is?

    Thanks,
    Jan

    P.S.:
    I searched this forum and could not find anything on creating new tests.
    Where to look?

  • #2
    Originally posted by janmartin3 View Post
    Hi all,

    I like to create a test for software not covered so far.
    Unfortunately when clicking Create a Suite at http://openbenchmarking.org/create_suite
    I am getting only:
    Coming Soon
    This feature is currently disabled.


    Is there a way to do without?
    Or does anyone know how soon 'soon' is?

    Thanks,
    Jan

    P.S.:
    I searched this forum and could not find anything on creating new tests.
    Where to look?
    Creating a test profile is what you're after... I don't have a concise guide covering the entire process at the moment due to lack of time, but most people are able to figure it out by looking at an example and either modifying it or just learning from it. So an easy example would be to look at a test profile like c-ray or smallpt or openarena.

    1. first fetch the test profile locally.... that can be done by a variety of ways, it's all done automatically by PTS. Easy way is just like running: phoronix-test-suite info smallpt .... which will download the data from OpenBenchmarking.org if it's not already present locally.
    2. look inside ~/.phoronix-test-suite/test-profiles/pts/smallpt*
    3. from there you should be able to figure it out... pretty easy, collection of bash scripts and XML.

    Then any further questions can be asked in the forums.
    Michael Larabel
    http://www.michaellarabel.com/

    Comment


    • #3
      Just found out that
      phoronix-test-suite build-suite
      creates a framework for a new test suite.

      Also in
      .phoronix-test-suite/installed-tests
      there are a few tests to look at for inspiration.

      Is there a more detailed HowTo available anywhere?

      Thanks,
      Jan

      Comment


      • #4
        Originally posted by janmartin3 View Post
        Just found out that
        phoronix-test-suite build-suite
        creates a framework for a new test suite.

        Also in
        .phoronix-test-suite/installed-tests
        there are a few tests to look at for inspiration.

        Is there a more detailed HowTo available anywhere?

        Thanks,
        Jan
        That's to build a suite, which is a collection of tests to be executed. If you want to support a new piece of software, that's just a test profile, which my previous post covers.

        Within the installed-tests/ directory is where tests are installed. If you want to create a test profile look within the test-profiles/ directory.

        Only documentation would be what's on here or the linked PDF - http://phoronix-test-suite.com/?k=documentation - from that page.
        Michael Larabel
        http://www.michaellarabel.com/

        Comment


        • #5
          Made directory with xmls and scripts - now what?

          Originally posted by Michael View Post
          Within the installed-tests/ directory is where tests are installed. If you want to create a test profile look within the test-profiles/ directory.

          Only documentation would be what's on here or the linked PDF - http://phoronix-test-suite.com/?k=documentation - from that page.
          Sorry to resurrect this old thread, but I'm running into the same problem and, unfortunately, not much has changed regarding documentation for asset creation, AFAICT.

          Anyway, I created the xml files and bash scripts inside a directory - now what? I tried copying the dir into ~/.phoronix-test-suite/test-profiles/local, then pts/, with no results (the test didi not show up in list-available-test or in list-installed-tests). Is there some special syntax to debug-install a test from a local storage, rather than OB.org repository?

          Note - I'm on OS X, so no GUI here.

          Comment


          • #6
            Originally posted by myxal View Post
            Sorry to resurrect this old thread, but I'm running into the same problem and, unfortunately, not much has changed regarding documentation for asset creation, AFAICT.

            Anyway, I created the xml files and bash scripts inside a directory - now what? I tried copying the dir into ~/.phoronix-test-suite/test-profiles/local, then pts/, with no results (the test didi not show up in list-available-test or in list-installed-tests). Is there some special syntax to debug-install a test from a local storage, rather than OB.org repository?

            Note - I'm on OS X, so no GUI here.
            If you put the test profile in local/ it should work fine without OB. Just run 'phoronix-test-suite benchmark local/<whatever the sub-folder you called it>'
            Michael Larabel
            http://www.michaellarabel.com/

            Comment


            • #7
              Originally posted by Michael View Post
              If you put the test profile in local/ it should work fine without OB. Just run 'phoronix-test-suite benchmark local/<whatever the sub-folder you called it>'
              Thanks, now I see what was wrong - I included the version number in the directory name (scan-basic-0.1.0), but PTS apparently doesn't expect this.
              Useful note: just name the profile directory normally.

              Also, the test is never listed under "list-available-tests", only under "list-installed-tests" after installation - is this expected?

              Comment


              • #8
                Originally posted by myxal View Post
                Also, the test is never listed under "list-available-tests", only under "list-installed-tests" after installation - is this expected?
                With phoronix test suite 4.4/Git, the local tests will now be shown under list-available-tests. Thanks.
                Michael Larabel
                http://www.michaellarabel.com/

                Comment

                Working...
                X