Announcement

Collapse
No announcement yet.

Altering Default Test Options for batch-benchmark

Collapse
X
Collapse
 
  • Page of 1
  • Filter
  • Time
  • Show
Clear All
new posts
Previous template Next
  • #1

    Altering Default Test Options for batch-benchmark

    I am trying to run the fio test (Disk suite) as part of a set of benchmarks, but allowing default options makes it process 6144 tests.

    I want to pare this down to a reasonable number by, say, just testing non-buffered, direct I/O with POSIX AIO engine.

    From my understanding of the phoronix-test-suite run process, it looks like I need to change the options in ~/.phoronix-test-suite/test-profiles/pts/fio-1.8.2/test-definition.xml

    I have tried changing the 'DefaultEntry' under 'TestSettings/Options' to the entry number I want (by default, they were all 0 - I assume that meant 'all of the above' if there were to be a prompt). Unfortunately, that doesn't seem to do the trick and I still end up with 6144 tests when I run batch-benchmark on pts/fio.

    Am I misunderstanding something here? Can someone help me with the meanings of 'DefaultEntry' and 'Identifier', and how to ensure that only a particular set of options can be used while running fio in batch-benchmark mode?

    Thanks!
    Tags: None
  • #2
    Default entry and identifier aren't related to batch benchmarking. Just remove the (and contained Identifier and Value) within XML file if you wish to remove the options.... Or make your own test suite out of the tests you do want and then simply run that test suite. You can make the suite by phoronix-test-suite build-suite
    Michael Larabel
    https://www.michaellarabel.com/

    Comment

    • #3
      Michael,

      Thanks for the reply. I tried following your suggestion, but still don't get what I want. Maybe an example would be good - I am placing the original test-definition.xml and the version that I have modified below.

      Original:

      Code:
      <?xml version="1.0"?>
<!--Phoronix Test Suite v5.8.0m4 (Belev)-->
<PhoronixTestSuite>
  <TestInformation>
    <Title>Flexible IO Tester</Title>
    <AppVersion>2.1.13</AppVersion>
    <Description>Fio is an advanced disk benchmark that depends upon the kernel's AIO access library.</Description>
    <Executable>fio-run</Executable>
    <TimesToRun>3</TimesToRun>
  </TestInformation>
  <TestProfile>
    <Version>1.8.2</Version>
    <SupportedPlatforms>Linux</SupportedPlatforms>
    <SoftwareType>Benchmark</SoftwareType>
    <TestType>Disk</TestType>
    <License>Free</License>
    <Status>Verified</Status>
    <ExternalDependencies>build-utilities, libaio-development</ExternalDependencies>
    <EnvironmentSize>4</EnvironmentSize>
    <ProjectURL>http://git.kernel.dk/?p=fio.git;a=summary</ProjectURL>
    <RequiresCoreVersionMin>5310</RequiresCoreVersionMin>
    <Maintainer>Michael Larabel</Maintainer>
  </TestProfile>
  <TestSettings>
    <Option>
      <DisplayName>Type</DisplayName>
      <Identifier>type</Identifier>
      <DefaultEntry>4</DefaultEntry>
      <Menu>
        <Entry>
          <Name>Random Read</Name>
          <Value>randread</Value>
        </Entry>
        <Entry>
          <Name>Random Write</Name>
          <Value>randwrite</Value>
        </Entry>
        <Entry>
          <Name>Sequential Read</Name>
          <Value>read</Value>
        </Entry>
        <Entry>
          <Name>Sequential Write</Name>
          <Value>write</Value>
        </Entry>
      </Menu>
    </Option>
    <Option>
      <DisplayName>IO Engine</DisplayName>
      <Identifier>engine</Identifier>
      <DefaultEntry>0</DefaultEntry>
      <Menu>
        <Entry>
          <Name>POSIX AIO</Name>
          <Value>posixaio</Value>
        </Entry>
        <Entry>
          <Name>Sync</Name>
          <Value>sync</Value>
        </Entry>
        <Entry>
          <Name>Libaio</Name>
          <Value>libaio</Value>
        </Entry>
      </Menu>
    </Option>
    <Option>
      <DisplayName>Buffered</DisplayName>
      <Identifier>buffered</Identifier>
      <DefaultEntry>1</DefaultEntry>
      <Menu>
        <Entry>
          <Name>Yes</Name>
          <Value>1</Value>
        </Entry>
        <Entry>
          <Name>No</Name>
          <Value>0</Value>
        </Entry>
      </Menu>
    </Option>
    <Option>
      <DisplayName>Direct</DisplayName>
      <Identifier>direct</Identifier>
      <DefaultEntry>1</DefaultEntry>
      <Menu>
        <Entry>
          <Name>No</Name>
          <Value>0</Value>
        </Entry>
        <Entry>
          <Name>Yes</Name>
          <Value>1</Value>
        </Entry>
      </Menu>
    </Option>
    <Option>
      <DisplayName>Block Size</DisplayName>
      <Identifier>size</Identifier>
      <DefaultEntry>16</DefaultEntry>
      <Menu>
        <Entry>
          <Name>4KB</Name>
          <Value>4k</Value>
        </Entry>
        <Entry>
          <Name>8KB</Name>
          <Value>8k</Value>
        </Entry>
        <Entry>
          <Name>16KB</Name>
          <Value>16k</Value>
        </Entry>
        <Entry>
          <Name>32KB</Name>
          <Value>32k</Value>
        </Entry>
        <Entry>
          <Name>64KB</Name>
          <Value>64k</Value>
        </Entry>
        <Entry>
          <Name>128KB</Name>
          <Value>128k</Value>
        </Entry>
        <Entry>
          <Name>256KB</Name>
          <Value>256k</Value>
        </Entry>
        <Entry>
          <Name>512KB</Name>
          <Value>512k</Value>
        </Entry>
        <Entry>
          <Name>1MB</Name>
          <Value>1m</Value>
        </Entry>
        <Entry>
          <Name>2MB</Name>
          <Value>2m</Value>
        </Entry>
        <Entry>
          <Name>4MB</Name>
          <Value>4m</Value>
        </Entry>
        <Entry>
          <Name>8MB</Name>
          <Value>8m</Value>
        </Entry>
        <Entry>
          <Name>16MB</Name>
          <Value>16m</Value>
        </Entry>
        <Entry>
          <Name>32MB</Name>
          <Value>32m</Value>
        </Entry>
        <Entry>
          <Name>64MB</Name>
          <Value>64m</Value>
        </Entry>
        <Entry>
          <Name>128MB</Name>
          <Value>128m</Value>
        </Entry>
      </Menu>
    </Option>
    <Option>
      <DisplayName>Disk Target</DisplayName>
      <Identifier>auto-disk-mount-points</Identifier>
      <DefaultEntry>0</DefaultEntry>
    </Option>
    <Option>
      <DisplayName>Result</DisplayName>
      <Identifier>result</Identifier>
      <DefaultEntry>2</DefaultEntry>
      <Menu>
        <Entry>
          <Name>MB/s</Name>
          <Value>MBS</Value>
        </Entry>
        <Entry>
          <Name>IOPS</Name>
          <Value>IOPS</Value>
        </Entry>
      </Menu>
    </Option>
  </TestSettings>
</PhoronixTestSuite>
      My intention with the new test-definition.xml is to run fio with the following set of options:
      • Test all four combinations : randread, randwrite, read, write
      • Test only POSIX aio
      • Test buffered = 0, direct = 1
      • Test all 16 available block sizes from 4K to 128M
      • Test only default testing folder and NOT all available mount points
      • Provide both MBps and IOPS numbers in the result

      With the above intention in mind, I commented / removed the various options as below:

      Code:
      <?xml version="1.0"?>
<!--Phoronix Test Suite v5.8.0m4 (Belev)-->
<PhoronixTestSuite>
  <TestInformation>
    <Title>Flexible IO Tester</Title>
    <AppVersion>2.1.13</AppVersion>
    <Description>Fio is an advanced disk benchmark that depends upon the kernel's AIO access library.</Description>
    <Executable>fio-run</Executable>
    <TimesToRun>3</TimesToRun>
  </TestInformation>
  <TestProfile>
    <Version>1.8.2</Version>
    <SupportedPlatforms>Linux</SupportedPlatforms>
    <SoftwareType>Benchmark</SoftwareType>
    <TestType>Disk</TestType>
    <License>Free</License>
    <Status>Verified</Status>
    <ExternalDependencies>build-utilities, libaio-development</ExternalDependencies>
    <EnvironmentSize>4</EnvironmentSize>
    <ProjectURL>http://git.kernel.dk/?p=fio.git;a=summary</ProjectURL>
    <RequiresCoreVersionMin>5310</RequiresCoreVersionMin>
    <Maintainer>Michael Larabel</Maintainer>
  </TestProfile>
  <TestSettings>
<!--
    <Option>
      <DisplayName>Type</DisplayName>
      <Identifier>type</Identifier>
      <DefaultEntry>4</DefaultEntry>
      <Menu>
        <Entry>
          <Name>Random Read</Name>
          <Value>randread</Value>
        </Entry>
        <Entry>
          <Name>Random Write</Name>
          <Value>randwrite</Value>
        </Entry>
        <Entry>
          <Name>Sequential Read</Name>
          <Value>read</Value>
        </Entry>
        <Entry>
          <Name>Sequential Write</Name>
          <Value>write</Value>
        </Entry>
      </Menu>
    </Option>
-->
    <Option>
      <DisplayName>IO Engine</DisplayName>
      <Identifier>engine</Identifier>
      <DefaultEntry>0</DefaultEntry>
      <Menu>
        <Entry>
          <Name>POSIX AIO</Name>
          <Value>posixaio</Value>
        </Entry>
<!--
        <Entry>
          <Name>Sync</Name>
          <Value>sync</Value>
        </Entry>
        <Entry>
          <Name>Libaio</Name>
          <Value>libaio</Value>
        </Entry>
-->
      </Menu>
    </Option>
    <Option>
      <DisplayName>Buffered</DisplayName>
      <Identifier>buffered</Identifier>
      <DefaultEntry>1</DefaultEntry>
      <Menu>
<!--
        <Entry>
          <Name>Yes</Name>
          <Value>1</Value>
        </Entry>
-->
        <Entry>
          <Name>No</Name>
          <Value>0</Value>
        </Entry>
      </Menu>
    </Option>
    <Option>
      <DisplayName>Direct</DisplayName>
      <Identifier>direct</Identifier>
      <DefaultEntry>1</DefaultEntry>
      <Menu>
<!--
        <Entry>
          <Name>No</Name>
          <Value>0</Value>
        </Entry>
-->
        <Entry>
          <Name>Yes</Name>
          <Value>1</Value>
        </Entry>
      </Menu>
    </Option>
<!--
    <Option>
      <DisplayName>Block Size</DisplayName>
      <Identifier>size</Identifier>
      <DefaultEntry>16</DefaultEntry>
      <Menu>
        <Entry>
          <Name>4KB</Name>
          <Value>4k</Value>
        </Entry>
        <Entry>
          <Name>8KB</Name>
          <Value>8k</Value>
        </Entry>
        <Entry>
          <Name>16KB</Name>
          <Value>16k</Value>
        </Entry>
        <Entry>
          <Name>32KB</Name>
          <Value>32k</Value>
        </Entry>
        <Entry>
          <Name>64KB</Name>
          <Value>64k</Value>
        </Entry>
        <Entry>
          <Name>128KB</Name>
          <Value>128k</Value>
        </Entry>
        <Entry>
          <Name>256KB</Name>
          <Value>256k</Value>
        </Entry>
        <Entry>
          <Name>512KB</Name>
          <Value>512k</Value>
        </Entry>
        <Entry>
          <Name>1MB</Name>
          <Value>1m</Value>
        </Entry>
        <Entry>
          <Name>2MB</Name>
          <Value>2m</Value>
        </Entry>
        <Entry>
          <Name>4MB</Name>
          <Value>4m</Value>
        </Entry>
        <Entry>
          <Name>8MB</Name>
          <Value>8m</Value>
        </Entry>
        <Entry>
          <Name>16MB</Name>
          <Value>16m</Value>
        </Entry>
        <Entry>
          <Name>32MB</Name>
          <Value>32m</Value>
        </Entry>
        <Entry>
          <Name>64MB</Name>
          <Value>64m</Value>
        </Entry>
        <Entry>
          <Name>128MB</Name>
          <Value>128m</Value>
        </Entry>
      </Menu>
    </Option>
-->
<!--
    <Option>
      <DisplayName>Disk Target</DisplayName>
      <Identifier>auto-disk-mount-points</Identifier>
      <DefaultEntry>0</DefaultEntry>
    </Option>
-->
<!--
    <Option>
      <DisplayName>Result</DisplayName>
      <Identifier>result</Identifier>
      <DefaultEntry>2</DefaultEntry>
      <Menu>
        <Entry>
          <Name>MB/s</Name>
          <Value>MBS</Value>
        </Entry>
        <Entry>
          <Name>IOPS</Name>
          <Value>IOPS</Value>
        </Entry>
      </Menu>
    </Option>
-->
  </TestSettings>
</PhoronixTestSuite>
      Unfortunately, fio refuses to run. debug-run provides the following output:

      Code:
      Test Run Command: cd /home/ganesh/.phoronix-test-suite/installed-tests/pts/fio-1.8.2/ && ./fio-run posixaio 0 1 2>&1

fio: failed parsing rw=posixaio
fio: failed parsing direct=
fio: failed parsing directory=
fio: job global dropped
      Can you please let me know how the test-definition.xml should be for what I intend to run by default / in batch mode.

      Thank you very much for your help!


      Comment

      • #4
        Oh yeah, when removing those options, the FIO test profile handling is a bit more tricky in that it's expecting the ordering of the arguments to be passed. Did you try just building a suite out of it using phoronix-test-suite build-suite ?
        Michael Larabel
        https://www.michaellarabel.com/

        Comment

        • #5
          Yes, I finally went that route and I think I have got it working as I wanted

          Btw, for a brief time last night, phoronix.com / phoronix-test-suite.com was unreachable. I was incidentally trying build-suite at that time, and it just sort of hung / timed-out multiple times. Anyways, after the downtime got over, I was able to fix up a custom fio.

          Comment

          • #6
            Originally posted by Ganesh_AT View Post
            Yes, I finally went that route and I think I have got it working as I wanted

            Btw, for a brief time last night, phoronix.com / phoronix-test-suite.com was unreachable. I was incidentally trying build-suite at that time, and it just sort of hung / timed-out multiple times. Anyways, after the downtime got over, I was able to fix up a custom fio.
            Okay, great to hear you got it working right. Using build-suite is the preferred (and easier) approach to having to modify the XML files....

            Hmm strange, the server shouldn't have been acting up last night.
            Michael Larabel
            https://www.michaellarabel.com/

            Comment

            Previous template Next
            Working...
            X