Announcement

Collapse
No announcement yet.

pts/dbench vs system/dbench

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • pts/dbench vs system/dbench

    I'm trying to figure out why dbench wasn't run in my local/fs-benchmark test suite. It was working previously; but now it's being silently skipped.

    While trying to figure out why, I noticed this discrepancy:

    [email protected]:~# phoronix-test-suite list-installed-tests | grep dbench
    pts/dbench-1.0.0 - Dbench
    pts/dbench-1.0.1 - Dbench
    [email protected]:~# phoronix-test-suite list-available-tests | grep dbench
    system/dbench - Dbench Disk

    Did the dbench test suite change from pts/dbench to system/dbench? I note this isn't true with, say, fs-mark:

    [email protected]:~# phoronix-test-suite list-installed-tests | grep fs-mark
    pts/fs-mark-1.0.2 - FS-Mark
    [email protected]:~# phoronix-test-suite list-available-tests | grep fs-mark
    pts/fs-mark - FS-Mark Disk

    Could this have something to do with the fact that dbench is marked as deprecated?

    Where else should I look to understand why dbench was skipped despite the fact that it's in the suite definition file.

    # cat test-suites/local/fs-benchmarks/suite-definition.xml
    ...
    <Execute>
    <Test>pts/dbench</Test>
    <Arguments>1</Arguments>
    <Description>1 Clients</Description>
    </Execute>
    <Execute>
    <Test>pts/dbench</Test>
    <Arguments>12</Arguments>
    <Description>12 Clients</Description>
    </Execute>
    ...

    This is unchanged from the pts/disk suite definition file, and as I said, this was working before, and isn't working now, and I'm trying to figure out why.

    Thanks!!

  • #2
    Hmmm, if you run `phoronix-test-suite info <suite name>` does it show up?

    The difference between pts/dbench and system/dbench are that the "system/" tests tend to rely on the system/package-manager-provided packages/software under test rather than building from source. Though that shouldn't be interfering with anything in the pts/* namespace for instance and you could even run both from the same suite.

    If dbench shows up in the info output mentioned above, does it run fine if you do: phoronix-test-suite debug-run pts/dbench ?
    Michael Larabel
    http://www.michaellarabel.com/

    Comment


    • #3
      Hmm, so one the problems is that the first time I ran it, the first hint of trouble was when I ran "phoronix-test-suite batch-run fs-benmarks" and it printed "Test 1 of 9". When I let it finished, I realized it had skipped the two dbench benchmarks. (Note: local/fs-benchmark started as a fork of pts/disk, but I had removed the FIO benchmarks, since those benchmarks essentially never changed across many kernel versions, and in the case of large I/O workloads, especially large Direct I/O workloads, it was more about measuring the underlying disk speeds and not the file system overhead, which on file systems like xfs and ext4, tend to be closer to zero especially for DIO.) Previously, fs-benchmark would run 11 tests, so it would say, "Test 1 of 11"... "Test 11 of 11".

      I tried using "pts info pts/dbench", and it showed the test successfully. I then ran "pts info local/fs-benchmark" and it reported:

      Phoronix Test Suite v9.8.0
      FS Benchmarks

      Suite Description: File system benchmarks

      Run Identifier: local/fs-benchmarks
      Suite Version: 1.0.0
      Maintainer: Theodore Ts'o
      Status:
      Suite Type: Disk
      Unique Tests: 5
      Contained Tests:
      SQLite
      FS-Mark Test: 1000 Files, 1MB Size
      FS-Mark Test: 1000 Files, 1MB Size, No Sync/FSync
      FS-Mark Test: 5000 Files, 1MB Size, 4 Threads
      FS-Mark Test: 4000 Files, 32 Sub Dirs, 1MB Size
      Compile Bench Test: Initial Create
      Compile Bench Test: Compile
      Compile Bench Test: Read Compiled Tree
      Dbench 1 Clients
      Dbench 12 Clients
      PostMark
      18 Tests / 5 Unique Tests

      18 Tests? Huh? Then when I tried running batch-run fs-benchmarks, it reported:

      FS-Mark 3.3:
      pts/fs-mark-1.0.2 [Test: 1000 Files, 1MB Size]
      Test 1 of 10
      ^^^^^^^

      Huh? It was earlier saying Test of 1 of 9, and it ran 9 tests. (When it should have been 11.) How it's saying 10 tests? So I typed control C, and did a few more phoronix-test-suite info commands, and then when I tried the batch-run again, it said, running "Test 1 of 11". So there is some kind of Heisinbug going on here.

      For now, I've created a local/fs-bench suite where I use system/dbench instead of pts/dbench, and I've also forked pts/fs-mark to create a local/fs-mark which runs fs-mark using both 32k files as well as 1MB files. The reason why I want a smaller file size is I want things to be more obvious when there are improvements in the metadata overhead, as opposed to data writes which are just measuring disk speeds, and not file system overhead. I'll try to switch back to pts/dbench but the problem around 3 hours to do a full benchmark run, and dbench happens near the end of the suite. It would be nice if there was a mode where it prints the tests that it would have run, since apparently the number of tests reported by "pts info <test-suite>" seems to have at best a loose connection to the number of tests *actually* run when "pts batch-run <test-suite>" is executed.

      Comment

      Working...
      X