Tweet
Hi,
Great work on Phoronoix! I'm really enjoying using it.
I'm trying to use Phoronix to compare performance characteristics of different Linux kernels. In order to ensure that my comparison isn't just looking at noise I planned to run universe-cli several times, and take an average for each kernel.
In general this is working quite well. I've only eye-balled the data, but for most tests, the high and low scores are within better than 5% of each other.
Unfortunately, I've found that the times I get from the Flexible IO Tester benchmark vary by a factor of 2, even on the same kernel. The data is below.
Run 1: 15.28 seconds
Run 2: 21.85 "
Run 3: 11.56 "
Run 4: 18.33 "
Run 5: 11.72 "
As you can see, the variance is extremely high; and I suspect that we're seeing more noise than signal. All of these times were collected using identical hardware, and an identical kernel. To rule out interference from a background process, I ran these in single user mode.
Can anyone help explain what this test is measuring? Can it be reworked to give more stable results?
Thanks!
--Jeff
Great work on Phoronoix! I'm really enjoying using it.
I'm trying to use Phoronix to compare performance characteristics of different Linux kernels. In order to ensure that my comparison isn't just looking at noise I planned to run universe-cli several times, and take an average for each kernel.
In general this is working quite well. I've only eye-balled the data, but for most tests, the high and low scores are within better than 5% of each other.
Unfortunately, I've found that the times I get from the Flexible IO Tester benchmark vary by a factor of 2, even on the same kernel. The data is below.
Run 1: 15.28 seconds
Run 2: 21.85 "
Run 3: 11.56 "
Run 4: 18.33 "
Run 5: 11.72 "
As you can see, the variance is extremely high; and I suspect that we're seeing more noise than signal. All of these times were collected using identical hardware, and an identical kernel. To rule out interference from a background process, I ran these in single user mode.
Can anyone help explain what this test is measuring? Can it be reworked to give more stable results?
Thanks!
--Jeff
Comment