Quickly & Easily Running Benchmarks On Docker With "phoronix/pts"

[blueweb]

Aside from the above problem of installing dependencies, some tests don't run. I tried indigobench which downloaded and installed without error, but when run it gives a message "The test run did not produce a result." The benchmark does run on my system outside of docker.

[Michael]

Aside from the above problem of installing dependencies, some tests don't run. I tried indigobench which downloaded and installed without error, but when run it gives a message "The test run did not produce a result." The benchmark does run on my system outside of docker.

There may be some corner-cases... Like with indigobench, it launches a window even when running CPU benchmark... But without X11 support, it will fail. PTS will know for tests marked 'Graphics' that if no Wayland/X11 on Linux is present, the test shouldn't run as it will fail. But in this case it's not marked as Graphics. There are a few corner cases like that.

So either needing to add more meta-data to the test profiles or the current approach of just assuming users are going to use sensible tests or explore if there are problems.

[blueweb]

Thanks for the explanation. I figured the problem might be due to the window launching aspect. Still, it would be nice to have a message instead of mysterious errors.

And is it possible to install dependencies in the docker image? Otherwise it would be good to know which tests require dependencies and thus cannot be run.

[Michael]

Thanks for the explanation. I figured the problem might be due to the window launching aspect. Still, it would be nice to have a message instead of mysterious errors.

And is it possible to install dependencies in the docker image? Otherwise it would be good to know which tests require dependencies and thus cannot be run.

Were there any other tests in particular you found mysterious on Docker? I could add like a 'RequiresDisplay' property to test profiles to enforce the behavior that is assumed when TestType = Graphics, but I think it is just indigo and maybe one or two other tests that would benefit from as such... so just a matter of weighing the benefits. But yeah always looking for ways to improve the experience more broadly.

[Michael]

Were there any other tests in particular you found mysterious on Docker? I could add like a 'RequiresDisplay' property to test profiles to enforce the behavior that is assumed when TestType = Graphics, but I think it is just indigo and maybe one or two other tests that would benefit from as such... so just a matter of weighing the benefits. But yeah always looking for ways to improve the experience more broadly.

Ended up adding a RequiresDisplay anyhow, https://github.com/phoronix-test-sui...e42c8f884aad37 so will go ahead and update indigobench shortly.

[Michael]

And pts/indigobench-1.0.2 now has the RequiresDisplay = TRUE - http://openbenchmarking.org/test/pts/indigobench

The Docker image will be respun next week btw for the official 8.2 release.

[blueweb]

Were there any other tests in particular you found mysterious on Docker?

It's more a lack of familiarity with individual tests, and knowledge of what makes sense for a given test. I think a user downloading and trying different tests isn't too uncommon, and if tests just give errors or don't run, that makes the test suite seem less user-friendly. I suppose whether this is, or should be, addressed depends on what is the core mission of PTS.

The Docker image will be respun next week btw for the official 8.2 release.

Will the release address the system dependencies issues?

[Michael]

It's more a lack of familiarity with individual tests, and knowledge of what makes sense for a given test. I think a user downloading and trying different tests isn't too uncommon, and if tests just give errors or don't run, that makes the test suite seem less user-friendly. I suppose whether this is, or should be, addressed depends on what is the core mission of PTS.

Will the release address the system dependencies issues?

What system dependency issues have you ran into in particular or with what tests? The Docker image installs Clear's package/bundle of os-testsuite-phoronix-cli, which should provide all available dependencies for the CLI/non-display PTS tests that work under Clear.

[blueweb]

What system dependency issues have you ran into in particular or with what tests? The Docker image installs Clear's package/bundle of os-testsuite-phoronix-cli, which should provide all available dependencies for the CLI/non-display PTS tests that work under Clear.

For example, hpcc does not install due to dependencies:

Some additional dependencies are required, but they could not be installed automatically for your operating system.

Below are the software packages that must be installed.

Linear Algebra Pack

Possible Package Names: liblapack-dev, lapack, lapack-devel, liblapack-devel
  
 The above dependencies should be installed before proceeding. Press any key when you're ready to continue.