Phoronix Test Suite 8.2 Released With Official Docker Benchmarking Image, New Testing Improvements

[phoronix]

Phoronix: Phoronix Test Suite 8.2 Released With Official Docker Benchmarking Image, New Testing Improvements

Phoronix Test Suite 8.2.0-Rakkestad is officially shipping today as the latest quarterly feature release to our open-source, cross-platform automated benchmarking software. In addition to the Phoronix Test Suite 8.2 itself, there is the first official release of the newly-introduced "phoronix/pts" Docker image on Docker Hub for easily carrying out the benchmarks with the pre-seeded image.

http://www.phoronix.com/vr.php?view=26847

[perpetually high]

locked and loaded, Michael



ps: the 4.20GHz was purely coincidental

[varikonniemi]

Impressive work, and the docker image is a nice addition to the toolbox.

[Michael]

Impressive work, and the docker image is a nice addition to the toolbox.

Thanks!

[polarathene]

Thanks!

Is there a comparison of running PTS natively/locally vs the Docker image? Any differences or overhead? It's only useful for Linux right? On Windows it'd presumably use a VM taking more of a hit(maybe not if using a Windows based Docker container?), no idea about macOS?

[Michael]

Is there a comparison of running PTS natively/locally vs the Docker image? Any differences or overhead? It's only useful for Linux right? On Windows it'd presumably use a VM taking more of a hit(maybe not if using a Windows based Docker container?), no idea about macOS?

Will have some published soon.

[starshipeleven]

Is there a comparison of running PTS natively/locally vs the Docker image? Any differences or overhead? It's only useful for Linux right? On Windows it'd presumably use a VM taking more of a hit(maybe not if using a Windows based Docker container?), no idea about macOS?

Afaik Docker has near-bare-metal performance, as that's basically an application running in a fancier "chroot"-like thing, but it's still using the same kernel as the host.

On Windows you can run Docker but it uses a Linux kernel running as an application inside of Windows to run the docker image (they call it "linuxkit"), it's not anywhere as heavy as a full-blown VM, but it's not bare metal.

I think they are doing the same for macOS, so its supported but it's again run by a linux kernel, either linuxkit or some kind of VM. I don't know the details.

[blueweb]

Basic docker question: is there a way to update the pts docker container (with installed pts tests) to the latest pts docker image version? Or do I have to create a new container from the new image and download tests all over again?

[polarathene]

Afaik Docker has near-bare-metal performance, as that's basically an application running in a fancier "chroot"-like thing, but it's still using the same kernel as the host.

On Windows you can run Docker but it uses a Linux kernel running as an application inside of Windows to run the docker image (they call it "linuxkit"), it's not anywhere as heavy as a full-blown VM, but it's not bare metal.

I think they are doing the same for macOS, so its supported but it's again run by a linux kernel, either linuxkit or some kind of VM. I don't know the details.

I haven't followed it's development on macOS/Windows, but I remember back in 2016 it used to run with VirtualBox VM originally, then the boot2docker minimal VM, and now the linuxkit thing?

I'd still be interested in a comparison of bare-metal vs container, I think there have been benchmarks in the past of some services like that where Docker perf was impacted by containerizing some services, so it'd be especially of interest when it comes to benchmarks. GPU benchmarks too, they don't have access to the GPU afaik, that requires the nvidia docker container if you have nvidia card, not sure about Intel or AMD support. Similar for anything that would usually render some output to the display like a game benchmark may be impacted by running in a headless benchmark? I think disk I/O can be affected as well(especially if going through a VM on macOS/Windows depending on the I/O settings they have setup).

[blueweb]

Basic docker question: is there a way to update the pts docker container (with installed pts tests) to the latest pts docker image version? Or do I have to create a new container from the new image and download tests all over again?

It looks like updating containers when images change isn't really possible. Containers are supposed to be stateless after all, right?

So maybe the tests can be installed to a volume mounted from the host so that installed tests remain when updating the pts image and container, which can then also remain stateless? Just a thought, though I'm not a docker expert.

I think the 1) persistence of tests, as well as 2) installation of system dependencies are two issues that need addressing to make the pts docker image fully usable. That is, unless it isn't meant as a replacement for the natively installed pts.