Below is a table that summarizes installation and run status of the Phoronix Test Suite CPU suite.

Some background context of how this fits and where I hope to head from here…

Where I’d like to head is comparing micro-architectural features (e.g. caches, branch predictors,…), NUMA properties, OS features/policies (e.g. interrupt pinning) and other aspects with a set of benchmarks. There is however, a bit of circular dependency here. To make good comparisons, one needs good benchmarks and to get good benchmarks one either needs to have a representative sample or at least a good understanding of the range.

I don’t necessarily have these benchmarks up front, so my initial idea is to “prime the pump” by comparing a set of semi-random benchmarks on their properties and iterate with both the analysis and eventually adding other workloads.

The Phoronix Test Suite is a reasonable starting point to look at an initial set of codes to start comparing because of a few reasons:

1. It is free, with ~200 total tests including ~90 listed as “processor”
2. There is a test harness to run tests, report results, etc

On the downsides, the tests are of varying quality e.g. a number won’t build or install for variety of reasons and other have missing download files. Also, it seems many of the tests are smaller in terms of run-time, execution size, etc.

However, that is part of the idea off priming the pump to start. To make this start I wanted a number of tests – not so many that it becomes unwieldy and not so few that it becomes harder to generalize. Hence, I picked the “cpu” suite, a set of 25 benchmarks banded together.

Results of the initial install + run exercise are listed below. Of the 25, I had problems getting four benchmarks to quickly run out of the box. So I’ll skip these for now. A quick diagnosis of what prevents these four from running:

• graphics-magick fails to download with missing tarball
• gcrypt code doesn’t compile on Ubuntu 17.10, diagnosis suggests some code cleanup is required
• pgbench has security checks to prevent running as root; while might workaround by running the test as a mortal I also have some of my tools requiring a root run, skip for now
• NAS parallel benchmarks require correct install-time variables for MPI. Can be done but skip for now.

Note what also seems true is some of these microbenchmarks have known compiler optimizations (e.g. john-the-ripper) or tuning (e.g. ffmpeg, stream) needed to get absolute highest scores. The values below are for the benchmark as it comes “out of the box”, with some optimizations still pending.