Targeted Intel oneAPI DPC++ Compiler Optimization Rules Out 2k+ SPEC CPU Submissions

SPEC has effectively invalidated more than two thousand SPEC CPU 2017 benchmark submissions after it was discovered the Intel oneAPI DPC++ compiler was effectively "cheating" per their standards with a targeted optimization.

Multiple folks have written in today about a number of SPEC CPU submissions now carrying this compiler notice:

"SPEC has ruled that the compiler used for this result was performing a compilation that specifically improves the performance of the 523.xalancbmk_r / 623.xalancbmk_s benchmarks using a priori knowledge of the SPEC code and dataset to perform a transformation that has narrow applicability.

In order to encourage optimizations that have wide applicability (see rule 1.4 https://www.spec.org/cpu2017/Docs/runrules.html#rule_1.4), SPEC will no longer publish results using this optimization.

This result is left in the SPEC results database for historical reference."

There are 2000~2500 submissions that appear impacted by this optimization with the Intel oneAPI DPC++ compiler.

What I've heard is that Intel oneAPI 2022.0 through 2023.0 are affected by this outlawed compiler optimization. So the very latest post-2023.0 are okay according to the SPEC rules as well as the pre-2022 data. In the benchmark test targeted by this compiler optimization it could significantly increase the result. In the grand scheme of things for the overall SPECint scoring could be impacted by several percent.

Intel's oneAPI DPC++ compiler base is open-source via intel/llvm on GitHub with being derived from LLVM/Clang but not having to follow any rigorous outside code review or the like with what you would find with community or multi-vendor open-source projects. Intel does regularly contribute to upstream LLVM (and GCC) too though in this case the issue only affects their DPC++ compiler with the hyper-scoped optimization.

Update: On receiving further information it's apparently 2,600+ results affected due to this oneAPI DPC++ compiler behavior. The SPEC CPU specific optimization could result in a 9% overall SPECint speed uplift and around 4% for the SPECint rate.