Maximizing HEVC/VP9/AV1 Video Encoding On Intel Xeon Cascade Lake With SVT + Clear Linux

[phoronix]

Phoronix: Maximizing HEVC/VP9/AV1 Video Encoding On Intel Xeon Cascade Lake With SVT + Clear Linux

Continuing on from yesterday's Linux OS comparison/benchmarks on Intel 2nd Gen Xeon Scalable Cascade Lake CPUs, here are some follow-up tests focusing on the video encode performance for this dual Xeon Platinum 8280 server when focusing on Intel's high-performance "Streaming Video Technology" (SVT) encoders for VP9, AV1, and H.265/HEVC...

http://www.phoronix.com/scan.php?page=news_item&px=Intel-SVT-Clear-Cascade-Lake

[milkylainen]

I wish the SVT suite had h264 too... Just to make it more complete and make decent software encoding a reality on far less powerful x86 hardware.

[GreenReaper]

"When feeding the same content for VP9 encoding through SVT-VP9 with the default settings, Clear Linux was running at 272 FPS but when using Clear Linux it topped at over 400 FPS!" - I think the first one here should be Ubuntu, not Clear Linux.

I'd look to system libraries for your answer. Intel has a bunch of glibc patches to make more aggressive use of AVX2, FMA and the like. They also appear to remove certain code handling edge cases that are not expected to occur frequently and which slows the fast path down.

[sdack]

I don't have the hardware here, so I don't know, but my guess is the difference will be in the compilers - Clear Linux is using GCC 8.3.1 20190405 while Ubuntu is still on GCC 7.3.0.

GCC 7.3 doesn't know -march=cascadelake and might not know what to pick when -march=native is selected. It could possibly be falling back to -march=generic as architecture. The most recent Intel architecture known by GCC 7.3 is -march=skylake and -march=skylake-avx512. It might be worth a try to select those for Ubuntu and to see if it can catch up.

It's a huge gap though ...

[miskol]

base on this
x265 has lots of AVX512 optimalisations

https://bitbucket.org/multicoreware/x265/src/b36242b9f354b8773e38674b876b0ca5dfc35ad2/source/common/x86/?at=default

but I don't know if x265 will build AVX2 and AVX512 variants in build time
and choose best match for CPU

[miskol]

michael
can you add ubuntu 19.04 beta ?

[Michael]

michael
can you add ubuntu 19.04 beta ?

As shown yesterday, Ubuntu 19.04 beta performance isn't much different from 18.04 in most areas, video encode inclided.

[Michael]

I don't have the hardware here, so I don't know, but my guess is the difference will be in the compilers - Clear Linux is using GCC 8.3.1 20190405 while Ubuntu is still on GCC 7.3.0.

GCC 7.3 doesn't know -march=cascadelake and might not know what to pick when -march=native is selected. It could possibly be falling back to -march=generic as architecture. The most recent Intel architecture known by GCC 7.3 is -march=skylake and -march=skylake-avx512. It might be worth a try to select those for Ubuntu and to see if it can catch up.

It's a huge gap though ...

It should be flaling back to skylake-avx512.

[jntesteves]

I wish the SVT suite had h264 too... Just to make it more complete and make decent software encoding a reality on far less powerful x86 hardware.

I don't think Intel's SVT will do you any good on "far less powerful x86 hardware". It's greatest advantage is on the most expensive current server hardware having avx512.

And it would most likely never come near x264's output quality anyway, so, why bother.

[coder]

It should be flaling back to skylake-avx512.

Is there a way to know? Can gcc show you what -march=native translates into?