NVIDIA Jetson AGX Xavier Benchmarks - Incredible Performance On The Edge
Each year it's been quite fascinating to see the advance of NVIDIA's Tegra-powered Jetson developer boards with their increasing GPU and CPU capabilities. With the NVIDIA Jetson AGX Xavier that began shipping at the start of this quarter (as well as the AGX Xavier Module now shipping as of this month), there is a tremendous performance upgrade compared to the previous-generation Jetson TX2. I have been benchmarking the Jetson AGX Xavier the past number of weeks and continue to be surprised by its performance potential for relatively low power that makes it suitable for robotics and other AI applications. Here is a bulk of the initial benchmarks I've been running on the NVIDIA Jetson AGX with its 512-core Volta GPU and eight ARMv8.2 Carmel CPU cores.
Like the earlier Jetson developer boards, the AGX Xavier is catered towards robotics, autonomous machines, and other use-cases needing a lot of compute potential at the "edge" with NVIDIA's fairly accurate claims of 20x the performance and 10x the power efficiency compared to the Jetson TX2. The Jetson AGX Xavier features a 512-core Volta GPU complete with tensor cores, eight ARMv8.2 "Carmel" processor cores, 16GB of LPDDR4x memory, 32GB of eMMC5.1 storage, two NVDLA deep learning accelerators, and a 7-way VLIW vision processor. Those capabilities are found on the Xavier module while the Developer Kit offers up a PCIe x8 slot, Gigabit Ethernet, dual USB 3.1, one NVMe slot, 40-pin GPIO header, HDMI 2.0, and other basic connectivity for making a nice ARMv8 developer box complete with CUDA, TensorRT, and NVIDIA's other libraries.
The immense power packed in the Jetson AGX Xavier does mark a steep price increase at $2500 USD for the entire developer kit or currently the NVIDIA Developer Program offers the developer kit at $1,299 USD. The new AGX Xavier module is $1,399 or $1,099 in quantities of 1k+ units. The previous-generation Jetson TX2 is still available at around $550 USD with its dual Denver ARMv8 CPU cores and four Cortex-A57 cores paired with a NVIDIA Pascal GPU sporting 256 CUDA cores, 8GB of LPDDR4 memory, and no deep learning accelerators or tensor cores.
Setting up the Jetson AGX Xavier Developer Kit was as easy as previous Jetson boards thanks to NVIDIA's JetPack software. The current Linux4Tegra sample file-system for the AGX Xavier is based on the current Ubuntu 18.04 LTS, which is nice compared to the now dated Ubuntu 16.04 packages currently used by older versions of Jetson.
For this round of NVIDIA AGX Xavier benchmarking is a look at the TensorRT inference performance with VGG16 / AlexNet / ResNet50 / GoogleNet at INT8 and FP16 with a variety of batch sizes. Following that are a variety of CPU-only ARM Linux benchmarks just for seeing how these NVIDIA Carmel cores compare to the ARM CPU performance on other SoCs / lower-cost developer boards. Finally, there are various thermal and power consumption / performance-per-Watt metrics. All of the benchmarks in this article were run while the Jetson boards were set to their maximum performance mode.