Announcement

Collapse
No announcement yet.

NVIDIA Releases TensorRT 8.0 With Big Performance Improvements

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • NVIDIA Releases TensorRT 8.0 With Big Performance Improvements

    Phoronix: NVIDIA Releases TensorRT 8.0 With Big Performance Improvements

    NVIDIA today is making available a much faster version of TensorRT, its SDK for optimized deep learning inference on their GPUs...

    Phoronix, Linux Hardware Reviews, Linux hardware benchmarks, Linux server benchmarks, Linux benchmarking, Desktop Linux, Linux performance, Open Source graphics, Linux How To, Ubuntu benchmarks, Ubuntu hardware, Phoronix Test Suite

  • #2
    Cool. It's been available in EA for months. OTOH still no CUDA 11.4 compatibility, which has been officially available for 3 weeks, which makes this TensorRT release even more pointless.
    Last edited by anarki2; 20 July 2021, 07:16 PM.

    Comment


    • #3
      Originally posted by anarki2 View Post
      Cool. It's been available in EA for months. OTOH still no CUDA 11.4 compatibility, which has been officially available for 3 weeks, which makes this TensorRT release even more pointless.
      See https://docs.nvidia.com/deeplearning....html#fntarg_2. CUDA 11.4 is supported by the GA now if you have a 470.xx driver. Making future CUDA versions work more seamlessly is something being worked on by the CUDA and TensorRT teams.

      Comment


      • #4
        Originally posted by zeroepoch View Post

        See https://docs.nvidia.com/deeplearning....html#fntarg_2. CUDA 11.4 is supported by the GA now if you have a 470.xx driver. Making future CUDA versions work more seamlessly is something being worked on by the CUDA and TensorRT teams.
        No it does not work. The TensorRT deb package won't install as it depends on the CUDA 11.3 deb packages. If you go to the TensorRT download page, it also doesn't list a 11.4 version, only 11.3 and below.

        The page you linked doesn't mention CUDA 11.4 either, so I'm not sure what you're talking about tbh.

        If you mean it's your empirical experience that if you manually extract the TensorRT 8 tarball somewhere and forcefully blend it with CUDA 11.4 and it appears to work for your specific use case, that's a different story. It is called "luck", not "support".
        Last edited by anarki2; 21 July 2021, 09:22 AM.

        Comment


        • #5
          Originally posted by anarki2 View Post
          No it does not work. The TensorRT deb package won't install as it depends on the CUDA 11.3 deb packages.
          You're correct. You need to use the tarfile package and you also need the 470 driver as documented in that note. The Debian packages won't work because of the NVRTC version and it being pinned to 11.3 or less due to packaging requirements. It may not work for all cases, but you don't need to have CUDA 11.3 installed to use TensorRT in this scenario.

          Comment


          • #6
            Actually, I stand corrected, the very latest GA release, while officially only supporting 11.3, now proceeds to install with CUDA 11.4 installed, so shame on me.

            Comment

            Working...
            X