How do you install CUDA 11 on Ubuntu 20.10 and verify the installation

Install the NVIDIA driver

This might be an optional step, but it is always good to first remove potential previously installed NVIDIA drivers:

sudo apt-get purge *nvidia*
sudo apt autoremove

Next, let's install the latest driver:

sudo apt install nvidia-driver-455

After this, we need to restart the computer to finalize the driver installation. Next we can verify whether the drive was succesfully installed:

nvidia-smi

This should contain the following or similar:

NVIDIA-SMI 455.28 Driver Version: 455.28

Install CUDA Toolkit

Next we can install the CUDA toolkit:

sudo apt install nvidia-cuda-toolkit

We also need to set the CUDA_PATH. Add this

export CUDA_PATH=/usr

at the end of your .bashrc and run

source ~/.bashrc

Now your CUDA installation should be complete, and

nvidia-smi

should indicate that you have CUDA 11.1 installed.

Test the CUDA toolkit installation /configuration

One of the best way to verify whether CUDA is properly installed is using the official "CUDA-sample". Ubuntu does not package them as part of "nvidia-cuda-toolkit" but we can download them directly from NVIDIA's github page:

wget https://github.com/NVIDIA/cuda-samples/archive/v11.1.tar.gz
tar xvf v11.1.tar.gz 
cd cuda-samples-11.1

For whatever reason, NVIDIA did not chose to include a modern build system (e.g. cmake), but ships a plain old Makefile instead. If just running "make" does not work for you, carefully read the error messages and see whether e.g. some required dependencies are not installed.

In order to help the build process a little, it might be advisable to specify the compute architecture of your GPU.

1. You can find out your GPU by running nvidia-smi. Mine is a Quadro RTX 3000.
2. Next google your GPU to find out the corresponding compute architecture. For the Quadro RTX 3000, it is "turing", version 7.5.
3. Specify the architecture version when running make, e.g.

make SMS="75"

If the compilation was succesful, you can try out one of the samples. For instance:

./bin/x86_64/linux/release/immaTensorCoreGemm 

You should see the following or similar output:

M: 4096 (16 x 256)
N: 4096 (16 x 256)
K: 4096 (16 x 256)
Preparing data for GPU...
Required shared memory size: 64 Kb
Computing... using high performance kernel compute_gemm_imma 
Time: 6.030176 ms
TOPS: 22.79

since all of the explanations i found so far were not satisfying, here are the steps i came up with to install the latest nvidia driver (465) with cuda 11.3

first you have to uninstall all cuda and nvidia related drivers and packages

sudo apt-get purge nvidia-*
sudo apt-get purge cuda*
sudo apt autoremove

then

reboot

then (if not already done) disable nouveau as described here: https://askubuntu.com/a/951892/634048 and reboot again

download cuda and install.

wget https://developer.download.nvidia.com/compute/cuda/11.3.1/local_installers/cuda_11.3.1_465.19.01_linux.run
sudo sh cuda_11.3.1_465.19.01_linux.run

Follow the post-installation instructions found on the CUDA Toolkit Installation Guide for Linux. This involves updating the PATH and environment variables:

export PATH=/usr/local/cuda-11.3/bin${PATH:+:${PATH}}
export LD_LIBRARY_PATH=/usr/local/cuda-11.3/lib64\
                 ${LD_LIBRARY_PATH:+:${LD_LIBRARY_PATH}}

IMPORTANT if you need 32bit support - there are several applications only running with 32-bit drivers (like steam)

opt out of installation of nvidia drivers for cuda installation and install drivers from here: https://www.nvidia.com/download/driverResults.aspx/171980/en-us also check if driver is compatible for your model! (in general that should be the case)

sudo sh 'NVIDIA-Linux-x86_64-465.19.01.run' 

and opt in for 32-bit support when asked

done

I did NOT test it for any other versions than 20.04, but it should work for 18.04 to 21.04