NVIDIA CUDA

What is CUDA?

CUDA (Compute Unified Device Architecture) is a parallel computing platform and application programming interface (API) model created by NVIDIA. It allows software developers to use a CUDA-enabled graphics processing unit (GPU) for general-purpose processing.

Install NVIDIA Driver

Ubuntu

1. Update your server to ensure all packages are up to date:

$ sudo apt update && sudo apt upgrade -y

2. Verify Hardware Compatibility

$ lspci | grep -i nvidia

3. Add NVIDIA Repository

$ sudo add-apt-repository ppa:graphics-drivers/ppa
$ sudo apt update

4. Install Build Tools

$ sudo apt install build-essential dkms

5. Install NVIDIA Drivers

Find the recommended driver from the list of available drivers:

$ ubuntu-drivers devices

Install the recommended driver (replace <driver-version> with the specific version):

$ sudo apt install nvidia-driver-<driver-version>

6. Reboot your system:

$ sudo reboot

7. Verify Driver Installation

$ nvidia-smi

Install CUDA Toolkit

First, run nvcc --version to verify that the CUDA compiler is installed:

$ nvcc --version

CUDA 12.8 + Ubuntu 24.04

$ wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2404/x86_64/cuda-keyring_1.1-1_all.deb
$ sudo dpkg -i cuda-keyring_1.1-1_all.deb
$ sudo apt-get update
$ sudo apt-get -y install cuda-toolkit-12-8

CUDA 12.6 + Ubuntu 24.04

$ wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2404/x86_64/cuda-keyring_1.1-1_all.deb
$ sudo dpkg -i cuda-keyring_1.1-1_all.deb
$ sudo apt-get update
$ sudo apt-get -y install cuda-toolkit-12-6

CUDA 12.4 + Ubuntu 22.04

$ wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2204/x86_64/cuda-keyring_1.1-1_all.deb
$ sudo dpkg -i cuda-keyring_1.1-1_all.deb
$ sudo apt-get update
$ sudo apt-get -y install cuda-toolkit-12-4

After running the above command, you should update your .bashrc file to include the CUDA paths. For example you installed CUDA 12.4:

$ echo 'export PATH=/usr/local/cuda-12.4/bin:$PATH' >> ~/.bashrc
$ echo 'export LD_LIBRARY_PATH=/usr/local/cuda-12.4/lib64:$LD_LIBRARY_PATH' >> ~/.bashrc
$ source ~/.bashrc

Run nvcc --version again to verify that the CUDA compiler is installed:

nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2024 NVIDIA Corporation
Built on Thu_Mar_28_02:18:24_PDT_2024
Cuda compilation tools, release 12.4, V12.4.131
Build cuda_12.4.r12.4/compiler.34097967_0

Performance Monitor

It's recommended to install nvidia-smi and nvtop for monitoring GPU performance.

nvidia-smi

$ watch -n 1 nvidia-smi

• -n 1: Refresh every 1 second.

nvtop

Alternatively, you can use nvtop for a more detailed view of GPU performance:

$ sudo apt install nvtop