--- title: "How to Install the CUDA Toolkit on Ubuntu 24.04" description: "Install the NVIDIA CUDA Toolkit and cuDNN on Ubuntu 24.04 from the official repository, set up PATH, switch between CUDA versions and compile a test program." category: "AI and ML Infrastructure" language: en source: https://cubepath.com/docs/ai-and-ml-infrastructure/cuda-toolkit-installation-and-configuration updated: 2026-09-25 --- # How to Install the CUDA Toolkit on Ubuntu 24.04 The NVIDIA CUDA Toolkit contains the `nvcc` compiler, the CUDA runtime and math libraries, and the headers needed to build GPU-accelerated software. In this tutorial you will install the toolkit from NVIDIA's official APT repository on Ubuntu 24.04, add cuDNN, configure your shell, switch between toolkit versions, and compile a small CUDA program to prove everything works. ## Prerequisites To follow this guide you need: - A server running Ubuntu 24.04 LTS with an NVIDIA GPU. - A working NVIDIA driver: `nvidia-smi` must list your GPU. If it does not, follow a driver installation guide first. - A non-root user with `sudo` privileges. - About 8 GB of free disk space for the toolkit and cuDNN. > Note: If you only plan to run PyTorch or TensorFlow installed with `pip`, you may not need the toolkit at all. Their wheels bundle the CUDA runtime and cuDNN they were built with and only require a recent enough driver. Install the system toolkit when you compile CUDA code, build extensions from source, or use software that expects `nvcc`. ## Step 1 - Choosing a CUDA version Every CUDA release needs a minimum driver version. Check the driver you have and the newest CUDA version it supports: ```bash nvidia-smi --query-gpu=name,driver_version,compute_cap --format=csv nvidia-smi | grep "CUDA Version" ``` ```output name, driver_version, compute_cap NVIDIA L40S, 580.95.05, 8.9 | NVIDIA-SMI 580.95.05 Driver Version: 580.95.05 CUDA Version: 13.0 | ``` The toolkit you install must not be newer than the `CUDA Version` shown. As a reference, CUDA 12.x works with drivers 525 and newer, and CUDA 13.x needs driver 580 or newer. Also check which version your frameworks target: the PyTorch "Get Started" page and the TensorFlow install page list the CUDA versions they support. Note the `compute_cap` value; you will use it when compiling. ## Step 2 - Adding the NVIDIA CUDA repository NVIDIA publishes a `cuda-keyring` package that installs its signing key under `/usr/share/keyrings` and the repository definition under `/etc/apt/sources.list.d`. Download and install it: ```bash wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2404/x86_64/cuda-keyring_1.1-1_all.deb sudo dpkg -i cuda-keyring_1.1-1_all.deb sudo apt update ``` On an arm64 server (for example Grace Hopper), replace `x86_64` with `sbsa` in the URL. List the toolkit versions available to you: ```bash apt-cache search --names-only '^cuda-toolkit-[0-9]+-[0-9]+$' ``` ```output cuda-toolkit-12-8 - CUDA Toolkit 12.8 meta-package cuda-toolkit-12-9 - CUDA Toolkit 12.9 meta-package cuda-toolkit-13-0 - CUDA Toolkit 13.0 meta-package ``` ## Step 3 - Installing the toolkit Install the compiler toolchain that `nvcc` relies on for host code, then the toolkit version you chose. Use the versioned `cuda-toolkit-X-Y` package: it installs only the toolkit and never touches your driver. Avoid the `cuda` and `cuda-X-Y` meta packages, which also pull in a driver. ```bash sudo apt install build-essential sudo apt install cuda-toolkit-13-0 ``` The toolkit is installed under `/usr/local/cuda-13.0`, and a `/usr/local/cuda` symlink points to the default version: ```bash ls -l /usr/local | grep cuda ``` ```output lrwxrwxrwx 1 root root 22 Sep 25 10:12 cuda -> /etc/alternatives/cuda lrwxrwxrwx 1 root root 25 Sep 25 10:12 cuda-13 -> /etc/alternatives/cuda-13 drwxr-xr-x 15 root root 4096 Sep 25 10:12 cuda-13.0 ``` ## Step 4 - Configuring your environment The toolkit's binaries are not on the default `PATH`. Add them system-wide with a profile script so every login shell finds `nvcc`: ```bash sudo nano /etc/profile.d/cuda.sh ``` ```bash export CUDA_HOME=/usr/local/cuda export PATH="$CUDA_HOME/bin:$PATH" ``` Pointing at `/usr/local/cuda` rather than a specific version means you can switch versions later without editing this file. Load it in the current session: ```bash source /etc/profile.d/cuda.sh ``` Verify the compiler: ```bash nvcc --version ``` ```output nvcc: NVIDIA (R) Cuda compiler driver Copyright (c) 2005-2025 NVIDIA Corporation Cuda compilation tools, release 13.0, V13.0.88 ``` The Debian packages register the CUDA library directory with the dynamic linker, so you normally do not need `LD_LIBRARY_PATH`. Check that the runtime library is found: ```bash ldconfig -p | grep libcudart ``` ```output libcudart.so.13 (libc6,x86-64) => /usr/local/cuda-13.0/targets/x86_64-linux/lib/libcudart.so.13 ``` If this prints nothing, add `export LD_LIBRARY_PATH="$CUDA_HOME/lib64${LD_LIBRARY_PATH:+:$LD_LIBRARY_PATH}"` to `/etc/profile.d/cuda.sh`. ## Step 5 - Compiling and running a test program A quick program that queries the GPU and runs a kernel proves that the compiler, runtime and driver all agree. Create a working directory and the source file: ```bash mkdir -p ~/cuda-test && cd ~/cuda-test nano vector_add.cu ``` ```cpp #include #include #define CHECK(call) do { cudaError_t e = (call); if (e != cudaSuccess) { \ fprintf(stderr, "CUDA error: %s (%s:%d)\n", cudaGetErrorString(e), __FILE__, __LINE__); return 1; } } while (0) __global__ void add(const float *a, const float *b, float *c, int n) { int i = blockIdx.x * blockDim.x + threadIdx.x; if (i < n) c[i] = a[i] + b[i]; } int main() { cudaDeviceProp prop; CHECK(cudaGetDeviceProperties(&prop, 0)); printf("GPU: %s, compute capability %d.%d\n", prop.name, prop.major, prop.minor); const int n = 1 << 20; float *a, *b, *c; CHECK(cudaMallocManaged(&a, n * sizeof(float))); CHECK(cudaMallocManaged(&b, n * sizeof(float))); CHECK(cudaMallocManaged(&c, n * sizeof(float))); for (int i = 0; i < n; i++) { a[i] = 1.0f; b[i] = 2.0f; } add<<<(n + 255) / 256, 256>>>(a, b, c, n); CHECK(cudaGetLastError()); CHECK(cudaDeviceSynchronize()); for (int i = 0; i < n; i++) { if (c[i] != 3.0f) { printf("FAILED at %d\n", i); return 1; } } printf("Result = PASS (%d elements)\n", n); cudaFree(a); cudaFree(b); cudaFree(c); return 0; } ``` Compile it for the GPU in this machine. `-arch=native` tells `nvcc` to detect the installed GPU's architecture: ```bash nvcc -arch=native -o vector_add vector_add.cu ./vector_add ``` ```output GPU: NVIDIA L40S, compute capability 8.9 Result = PASS (1048576 elements) ``` If you build binaries to run on other machines, target the architectures explicitly instead, for example `-arch=sm_89` for compute capability 8.9 or `-arch=sm_90` for Hopper. ## Step 6 - Installing cuDNN cuDNN is NVIDIA's library of deep learning primitives (convolutions, attention, normalization). It is available from the same repository. Install the build that matches your CUDA major version: ```bash sudo apt install cudnn9-cuda-13 ``` For a CUDA 12 toolkit, install `cudnn9-cuda-12` instead. Confirm which cuDNN packages were installed and their version: ```bash dpkg -l | grep -i cudnn ``` ```output ii cudnn9-cuda-13 9.13.0.50-1 amd64 NVIDIA cuDNN for CUDA 13 ii libcudnn9-cuda-13 9.13.0.50-1 amd64 cuDNN runtime libraries for CUDA 13 ii libcudnn9-dev-cuda-13 9.13.0.50-1 amd64 cuDNN development libraries for CUDA 13 ii libcudnn9-headers-cuda-13 9.13.0.50-1 amd64 cuDNN header files for CUDA 13 ``` Also check that the dynamic linker can find the library: ```bash ldconfig -p | grep libcudnn.so ``` ## Step 7 - Managing multiple CUDA versions Different projects often need different CUDA versions. Toolkits install side by side, each in its own `/usr/local/cuda-X.Y` directory: ```bash sudo apt install cuda-toolkit-12-9 ``` The `/usr/local/cuda` symlink is managed by `update-alternatives`. List the installed versions and choose the system default: ```bash sudo update-alternatives --config cuda ``` ```output There are 2 choices for the alternative cuda (providing /usr/local/cuda). Selection Path Priority Status ------------------------------------------------------------ * 0 /usr/local/cuda-13.0 130 auto mode 1 /usr/local/cuda-12.9 129 manual mode 2 /usr/local/cuda-13.0 130 manual mode Press to keep the current choice[*], or type selection number: ``` Because `/etc/profile.d/cuda.sh` uses `/usr/local/cuda`, open a new shell and `nvcc --version` reports the new default. To use a different version for a single session or build without changing the system default, override the variables in that shell only: ```bash export CUDA_HOME=/usr/local/cuda-12.9 export PATH="$CUDA_HOME/bin:$PATH" nvcc --version | grep release ``` ```output Cuda compilation tools, release 12.9, V12.9.86 ``` ## Troubleshooting **`nvcc: command not found`.** The toolkit is installed but not on your `PATH`. Confirm `/usr/local/cuda/bin/nvcc` exists and that you opened a new login shell or ran `source /etc/profile.d/cuda.sh`. Note that `sudo` resets `PATH`, so use the full path `/usr/local/cuda/bin/nvcc` in commands run with `sudo`. **`CUDA error: CUDA driver version is insufficient for CUDA runtime version`.** The toolkit is newer than the driver supports. Compare with the `CUDA Version` shown by `nvidia-smi`, then either upgrade the driver or switch to an older toolkit with `update-alternatives --config cuda`. **`no kernel image is available for execution on the device`.** The binary was compiled for a different GPU architecture. Rebuild with `-arch=native`, or with the `sm_XX` value that matches your `compute_cap`. **apt wants to install a driver together with the toolkit.** You installed `cuda` or `cuda-X-Y` instead of `cuda-toolkit-X-Y`. Cancel the transaction and install the `cuda-toolkit-` package so your existing driver is left alone. **PyTorch reports a different CUDA version than `nvcc`.** This is expected. `python3 -c "import torch; print(torch.version.cuda)"` shows the runtime bundled in the wheel, which is independent of the system toolkit. Only the driver has to be new enough for both. ## Conclusion You installed the CUDA Toolkit from NVIDIA's repository on Ubuntu 24.04, configured the shell through `/etc/profile.d`, added cuDNN, and confirmed the setup with a compiled vector addition kernel. Versions live side by side and `update-alternatives` controls the default. Next, you can build the official CUDA samples from NVIDIA's `cuda-samples` repository, set up a Python virtual environment with PyTorch, or install the NVIDIA Container Toolkit to run CUDA workloads in Docker.