GPU Setup for KDB-X¶

This guide explains how to configure and verify a GPU-capable environment for running KDB-X GPU-accelerated workloads.

Prerequisites¶

Docker (optional but recommended)
Linux host with CUDA Enabled Data Centre GPUs
NVIDIA drivers able to run CUDA 13.1 applications

Overview of CUDA enviroment¶

The CUDA software environment consists of three parts:

CUDA Toolkit (runtime, libraries, tools) - user-mode SDK used to build CUDA applications
CUDA driver - user-mode driver component used to run CUDA applications (libcuda.so)
NVIDIA GPU device driver - kernel-mode driver component for NVIDIA GPUs (nvidia.ko)

On Linux systems, the CUDA driver and kernel mode components are delivered together in the NVIDIA display driver package, as shown below:

The CUDA Toolkit is generally optional when GPU nodes are only used to run applications (but is required to develop and build applications).

While the KDB-X module does not specifically require toolkit installation on the host to be able to run, it does have dependencies on several shared CUDA libraries. These can be made available to the application by various methods (one being via the toolkit).

The purpose of this document is to provide users guidance on the easiest way to run the KDB-X GPU module CUDA application. It will address:

host installation of the required drivers compatible with the KDB-X module CUDA build
instructions on how to run KDB-X application in an environment that has the required CUDA shared libraries

NOTE: In order to run any CUDA application, at a minimum the system must have a CUDA-enabled GPU and an NVIDIA driver that is compatible with the CUDA Toolkit used to build the applications. If the CUDA Toolkit (including shared libraries) has been previously installed on the host, it is likely the drivers have also been installed but you will need to check compatibility against the KDB-X CUDA build.

CUDA Enabled GPUs¶

To run the module, the host machine must have a CUDA enabled Data Centre GPU(s) available. Check if a CUDA-Capable GPU is available on the host:

lspci | grep -i nvidia

38:00.0 3D controller: NVIDIA Corporation AD104GL [L4] (rev a1)
3a:00.0 3D controller: NVIDIA Corporation AD104GL [L4] (rev a1)
3c:00.0 3D controller: NVIDIA Corporation AD104GL [L4] (rev a1)
3e:00.0 3D controller: NVIDIA Corporation AD104GL [L4] (rev a1)

In this example, the L4 card is available, which is a a data centre type card.

NOTE: If you do not see any settings, contact your system administrator.

A list of NVIDIA CUDA-Capable GPUs are available from CUDA GPU Compute Capability.

Check existing drivers via NVIDIA SMI (System Management Interface)¶

As mentioned above, the CUDA Toolkit and/or drivers may already be installed. To verify, run the followimg command:

nvidia-smi

If the GPUs are displayed, the drivers are already installed, as the following example shows:

+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 590.48.01              Driver Version: 590.48.01      CUDA Version: 13.1     |
+-----------------------------------------+------------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id          Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |           Memory-Usage | GPU-Util  Compute M. |
|                                         |                        |               MIG M. |
|=========================================+========================+======================|
|   0  NVIDIA L4                      Off |   00000000:38:00.0 Off |                    0 |
| N/A   38C    P0             27W /   72W |       0MiB /  23034MiB |      0%      Default |
|                                         |                        |                  N/A |
+-----------------------------------------+------------------------+----------------------+
|   1  NVIDIA L4                      Off |   00000000:3A:00.0 Off |                    0 |
| N/A   36C    P0             27W /   72W |       0MiB /  23034MiB |      0%      Default |
|                                         |                        |                  N/A |
+-----------------------------------------+------------------------+----------------------+
|   2  NVIDIA L4                      Off |   00000000:3C:00.0 Off |                    0 |
| N/A   40C    P0             28W /   72W |       0MiB /  23034MiB |      2%      Default |
|                                         |                        |                  N/A |
+-----------------------------------------+------------------------+----------------------+
|   3  NVIDIA L4                      Off |   00000000:3E:00.0 Off |                    0 |
| N/A   38C    P0             27W /   72W |       0MiB /  23034MiB |      0%      Default |
|                                         |                        |                  N/A |
+-----------------------------------------+------------------------+----------------------+

+-----------------------------------------------------------------------------------------+
| Processes:                                                                              |
|  GPU   GI   CI              PID   Type   Process name                        GPU Memory |
|        ID   ID                                                               Usage      |
|=========================================================================================|
|  No running processes found                                                             |
+-----------------------------------------------------------------------------------------+

If you see nvidia-sm not found, the drivers are not installed and you should proceed to the next section.
If a driver is installed, you still need to check forward compatibility.

Driver dependencies and CUDA compatibility¶

The KDB-X GPU module is compatible with the CUDA Toolkit 13.1.

Backwards compatibility ensures that a newer NVIDIA driver can be used to run an application built on an older CUDA Toolkit. We recommend installing the latest driver. The following links provide guidance on installing version v590 for your OS:

Driver Installation Guide - v590

For driver details, see Data Center driver documentation. For reference, a list of all driver versions can be seen here.

You will now need to install required drivers for a compatible host OS.

For more guidance, see these examples for installation and verification: * Ubuntu OS CUDA Drivers * RedHat OS CUDA Drivers

Forward compatibility of existing drivers¶

If GPU drivers are already installed on the host, you still need to check forward compatibility for CUDA 13.1 and you may need to install the CUDA forward compatibility package if a full driver update is not possible.

Fabric Manager¶

GPUs that use NVSwitch/NVLink for GPU-to-GPU communication, such as H100's, require Fabric Manager.

The host needs to install and start the fabric manager service. The installation process is dependent on your hardware and deployment model - for example, the instructions are linked here for bare metal mode

CUDA Library dependencies¶

In addition to the drivers, the KDB-X module is dynamically linked to the CUDA runtime and multiple other CUDA libraries. These can be provided by the following various methods.

Required CUDA libraries¶

gpu.li64.so has direct dynamic dependencies on three CUDA Toolkit libraries. They are not bundled with the module, and the dynamic loader resolves them when the module is loaded. They are therefore required even if a process never calls a tensor API.

Library	SONAME the module needs	Used for
cuBLAS	`libcublas.so.13`	Tensor matrix multiplication (`mmu`) and related linear algebra
cuSOLVER	`libcusolver.so.12`	Tensor `inv`, `chol` and `lsq`
cuRAND	`libcurand.so.10`	GPU tensor random generation (`.gpu.tensor.random`)

The module requires the CUDA Toolkit 13.1 runtime libraries. Install the 13.1 set to match. A newer 13.x runtime normally provides the same SONAMEs, but 13.1 is the tested configuration.

You can list the libraries a given module file depends on with:

readelf -d gpu.li64.so | grep NEEDED

Installing the cuBLAS, cuSOLVER and cuRAND runtime libraries (Ubuntu)¶

These steps assume the NVIDIA CUDA apt repository is configured as in Install drivers above.

Install the CUDA 13.1 library set, which includes cuBLAS, cuRAND, cuSOLVER and their supporting libraries (cuBLASLt, cuSPARSE, nvJitLink):

sudo apt update
sudo apt install cuda-libraries-13-1

If you need only the three packages, install them explicitly:

sudo apt install libcublas-13-1 libcusolver-13-1 libcurand-13-1

The libraries install under /usr/local/cuda-13.1/lib64. They are usually found without further configuration because /usr/local/cuda links to the installed toolkit. If several toolkits are installed, set LD_LIBRARY_PATH to the 13.1 lib64 directory before loading the module:

export LD_LIBRARY_PATH=/usr/local/cuda-13.1/lib64:$LD_LIBRARY_PATH

Verifying the dependencies¶

Check that every dependency resolves:

ldd gpu.li64.so | grep "not found"

This command should produce no output. If it lists any library, the module will fail to load with an error such as error while loading shared libraries: libcublas.so.13: cannot open shared object file. Install the missing package or correct LD_LIBRARY_PATH and check again.

Supporting libraries and containers¶

cuBLAS, cuSOLVER and cuRAND also load supporting Toolkit libraries such as cuBLASLt, cuSPARSE and nvJitLink. The Toolkit runtime package or image supplies the compatible set.

For containers, use the NVIDIA CUDA runtime image. The base image does not include cuBLAS, cuSOLVER or cuRAND, so it is not sufficient for this module, even though it is sufficient for the CUDA runtime alone.

NVIDIA Containers¶

We recommend running the KDB-X application via NVIDIA containers, which provide the required dependencies for running applications built for a specific CUDA toolkit version (see below).

The host running the CUDA container only requires the NVIDIA driver (previously installed above); the CUDA toolkit doesn’t have to be installed. This ensures the isolation of the dependencies required for a particular CUDA version. You do not need to install and configure, or more importantly, update any existing CUDA libraries used by other host applications (which may be running a different CUDA version).

CUDA applications have backward compatibilty with the latest drivers, which means any driver updates performed on the host will still work with the containerized CUDA application.

Running application on the Host (non-containerized)¶

You may want to run the application on the host environment for the following reasons:

Docker deamon is not available on host
Pulling and using docker images is not preferred
CUDA toolkit is already installed/will be installed on host

In this case, you must ensure the CUDA Toolkit CUDA 13.1 is available and/or CUDA 13.1 runtime and shared libraries are installed for use with the KDB-X app. For more information, see the CUDA Installation Guide for Linux - v13.1.

NVIDIA Docker images¶

CUDA images come in three flavors via the following tag nvidia/cuda:<version>-<flavor>-<os>:

base — minimal - libcudart (runtime) and compatibility libs to deploy pre-build applications
runtime — adds all the shared libraries from the CUDA toolkit for running pre-built applications using multiple CUDA libraries (example KDB-X GPU application)
devel — adds headers and nvcc toolchain (only if you need to compile)

We recommend the runtime image is used to run the KDB-X application. The base image alone does not provide the cuBLAS/cuSOLVER/cuRAND libraries required by tensor matrix and random operations.

NVIDIA Container Toolkit¶

The container image only packages the user-mode libraries. It cannot include the kernel-mode as the kernel module must match the host's exact Linux kernel version, which containers share.

// double check kernel-mode not kernal-mode

The NVIDIA container toolkit is installed to allow the docker runtime to bind the host's devices to the container at launch. This is required to enable the provided container CUDA libraries to communicate with installed host drivers. GPU enabled containers can then run with the --gpus flag to instruct the container to use the binded devices.

You should now install the container toolkit for your host OS.

For more guidance, see these examples for installation and verification: * Ubuntu OS Container Toolkit * Redhat OS Container Toolkit

Running a KDB-X CUDA application¶

With the environment now correctly configured, you can install the specific KDB-X components and run the application as per Running a KDB-X GPU Application.

Example install and component overview (Ubuntu)¶

This section provides an example installation on an Ubuntu 24.04 operating system which uses L4 data center GPUs. Its purpose is to walkthrough the steps while explaining the components being installed and the location of important files. While the steps and details will vary for other OSes, the dependencies remain the same.

Driver install¶

NVIDIA driver components¶

To run the provided application, the following CUDA drivers are needed:

CUDA user-mode driver (libcuda.so) - what the kdb-x application calls
GPU kernel-mode driver (nvidia.ko) - talks directly to GPU hardware (termed the kernel module)

Example dependency call chain:

KDB-X App → libcudart.so (CUDA Runtime API — high-level) → libcuda.so (CUDA Driver API — low-level, talks to kernel) → nvidia.ko (kernel module) → GPU

NOTE: The CUDA runtime libcudart.so is often statically linked and packaged with CUDA application, which is not the case with KDB-X apps.

Kernel driver module overview¶

As detailed above, the driver installation provides the user-mode driver libcuda.so and kernel-mode driver nvidia.ko.

The kernel-mode driver is the core module that is installed as part of other kernel modules:

nvidi.ko - Core driver — talks to the GPU hardware
nvidia_modeset.ko - Display/mode setting support
nvidia_drm.kp - DRM (Direct Rendering Manager) integration
nvidia_uvm.ko - Unified Virtual Memory — needed for CUDA

The kernel will load whatever modules are required for the application. For headless CUDA applications, at least nvidia and nvidia_uvm will be loaded.

The kernel modules are built during driver installation to the current kernel version, in our example:

echo $(uname -r)
6.14.0-1018-aws

Kernel versions and DKMS¶

If the kernel updates, the modules built for the old kernel will no longer work, requiring a manual reinstall of the driver. DKMS (Dynamic Kernel Module Support) is a framework that automatically rebuilds kernel modules on kernel updates.

Kernel update installed → DKMS detects new kernel → Rebuilds nvidia.ko against new kernel headers → Module ready after reboot

NVIDIA driver packages on most distros use DKMS by default to handle kernel module building.

Install drivers¶

Ensure headers are available to build kernel modules:

apt install linux-headers-$(uname -r)
ls -la /usr/src/linux-headers-$(uname -r)

You should see a directory full of header files and a Makefile. If that directory doesn't exist, the headers aren't installed.

Install the latest drivers from your OS package manager:

wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2204/x86_64/cuda-keyring_1.1-1_all.deb
sudo dpkg -i cuda-keyring_1.1-1_all.deb
sudo apt update
sudo apt install cuda-drivers

Verify installation¶

Check kernel modules have been built:

sudo ls -la /lib/modules/$(uname -r)/updates/dkms

drwxr-xr-x 2 root root     4096 Feb 12 11:51 .
drwxr-xr-x 3 root root     4096 Feb 12 11:51 ..
-rw-r--r-- 1 root root    73533 Feb 12 11:51 nvidia-drm.ko.zst
-rw-r--r-- 1 root root   721700 Feb 12 11:51 nvidia-modeset.ko.zst
-rw-r--r-- 1 root root     2192 Feb 12 11:51 nvidia-peermem.ko.zst
-rw-r--r-- 1 root root   817700 Feb 12 11:51 nvidia-uvm.ko.zst
-rw-r--r-- 1 root root 83713125 Feb 12 11:51 nvidia.ko.zst

If this directory is not present or empty, the drivers have not been succesfully built.

Check kernel modules are loaded:

lsmod | grep nvidia
nvidia_uvm           2076672  0
nvidia              99549184  1 nvidia_uvm

If the command returns empty, the modules are not loaded and you need to reboot the system.

Check user-mode drivers installed:

sudo find / -name "libcuda.so*" 2>/dev/null
/usr/lib/x86_64-linux-gnu/libcuda.so.1
/usr/lib/x86_64-linux-gnu/libcuda.so.590.48.01
/usr/lib/x86_64-linux-gnu/libcuda.so

If the command returns empty, the modules are not installed.

// reboot advised?

Check you can see GPUs:

nvidia-smi

If successful, this command will display GPUs as per above section.

Container toolkit install¶

Add repository and install¶

curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg
curl -s -L https://nvidia.github.io/libnvidia-container/stable/deb/nvidia-container-toolkit.list | \
  sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#g' | \
  sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list
sudo apt update
sudo apt install nvidia-container-toolkit

Configure docker:

sudo nvidia-ctk runtime configure --runtime=docker
sudo systemctl restart docker

Run a sample workload¶

docker run --rm --gpus all nvidia/cuda:13.1.0-runtime-ubuntu24.04 nvidia-smi

If successful, this command will display GPUs as per above section.

Inspect Container libraries¶

The libraries provided and available for CUDA applications running in the container can be listed as follows:

ls -la /usr/local/cuda-13.1/lib64/lib

libOpenCL.so.1                libcufft.so.12                libcurand.so.10.4.1.34        libnppial.so.13               libnppig.so.13.0.2.21         libnpps.so.13                 libnvjpeg.so.13.0.2.28
libOpenCL.so.1.0              libcufft.so.12.1.0.31         libcusolver.so.12             libnppial.so.13.0.2.21        libnppim.so.13                libnpps.so.13.0.2.21          libnvrtc-builtins.so.13.1
libOpenCL.so.1.0.0            libcufftw.so.12               libcusolver.so.12.0.7.41      libnppicc.so.13               libnppim.so.13.0.2.21         libnvJitLink.so.13            libnvrtc-builtins.so.13.1.80
libcublas.so.13               libcufftw.so.12.1.0.31        libcusolverMg.so.12           libnppicc.so.13.0.2.21        libnppist.so.13               libnvJitLink.so.13.1.80       libnvrtc.so.13
libcublas.so.13.2.0.9         libcufile.so.0                libcusolverMg.so.12.0.7.41    libnppidei.so.13              libnppist.so.13.0.2.21        libnvblas.so.13               libnvrtc.so.13.1.80
libcublasLt.so.13             libcufile.so.1.16.0           libcusparse.so.12             libnppidei.so.13.0.2.21       libnppisu.so.13               libnvblas.so.13.2.0.9         libnvtx3interop.so
libcublasLt.so.13.2.0.9       libcufile_rdma.so.1           libcusparse.so.12.7.2.19      libnppif.so.13                libnppisu.so.13.0.2.21        libnvfatbin.so.13             libnvtx3interop.so.1
libcudart.so.13               libcufile_rdma.so.1.16.0      libnppc.so.13                 libnppif.so.13.0.2.21         libnppitc.so.13               libnvfatbin.so.13.1.80        libnvtx3interop.so.1.1.0
libcudart.so.13.1.80          libcurand.so.10               libnppc.so.13.0.2.21          libnppig.so.13                libnppitc.so.13.0.2.21        libnvjpeg.so.13

User mode drivers are available from both the image and those injected by the container toolkit at runtime for use with the kernel drivers.

find / -name "libcuda.so*" 2>/dev/null

/usr/lib/x86_64-linux-gnu/libcuda.so.1
/usr/lib/x86_64-linux-gnu/libcuda.so.590.48.01
/usr/lib/x86_64-linux-gnu/libcuda.so
/usr/local/cuda-13.1/compat/libcuda.so.1
/usr/local/cuda-13.1/compat/libcuda.so.590.44.01
/usr/local/cuda-13.1/compat/libcuda.so

Note: The host's libcuda.so overrides whatever version is shipped with the container image.

Example install and component Overview (Red Hat)¶

This section provides an example installation on a Rocky-Linux 10 operating system which use L4 data center GPUs. Its purpose is to walkthrough the steps while explaining the components being installed and the location of important files. While the steps and details will vary for other OSes, the dependencies remain the same.