@ai-ml/nvidia-container-toolkit

In addition to this COPR, the package has now been added to Fedora as golang-github-nvidia-container-toolkit. Merging the efforts is underway.

Setup the NVIDIA kmod via RPMFusion. Ensure this is working before continuing.

Install the CUDA libraries:

$ sudo dnf install xorg-x11-drv-nvidia-cuda

Setup the COPR repo:

$ sudo dnf copr enable @ai-ml/nvidia-container-toolkit

Install the NVIDIA container toolkit:

$ sudo dnf install nvidia-container-toolkit nvidia-container-toolkit-selinux

Setup the CDI configuration:

$ sudo nvidia-ctk cdi generate -output /etc/cdi/nvidia.yaml

Test everything is working:

$ podman run --device nvidia.com/gpu=all --rm fedora:latest nvidia-smi

If everything is working, you should see something like:

Sun Jan 18 18:17:35 2026       
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 580.119.02             Driver Version: 580.119.02     CUDA Version: 13.0     |
+-----------------------------------------+------------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id          Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |           Memory-Usage | GPU-Util  Compute M. |
|                                         |                        |               MIG M. |
|=========================================+========================+======================|
|   0  NVIDIA GeForce RTX 3080 Ti     Off |   00000000:08:00.0  On |                  N/A |
|  0%   43C    P8             36W /  350W |     882MiB /  12288MiB |      6%      Default |
|                                         |                        |                  N/A |
+-----------------------------------------+------------------------+----------------------+

+-----------------------------------------------------------------------------------------+
| Processes:                                                                              |
|  GPU   GI   CI              PID   Type   Process name                        GPU Memory |
|        ID   ID                                                               Usage      |
|=========================================================================================|
|  No running processes found                                                             |
+-----------------------------------------------------------------------------------------+

Test the Difference Between CPU and GPU Performance

Download a PyTorch environment and ensure it runs:

$ podman run --device nvidia.com/gpu=all --rm docker.io/pytorch/pytorch:latest

Test CPU:

$ time podman run --device nvidia.com/gpu=all --rm docker.io/pytorch/pytorch:latest bash -c 'pip install pytorch-benchmark && python -c "import torch; import json; from torchvision.models import efficientnet_b0; from pytorch_benchmark import benchmark; model = efficientnet_b0().to(\"cpu\"); sample = torch.randn(8, 3, 224, 224); results = benchmark(model, sample, num_runs=200); print(json.dumps(results, indent=4))"'

Test GPU:

$ time podman run --device nvidia.com/gpu=all --rm docker.io/pytorch/pytorch:latest bash -c 'pip install pytorch-benchmark && python -c "import torch; import json; from torchvision.models import efficientnet_b0; from pytorch_benchmark import benchmark; model = efficientnet_b0().to(\"cuda\"); sample = torch.randn(8, 3, 224, 224); results = benchmark(model, sample, num_runs=200); print(json.dumps(results, indent=4))"'

Test SELinux Policy

Please note: We changed the package to toggle container_use_xserver_devices instead of container_use_devices. If the package toggled container_use_devices, it will be set back to default automatically if the user had not already changed this setting manually.

Optionally, test if the shipped NVIDIA DGX SELinux policy is working (the nvidia-container-toolkit-selinux package enables container_use_devices by default) :

$ sudo setsebool container_use_xserver_devices 0
$ podman run --device nvidia.com/gpu=all \
             --security-opt label=type:nvidia_container_t \
             --rm fedora:latest nvidia-smi

Re-run testing and then reset SELinux.

$ sudo setsebool container_use_xserver_devices 1

Monitoring CPU vs GPU Usage

Use the nvtop utility to monitor what load is on your CPU/GPU:

$ sudo dnf install nvtop
$ nvtop

Something Fun, Image Generation Using InvokeAI

Use InvokeAI to generate prompt based images using multiple models.

Run the server:

$ podman run --device nvidia.com/gpu=all \
             -p 9090:9090 \
             --rm ghcr.io/invoke-ai/invokeai

Visit the web UI at 0.0.0.0:9090 and start generating images based on prompts.

Please note, the models are downloaded into the container and will be deleted after the container is stopped.