Specify NVIDIA Runtime in Docker Compose: A Complete Guide (2026)

Running GPU-accelerated applications using Docker is a common requirement for developers working with machine learning frameworks like TensorFlow. While it's straightforward to use the --runtime=nvidia flag from the command line, configuring this within a docker-compose.yml file can be less obvious. In this guide, you'll learn how to specify the NVIDIA runtime in Docker Compose, ensuring your GPU resources are utilized efficiently.

Key Takeaways

• Learn how to configure the NVIDIA runtime in Docker Compose.
• Understand the prerequisites for setting up GPU access.
• Explore the architecture of NVIDIA Docker runtime.
• Troubleshoot common errors in GPU-enabled Docker setups.

Introduction

Docker Compose simplifies the process of managing multi-container Docker applications, but when it comes to GPU-accelerated applications, things can get a bit tricky. If you've been manually running containers with GPU access, you might wonder how to transition this setup to Docker Compose. This tutorial will guide you step-by-step through the process, ensuring you can leverage your GPU efficiently with Docker Compose.

Utilizing the NVIDIA runtime in Docker Compose is crucial for applications that require heavy computation, such as those in AI and machine learning. By the end of this guide, you'll be well-equipped to configure your Docker Compose files to run GPU-enabled applications effortlessly.

Prerequisites

• Basic understanding of Docker and Docker Compose.
• A system with an NVIDIA GPU and CUDA drivers installed.
• Docker and Docker Compose installed on your machine.
• NVIDIA Container Toolkit installed.

Step 1: Verify NVIDIA Docker Setup

Before proceeding with Docker Compose, ensure your Docker setup supports GPU runtime.

# Check if NVIDIA Docker runtime is available
docker run --rm --gpus all nvidia/cuda:11.0-base nvidia-smi

This command should display your GPU details if the setup is correct.

Step 2: Update Docker Daemon Configuration

Update your Docker daemon configuration to enable the NVIDIA runtime by default.

{
  "default-runtime": "nvidia",
  "runtimes": {
    "nvidia": {
      "path": "/usr/bin/nvidia-container-runtime",
      "runtimeArgs": []
    }
  }
}

Save this configuration to /etc/docker/daemon.json and restart Docker.

Step 3: Create Your Docker Compose File

In your project directory, create a docker-compose.yml file with the following content:

version: '3.8'
services:
  tensorflow:
    image: gcr.io/tensorflow/tensorflow:latest-gpu
    deploy:
      resources:
        reservations:
          devices:
            - capabilities: [gpu]

This file specifies the use of GPU resources for the TensorFlow container.

Step 4: Deploy the Docker Compose Stack

Run the following command to deploy your stack:

docker-compose up -d

This command starts your container with GPU access according to the specifications in your docker-compose.yml.

Common Errors/Troubleshooting

• Ensure that the NVIDIA Container Toolkit is installed and configured correctly.
• Verify that your Docker version supports GPU runtime.
• Check for typos in your docker-compose.yml file, especially in the capabilities section.
• Ensure your NVIDIA drivers are up to date.

Frequently Asked Questions

Can I use Docker Compose without modifying the Docker daemon?

Yes, you can specify the runtime in the Docker Compose file using the --gpus key for specific services.

Why is my container not detecting the GPU?

Ensure that the NVIDIA Container Toolkit is properly installed and that the drivers are up to date.

Is it possible to run multiple GPU containers with Docker Compose?

Yes, Docker Compose allows running multiple services, each with its own GPU resource allocation.

Frequently Asked Questions

Can I use Docker Compose without modifying the Docker daemon?

Yes, you can specify the runtime in the Docker Compose file using the --gpus key for specific services.

Why is my container not detecting the GPU?

Ensure that the NVIDIA Container Toolkit is properly installed and that the drivers are up to date.

Is it possible to run multiple GPU containers with Docker Compose?

Yes, Docker Compose allows running multiple services, each with its own GPU resource allocation.