Docker Compose: Complete Guide with Practical Examples

Docker Compose simplifies multi-container application orchestration by defining and running complex applications using a single YAML configuration file. This comprehensive guide covers everything from basic concepts to advanced production deployments with real-world examples.

Table of Contents

Introduction to Docker Compose

Docker Compose is a tool for defining and running multi-container Docker applications. Instead of managing containers individually with lengthy docker run commands, Compose uses a YAML file to configure all application services, networks, and volumes in one place.

Why Use Docker Compose?

  • Simplified Management: Define entire application stack in one file
  • Reproducibility: Share configurations with your team
  • Development Efficiency: Start entire stack with single command
  • Environment Consistency: Same configuration across dev, test, and staging
  • Service Dependencies: Define startup order and dependencies
  • Easy Scaling: Scale services up or down with one command

Docker Compose vs Kubernetes

  • Docker Compose: Single-host development and small deployments
  • Kubernetes: Multi-host production orchestration at scale

Prerequisites

Before starting, ensure you have:

  • Docker Engine installed (version 20.10 or higher)
  • Docker Compose installed (version 2.0 or higher)
  • Basic Docker knowledge (images, containers, volumes)
  • Text editor for YAML files
  • Understanding of your application architecture

Verify installation:

# Check Docker Compose version
docker compose version

# Or older standalone version
docker-compose --version

Installation Notes

Docker Compose v2 is now integrated as a Docker CLI plugin:

# New syntax (recommended)
docker compose up

# Old syntax (standalone)
docker-compose up

This guide uses the new docker compose syntax.

Docker Compose Basics

Creating Your First Compose File

Create a file named docker-compose.yml:

version: '3.8'

services:
  web:
    image: nginx:alpine
    ports:
      - "8080:80"

Start the application:

docker compose up

Stop the application:

docker compose down

Basic Workflow

# Start services
docker compose up -d

# View running services
docker compose ps

# View logs
docker compose logs

# Stop services
docker compose down

Compose File Structure

Version Specification

# Compose file format version
version: '3.8'

Note: Version 3.8 is widely supported. Newer versions (3.9, 3.10) add features.

Top-Level Keys

version: '3.8'

services:      # Define containers
  service1:
  service2:

networks:      # Define networks
  network1:

volumes:       # Define volumes
  volume1:

configs:       # Define configs (Swarm mode)
  config1:

secrets:       # Define secrets (Swarm mode)
  secret1:

Minimal Example

version: '3.8'

services:
  web:
    image: nginx:alpine
    ports:
      - "80:80"

Service Configuration

Image-Based Services

services:
  database:
    image: postgres:15-alpine
    environment:
      POSTGRES_PASSWORD: secret
    ports:
      - "5432:5432"

Build-Based Services

services:
  app:
    build: .                    # Build from current directory
    ports:
      - "3000:3000"

  api:
    build:
      context: ./api           # Build context
      dockerfile: Dockerfile.prod  # Custom Dockerfile
      args:                    # Build arguments
        NODE_ENV: production

Container Naming

services:
  web:
    container_name: my-web-server
    image: nginx:alpine

Port Mapping

services:
  web:
    image: nginx
    ports:
      - "8080:80"              # host:container
      - "8443:443"
      - "127.0.0.1:9000:9000"  # bind to specific interface
      - "3000-3005:3000-3005"  # port range

Environment Variables

services:
  app:
    image: node:18-alpine
    environment:
      NODE_ENV: production
      API_KEY: ${API_KEY}      # From host environment
      DATABASE_URL: postgres://db:5432/mydb

Environment Files

services:
  app:
    image: node:18-alpine
    env_file:
      - .env
      - .env.production

.env file:

NODE_ENV=production
API_KEY=your_api_key_here
DATABASE_URL=postgres://db:5432/mydb

Volumes

services:
  app:
    image: node:18-alpine
    volumes:
      - ./app:/usr/src/app           # bind mount
      - app-data:/app/data            # named volume
      - /var/run/docker.sock:/var/run/docker.sock:ro  # read-only

Depends On

services:
  web:
    image: nginx
    depends_on:
      - api
      - cache

  api:
    image: node:18-alpine
    depends_on:
      - database

  database:
    image: postgres:15-alpine

  cache:
    image: redis:alpine

Restart Policies

services:
  web:
    image: nginx
    restart: always            # always, no, on-failure, unless-stopped

Health Checks

services:
  web:
    image: nginx
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost"]
      interval: 30s
      timeout: 10s
      retries: 3
      start_period: 40s

Resource Limits

services:
  app:
    image: myapp
    deploy:
      resources:
        limits:
          cpus: '2'
          memory: 1G
        reservations:
          cpus: '0.5'
          memory: 512M

Command and Entrypoint

services:
  app:
    image: node:18-alpine
    command: npm start
    # Or with array syntax
    command: ["npm", "run", "dev"]

  worker:
    image: node:18-alpine
    entrypoint: /docker-entrypoint.sh

Working Directory

services:
  app:
    image: node:18-alpine
    working_dir: /usr/src/app

User

services:
  app:
    image: node:18-alpine
    user: "1000:1000"

Networks in Compose

Default Network

Docker Compose automatically creates a default network:

version: '3.8'

services:
  web:
    image: nginx

  api:
    image: node:18-alpine

Services can communicate using service names as hostnames.

Custom Networks

version: '3.8'

services:
  web:
    image: nginx
    networks:
      - frontend

  api:
    image: node:18-alpine
    networks:
      - frontend
      - backend

  database:
    image: postgres
    networks:
      - backend

networks:
  frontend:
  backend:

Network Configuration

networks:
  frontend:
    driver: bridge

  backend:
    driver: bridge
    ipam:
      config:
        - subnet: 172.28.0.0/16
          gateway: 172.28.0.1

External Networks

networks:
  existing-network:
    external: true
    name: my-pre-existing-network

Volumes in Compose

Named Volumes

version: '3.8'

services:
  database:
    image: postgres:15-alpine
    volumes:
      - db-data:/var/lib/postgresql/data

volumes:
  db-data:

Volume Configuration

volumes:
  db-data:
    driver: local
    driver_opts:
      type: none
      device: /path/on/host
      o: bind

External Volumes

volumes:
  existing-volume:
    external: true
    name: my-pre-existing-volume

Environment Variables

Variable Substitution

services:
  web:
    image: nginx:${NGINX_VERSION:-latest}
    ports:
      - "${WEB_PORT:-8080}:80"

.env File

Create .env in same directory as docker-compose.yml:

NGINX_VERSION=alpine
WEB_PORT=8080
POSTGRES_PASSWORD=secretpassword

Multiple Environment Files

services:
  app:
    image: myapp
    env_file:
      - ./common.env
      - ./prod.env

Docker Compose Commands

Starting Services

# Start all services
docker compose up

# Start in detached mode
docker compose up -d

# Start specific services
docker compose up web database

# Force recreate containers
docker compose up --force-recreate

# Build images before starting
docker compose up --build

Stopping Services

# Stop all services
docker compose stop

# Stop specific service
docker compose stop web

# Stop and remove containers, networks
docker compose down

# Remove volumes too
docker compose down -v

# Remove images too
docker compose down --rmi all

Viewing Status

# List running services
docker compose ps

# Show all services (including stopped)
docker compose ps -a

# View service logs
docker compose logs

# Follow logs
docker compose logs -f

# Logs for specific service
docker compose logs -f web

# Show last 100 lines
docker compose logs --tail=100

Executing Commands

# Execute command in service
docker compose exec web sh

# Run one-off command
docker compose run web npm install

# Run without dependencies
docker compose run --no-deps web npm test

Building Images

# Build all services
docker compose build

# Build specific service
docker compose build web

# Build without cache
docker compose build --no-cache

# Build with parallel execution
docker compose build --parallel

Scaling Services

# Scale specific service
docker compose up -d --scale web=3

# Scale multiple services
docker compose up -d --scale web=3 --scale worker=5

Validation

# Validate compose file
docker compose config

# Render compose file with variables
docker compose config --resolve-image-digests

Other Useful Commands

# Pause services
docker compose pause

# Unpause services
docker compose unpause

# Restart services
docker compose restart

# View resource usage
docker compose top

# Pull latest images
docker compose pull

Real-World Examples

Example 1: WordPress Stack

version: '3.8'

services:
  wordpress:
    image: wordpress:latest
    container_name: wordpress
    restart: unless-stopped
    ports:
      - "8080:80"
    environment:
      WORDPRESS_DB_HOST: database
      WORDPRESS_DB_USER: wordpress
      WORDPRESS_DB_PASSWORD: ${DB_PASSWORD}
      WORDPRESS_DB_NAME: wordpress
    volumes:
      - wordpress-data:/var/www/html
    depends_on:
      - database
    networks:
      - wp-network

  database:
    image: mysql:8.0
    container_name: wordpress-db
    restart: unless-stopped
    environment:
      MYSQL_DATABASE: wordpress
      MYSQL_USER: wordpress
      MYSQL_PASSWORD: ${DB_PASSWORD}
      MYSQL_ROOT_PASSWORD: ${DB_ROOT_PASSWORD}
    volumes:
      - db-data:/var/lib/mysql
    networks:
      - wp-network

  phpmyadmin:
    image: phpmyadmin:latest
    container_name: phpmyadmin
    restart: unless-stopped
    ports:
      - "8081:80"
    environment:
      PMA_HOST: database
      MYSQL_ROOT_PASSWORD: ${DB_ROOT_PASSWORD}
    depends_on:
      - database
    networks:
      - wp-network

volumes:
  wordpress-data:
  db-data:

networks:
  wp-network:
    driver: bridge

Example 2: Node.js Full Stack Application

version: '3.8'

services:
  frontend:
    build:
      context: ./frontend
      dockerfile: Dockerfile
    container_name: react-frontend
    restart: unless-stopped
    ports:
      - "3000:3000"
    environment:
      - REACT_APP_API_URL=http://localhost:5000
    volumes:
      - ./frontend:/app
      - /app/node_modules
    networks:
      - app-network

  backend:
    build:
      context: ./backend
      dockerfile: Dockerfile
    container_name: node-backend
    restart: unless-stopped
    ports:
      - "5000:5000"
    environment:
      - NODE_ENV=production
      - DATABASE_URL=postgresql://postgres:${DB_PASSWORD}@database:5432/myapp
      - REDIS_URL=redis://redis:6379
    volumes:
      - ./backend:/app
      - /app/node_modules
    depends_on:
      - database
      - redis
    networks:
      - app-network

  database:
    image: postgres:15-alpine
    container_name: postgres-db
    restart: unless-stopped
    environment:
      POSTGRES_USER: postgres
      POSTGRES_PASSWORD: ${DB_PASSWORD}
      POSTGRES_DB: myapp
    volumes:
      - postgres-data:/var/lib/postgresql/data
    networks:
      - app-network

  redis:
    image: redis:7-alpine
    container_name: redis-cache
    restart: unless-stopped
    ports:
      - "6379:6379"
    volumes:
      - redis-data:/data
    networks:
      - app-network

  nginx:
    image: nginx:alpine
    container_name: nginx-proxy
    restart: unless-stopped
    ports:
      - "80:80"
      - "443:443"
    volumes:
      - ./nginx/nginx.conf:/etc/nginx/nginx.conf:ro
      - ./nginx/ssl:/etc/nginx/ssl:ro
    depends_on:
      - frontend
      - backend
    networks:
      - app-network

volumes:
  postgres-data:
  redis-data:

networks:
  app-network:
    driver: bridge

Example 3: Microservices with Message Queue

version: '3.8'

services:
  api-gateway:
    build: ./api-gateway
    ports:
      - "8080:8080"
    environment:
      - USER_SERVICE_URL=http://user-service:3001
      - ORDER_SERVICE_URL=http://order-service:3002
    depends_on:
      - user-service
      - order-service
    networks:
      - microservices

  user-service:
    build: ./user-service
    environment:
      - DATABASE_URL=postgresql://postgres:${DB_PASSWORD}@postgres:5432/users
      - RABBITMQ_URL=amqp://rabbitmq:5672
    depends_on:
      - postgres
      - rabbitmq
    networks:
      - microservices

  order-service:
    build: ./order-service
    environment:
      - DATABASE_URL=postgresql://postgres:${DB_PASSWORD}@postgres:5432/orders
      - RABBITMQ_URL=amqp://rabbitmq:5672
    depends_on:
      - postgres
      - rabbitmq
    networks:
      - microservices

  postgres:
    image: postgres:15-alpine
    environment:
      POSTGRES_PASSWORD: ${DB_PASSWORD}
    volumes:
      - postgres-data:/var/lib/postgresql/data
      - ./init-db.sql:/docker-entrypoint-initdb.d/init.sql
    networks:
      - microservices

  rabbitmq:
    image: rabbitmq:3-management-alpine
    ports:
      - "5672:5672"
      - "15672:15672"
    environment:
      RABBITMQ_DEFAULT_USER: admin
      RABBITMQ_DEFAULT_PASS: ${RABBITMQ_PASSWORD}
    volumes:
      - rabbitmq-data:/var/lib/rabbitmq
    networks:
      - microservices

volumes:
  postgres-data:
  rabbitmq-data:

networks:
  microservices:
    driver: bridge

Example 4: Python Django Application

version: '3.8'

services:
  web:
    build: .
    command: gunicorn myproject.wsgi:application --bind 0.0.0.0:8000
    volumes:
      - .:/code
      - static-volume:/code/staticfiles
      - media-volume:/code/media
    expose:
      - 8000
    environment:
      - DEBUG=0
      - SECRET_KEY=${SECRET_KEY}
      - DATABASE_URL=postgresql://postgres:${DB_PASSWORD}@db:5432/myapp
      - REDIS_URL=redis://redis:6379/1
    depends_on:
      - db
      - redis
    networks:
      - django-network

  nginx:
    image: nginx:alpine
    volumes:
      - ./nginx/nginx.conf:/etc/nginx/nginx.conf:ro
      - static-volume:/code/staticfiles:ro
      - media-volume:/code/media:ro
    ports:
      - "80:80"
    depends_on:
      - web
    networks:
      - django-network

  db:
    image: postgres:15-alpine
    volumes:
      - postgres-data:/var/lib/postgresql/data
    environment:
      - POSTGRES_USER=postgres
      - POSTGRES_PASSWORD=${DB_PASSWORD}
      - POSTGRES_DB=myapp
    networks:
      - django-network

  redis:
    image: redis:7-alpine
    networks:
      - django-network

  celery:
    build: .
    command: celery -A myproject worker -l info
    volumes:
      - .:/code
    environment:
      - DATABASE_URL=postgresql://postgres:${DB_PASSWORD}@db:5432/myapp
      - REDIS_URL=redis://redis:6379/1
    depends_on:
      - db
      - redis
    networks:
      - django-network

  celery-beat:
    build: .
    command: celery -A myproject beat -l info
    volumes:
      - .:/code
    environment:
      - DATABASE_URL=postgresql://postgres:${DB_PASSWORD}@db:5432/myapp
      - REDIS_URL=redis://redis:6379/1
    depends_on:
      - db
      - redis
    networks:
      - django-network

volumes:
  postgres-data:
  static-volume:
  media-volume:

networks:
  django-network:
    driver: bridge

Example 5: Monitoring Stack (Prometheus + Grafana)

version: '3.8'

services:
  prometheus:
    image: prom/prometheus:latest
    container_name: prometheus
    restart: unless-stopped
    volumes:
      - ./prometheus/prometheus.yml:/etc/prometheus/prometheus.yml
      - prometheus-data:/prometheus
    command:
      - '--config.file=/etc/prometheus/prometheus.yml'
      - '--storage.tsdb.path=/prometheus'
    ports:
      - "9090:9090"
    networks:
      - monitoring

  grafana:
    image: grafana/grafana:latest
    container_name: grafana
    restart: unless-stopped
    environment:
      - GF_SECURITY_ADMIN_USER=admin
      - GF_SECURITY_ADMIN_PASSWORD=${GRAFANA_PASSWORD}
      - GF_INSTALL_PLUGINS=grafana-piechart-panel
    volumes:
      - grafana-data:/var/lib/grafana
      - ./grafana/provisioning:/etc/grafana/provisioning
    ports:
      - "3000:3000"
    depends_on:
      - prometheus
    networks:
      - monitoring

  node-exporter:
    image: prom/node-exporter:latest
    container_name: node-exporter
    restart: unless-stopped
    volumes:
      - /proc:/host/proc:ro
      - /sys:/host/sys:ro
      - /:/rootfs:ro
    command:
      - '--path.procfs=/host/proc'
      - '--path.sysfs=/host/sys'
      - '--collector.filesystem.mount-points-exclude=^/(sys|proc|dev|host|etc)($$|/)'
    ports:
      - "9100:9100"
    networks:
      - monitoring

  cadvisor:
    image: gcr.io/cadvisor/cadvisor:latest
    container_name: cadvisor
    restart: unless-stopped
    volumes:
      - /:/rootfs:ro
      - /var/run:/var/run:ro
      - /sys:/sys:ro
      - /var/lib/docker/:/var/lib/docker:ro
    ports:
      - "8080:8080"
    networks:
      - monitoring

volumes:
  prometheus-data:
  grafana-data:

networks:
  monitoring:
    driver: bridge

Production Best Practices

Use Version Control

Always commit docker-compose.yml to git:

git add docker-compose.yml .env.example
git commit -m "Add Docker Compose configuration"

Environment-Specific Files

# docker-compose.yml (base)
version: '3.8'
services:
  web:
    image: myapp

# docker-compose.override.yml (development)
version: '3.8'
services:
  web:
    volumes:
      - .:/app
    command: npm run dev

# docker-compose.prod.yml (production)
version: '3.8'
services:
  web:
    restart: unless-stopped
    logging:
      driver: "json-file"
      options:
        max-size: "10m"
        max-file: "3"

Run with specific configuration:

# Development (uses override automatically)
docker compose up

# Production
docker compose -f docker-compose.yml -f docker-compose.prod.yml up -d

Security Best Practices

services:
  app:
    image: myapp
    # Run as non-root user
    user: "1000:1000"

    # Read-only root filesystem
    read_only: true
    tmpfs:
      - /tmp

    # Drop capabilities
    cap_drop:
      - ALL
    cap_add:
      - NET_BIND_SERVICE

    # Security options
    security_opt:
      - no-new-privileges:true

Resource Limits

services:
  app:
    image: myapp
    deploy:
      resources:
        limits:
          cpus: '2'
          memory: 2G
        reservations:
          cpus: '1'
          memory: 1G

Health Checks

services:
  app:
    image: myapp
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:3000/health"]
      interval: 30s
      timeout: 5s
      retries: 3
      start_period: 40s

Logging Configuration

services:
  app:
    image: myapp
    logging:
      driver: "json-file"
      options:
        max-size: "10m"
        max-file: "3"
        labels: "production"

Use Secrets (Docker Swarm)

version: '3.8'

services:
  app:
    image: myapp
    secrets:
      - db_password
      - api_key

secrets:
  db_password:
    file: ./secrets/db_password.txt
  api_key:
    external: true

Troubleshooting

View Service Logs

# All services
docker compose logs

# Specific service
docker compose logs web

# Follow logs
docker compose logs -f

# Last 100 lines
docker compose logs --tail=100

Container Won't Start

# Check status
docker compose ps

# View logs
docker compose logs service-name

# Validate compose file
docker compose config

# Force recreate
docker compose up --force-recreate

Network Issues

# Inspect network
docker network ls
docker network inspect project_default

# Recreate network
docker compose down
docker compose up

Volume Permission Issues

# Check volume
docker volume inspect project_volume-name

# Fix permissions in container
docker compose exec service-name chown -R user:group /path

Clean Up Resources

# Stop and remove everything
docker compose down

# Remove volumes too
docker compose down -v

# Remove images
docker compose down --rmi all

# Complete cleanup
docker compose down -v --rmi all --remove-orphans

Debug Service

# Execute shell in running container
docker compose exec service-name sh

# Run one-off command
docker compose run --rm service-name sh

# View container processes
docker compose top service-name

Conclusion

Docker Compose streamlines multi-container application management, making it essential for modern development workflows. This guide covered everything from basic concepts to production-ready configurations.

Key Takeaways

  • Single Configuration: Define entire stack in docker-compose.yml
  • Service Dependencies: Use depends_on for startup order
  • Network Isolation: Automatic network creation with service discovery
  • Volume Management: Persist data with named and bind volumes
  • Environment Flexibility: Use .env files and variable substitution
  • Production Ready: Implement health checks, logging, and resource limits

Quick Reference

# Essential Commands
docker compose up -d              # Start services
docker compose down               # Stop services
docker compose ps                 # List services
docker compose logs -f            # View logs
docker compose exec web sh        # Access shell
docker compose build              # Build images
docker compose pull               # Pull images
docker compose restart            # Restart services

# Management
docker compose config             # Validate file
docker compose up --build         # Rebuild and start
docker compose down -v            # Remove with volumes
docker compose scale web=3        # Scale service

Next Steps

  1. Practice: Create compose files for your projects
  2. Optimize: Implement multi-stage builds in Dockerfiles
  3. Secure: Add health checks and security options
  4. Monitor: Integrate logging and monitoring solutions
  5. Orchestrate: Graduate to Kubernetes for production
  6. Automate: Integrate with CI/CD pipelines
  7. Document: Maintain comprehensive README with setup instructions

Docker Compose is perfect for development environments and single-host deployments. For multi-host production orchestration, consider Kubernetes or Docker Swarm.