--- title: "How to Roll Back Failed Deployments with Docker Compose" description: "Deploy versioned Docker images with automatic rollback on failed health checks, back up the database before migrations and undo bad Kubernetes rollouts." category: "CI/CD DevOps" language: en source: https://cubepath.com/docs/ci-cd-devops/rollback-strategies-for-failed-deployments updated: 2026-09-25 --- # How to Roll Back Failed Deployments with Docker Compose A rollback is only fast if it was designed before the deployment failed. In this tutorial you will set up a Docker Compose deployment on Ubuntu 24.04 where every release is an immutable image tag, write a deploy script that checks the new version's health and automatically returns to the previous one if it fails, and protect the database with a backup before each migration. The last section shows the equivalent commands for a Kubernetes Deployment. ## Prerequisites To follow this tutorial you need: - A server running Ubuntu 24.04 LTS, for example a CubePath VPS, with a non-root user with `sudo` privileges. - Docker Engine and the Docker Compose plugin installed, and your user in the `docker` group. - An application image published to a registry with version tags, for example `ghcr.io/your_org/myapp:1.4.2`. The image must expose a health endpoint such as `/health` and include `curl`, which the container health check uses. - A PostgreSQL database for the application. The examples run it in the same Compose project. ## Step 1 - Deciding when to roll back Rolling back is the right answer when the previous version is known to work and returning to it is safe. Fixing forward (shipping a new patch) is better when rolling back would cause its own damage. Decide the rules in advance so nobody has to argue about them during an incident: | Situation | Roll back | Fix forward | |-----------|-----------|-------------| | Health checks fail after deploy | Yes | | | Error rate or latency clearly worse than before | Yes | | | Security issue introduced by the release | Yes | | | Minor bug with a workaround | | Yes | | The release ran a destructive database migration | | Yes (the old code cannot read the new schema) | The last row is the reason database changes get their own step below: application code is easy to roll back, data is not. ## Step 2 - Using immutable, versioned image tags You can only go back to a version that still exists and has not changed. Tags like `latest` are overwritten by every build, so a rollback to `latest` gives you the broken version again. Tag every build with a version and, optionally, the Git commit: ```bash docker build -t ghcr.io/your_org/myapp:1.4.2 -t ghcr.io/your_org/myapp:sha-3f9c2e1 . docker push ghcr.io/your_org/myapp:1.4.2 docker push ghcr.io/your_org/myapp:sha-3f9c2e1 ``` Tag the same commit in Git so you can always map a running image back to its source: ```bash git tag -a v1.4.2 -m "Release 1.4.2" git push origin v1.4.2 ``` Never delete or re-push an existing version tag. Keep at least the last few releases in the registry so there is always something to roll back to. ## Step 3 - Parameterizing the Compose file by version On the server, keep the application in its own directory. Create it and give your user ownership: ```bash sudo mkdir -p /opt/myapp sudo chown "$USER":"$USER" /opt/myapp cd /opt/myapp ``` Create the Compose file: ```bash nano /opt/myapp/compose.yaml ``` ```yaml services: app: image: ghcr.io/your_org/myapp:${APP_VERSION:?APP_VERSION is not set} restart: unless-stopped env_file: app.env ports: - "127.0.0.1:8000:8000" depends_on: db: condition: service_healthy healthcheck: test: ["CMD", "curl", "-fsS", "http://localhost:8000/health"] interval: 10s timeout: 5s retries: 3 start_period: 20s db: image: postgres:16 restart: unless-stopped environment: POSTGRES_USER: app POSTGRES_PASSWORD: ${DB_PASSWORD:?DB_PASSWORD is not set} POSTGRES_DB: app volumes: - pgdata:/var/lib/postgresql/data healthcheck: test: ["CMD-SHELL", "pg_isready -U app -d app"] interval: 10s timeout: 5s retries: 5 volumes: pgdata: ``` The image tag comes from the `APP_VERSION` variable, and `:?` makes Compose refuse to start if it is missing, instead of silently pulling an unexpected tag. The `healthcheck` is what the deploy script will rely on to decide whether a release is good. Compose reads variables from a `.env` file in the project directory. Create it with the version that is currently known to work and a strong database password: ```bash nano /opt/myapp/.env ``` ```ini APP_VERSION=1.4.1 DB_PASSWORD=your_strong_password ``` Put the application's own settings in `app.env` (for example `DATABASE_URL=postgres://app:your_strong_password@db:5432/app`), then restrict both files, since they contain secrets: ```bash chmod 600 /opt/myapp/.env /opt/myapp/app.env ``` Start the stack and wait until every container is healthy: ```bash docker compose up -d --wait docker compose ps ``` ```output NAME IMAGE STATUS myapp-app-1 ghcr.io/your_org/myapp:1.4.1 Up 30 seconds (healthy) myapp-db-1 postgres:16 Up 41 seconds (healthy) ``` ## Step 4 - Writing a deploy script with automatic rollback A deploy script is worth having here because it enforces the same order every time: pull first, switch, wait for health, and only then record the new version. If the new version never becomes healthy, it restarts the previous one. Create the script: ```bash nano /opt/myapp/deploy.sh ``` ```bash #!/usr/bin/env bash # Usage: deploy.sh set -euo pipefail APP_DIR="/opt/myapp" NEW_VERSION="${1:?usage: deploy.sh }" WAIT_TIMEOUT=120 cd "$APP_DIR" CURRENT_VERSION="$(sed -n 's/^APP_VERSION=//p' .env)" if [[ -z "$CURRENT_VERSION" ]]; then echo "APP_VERSION not found in $APP_DIR/.env" >&2 exit 1 fi if [[ "$NEW_VERSION" == "$CURRENT_VERSION" ]]; then echo "Version $NEW_VERSION is already deployed" exit 0 fi echo "Pulling $NEW_VERSION (current: $CURRENT_VERSION)" APP_VERSION="$NEW_VERSION" docker compose pull app echo "Starting $NEW_VERSION" if APP_VERSION="$NEW_VERSION" docker compose up -d --wait --wait-timeout "$WAIT_TIMEOUT" app; then sed -i "s/^APP_VERSION=.*/APP_VERSION=${NEW_VERSION}/" .env echo "$(date -Is) deployed $NEW_VERSION (previous: $CURRENT_VERSION)" >> deploy.log echo "Deployed $NEW_VERSION" else echo "Version $NEW_VERSION is unhealthy, rolling back to $CURRENT_VERSION" >&2 docker compose logs --tail 50 app >&2 || true APP_VERSION="$CURRENT_VERSION" docker compose up -d --wait --wait-timeout "$WAIT_TIMEOUT" app echo "$(date -Is) FAILED $NEW_VERSION, rolled back to $CURRENT_VERSION" >> deploy.log exit 1 fi ``` How it works: - A variable set in the shell takes precedence over the value in `.env`, so `APP_VERSION="$NEW_VERSION" docker compose ...` tries the new version without touching the file. - Pulling first means a missing tag or a registry outage fails the deploy before anything is stopped. - `docker compose up --wait` exits with an error if the container does not reach the `healthy` state within the timeout, which triggers the rollback branch. - `.env` is updated only after a successful deploy, so it always holds the last known good version, and a server reboot or a plain `docker compose up -d` starts that version. Make the script executable: ```bash chmod +x /opt/myapp/deploy.sh ``` Deploy a new version: ```bash /opt/myapp/deploy.sh 1.4.2 ``` ```output Pulling 1.4.2 (current: 1.4.1) Starting 1.4.2 Deployed 1.4.2 ``` To test the rollback path on purpose, deploy an image whose health endpoint returns an error (for example, a build with a wrong `DATABASE_URL`). The script ends with: ```output Version 1.4.3 is unhealthy, rolling back to 1.4.2 ... ``` Confirm that the previous version is running and healthy again: ```bash docker compose ps app cat /opt/myapp/deploy.log ``` A manual rollback uses the same script with the older version number, for example `/opt/myapp/deploy.sh 1.4.1`. Test this before you need it: a rollback procedure that has never been run is not a procedure. ## Step 5 - Protecting the database during migrations Rolling back the application does not roll back the database. If release 1.4.2 dropped a column, version 1.4.1 will fail against the new schema no matter how quickly you switch images. Two habits make this manageable. **Make migrations backward compatible (expand and contract).** Split breaking schema changes across releases so that the previous version of the code always works with the current schema: 1. Release N adds the new column or table and writes to both old and new structures. 2. Release N+1 reads from the new structure. 3. Release N+2, once N+1 has been stable for a while, removes the old column. With this pattern, rolling back one release never meets a schema it does not understand. **Take a backup right before migrating.** Create a directory for backups: ```bash mkdir -p /opt/myapp/backups ``` Dump the database in PostgreSQL's custom format, which `pg_restore` can restore selectively: ```bash cd /opt/myapp docker compose exec -T db pg_dump -U app -Fc app > "backups/app-$(date +%Y%m%d-%H%M%S)-pre-1.4.2.dump" ls -lh backups/ ``` Then run the release's migrations with whatever tool your application uses. If a migration goes wrong and has no working down migration, stop the application and restore the dump taken just before it: ```bash docker compose stop app docker compose exec -T db pg_restore -U app -d app --clean --if-exists < backups/app-20260925-101500-pre-1.4.2.dump /opt/myapp/deploy.sh 1.4.1 ``` `--clean --if-exists` drops the existing objects before recreating them from the dump. Any data written between the backup and the restore is lost, so restore only when the migration itself is the problem, and keep the application stopped in between. > Warning: A dump inside the same server protects you from a bad migration, not from losing the server. Copy important dumps to external storage as part of your regular backup plan. ## Step 6 - Rolling back a Kubernetes Deployment If your application runs on Kubernetes, the Deployment controller already keeps previous ReplicaSets, so a rollback is a built-in command. Kubernetes does not roll back on its own, though: a failed rollout only stops progressing and is marked as failed. Record why each rollout happened, so the history is readable: ```bash kubectl set image deployment/myapp myapp=ghcr.io/your_org/myapp:1.4.2 kubectl annotate deployment/myapp kubernetes.io/change-cause="deploy 1.4.2" ``` Wait for the rollout and fail if it does not finish in time. In a CI pipeline, this is the command whose exit code decides whether to roll back: ```bash kubectl rollout status deployment/myapp --timeout=180s ``` List the revisions: ```bash kubectl rollout history deployment/myapp ``` ```output deployment.apps/myapp REVISION CHANGE-CAUSE 3 deploy 1.4.1 4 deploy 1.4.2 ``` Return to the previous revision, or to a specific one: ```bash kubectl rollout undo deployment/myapp kubectl rollout undo deployment/myapp --to-revision=3 ``` Check that the rollback completed and which image is running: ```bash kubectl rollout status deployment/myapp kubectl get deployment myapp -o jsonpath='{.spec.template.spec.containers[0].image}{"\n"}' ``` The number of revisions kept is controlled by `spec.revisionHistoryLimit` (10 by default). Readiness probes matter as much as the Compose health check did: without them, Kubernetes considers a Pod ready as soon as it starts and the rollout "succeeds" with a broken version. If you manage manifests with GitOps (Argo CD or Flux), roll back by reverting the commit in Git instead, otherwise the controller will reapply the broken version on its next sync: ```bash git revert git push ``` ## Troubleshooting **`required variable APP_VERSION is missing a value`.** The `.env` file is missing or you ran Compose from another directory. Run commands from `/opt/myapp` or pass `--project-directory /opt/myapp`. **The container stays in `starting` and the deploy times out.** The health check command is failing inside the container. Run `docker inspect --format '{{json .State.Health}}' myapp-app-1` to see the output of the last checks, and confirm that `curl` exists in the image. **The rollback itself fails.** The previous image was deleted from the registry or the old code cannot work with the migrated schema. Keep old tags in the registry and follow the expand and contract pattern from Step 5. ## Conclusion You now deploy immutable, versioned images through a script that checks health and returns to the last known good version on its own, you take a database dump before every migration, and you know the Kubernetes commands to inspect and undo a rollout. Next, call `deploy.sh` from your CI pipeline over SSH, add a smoke test after the health check, and schedule a rollback drill so the procedure is exercised regularly.