ACME Protocol and Certificate Automation

The ACME (Automated Certificate Management Environment) protocol automates certificate issuance and renewal, eliminating manual processes. Let's Encrypt popularized ACME, but other CAs support it. This guide covers the ACME protocol fundamentals, implementation details, and popular ACME clients for different platforms.

Table of Contents

• Prerequisites
• ACME Protocol Overview
• Challenge Types
• Certbot ACME Client
• acme.sh Client
• lego Client
• Automatic Renewal
• Multi-Domain Certificates
• Staging Environment Testing
• Rate Limiting
• Troubleshooting
• Conclusion

Prerequisites

Before implementing ACME automation, ensure you have:

• Domain name with DNS control
• Linux server with internet access
• Understanding of DNS and web server basics
• Root or sudo access
• Email address for certificate notifications

ACME Protocol Overview

ACME automates certificate lifecycle management:

1. Account Creation: Create account with CA (Let's Encrypt)
2. Order: Request certificate for domain(s)
3. Challenge: Prove domain ownership (HTTP, DNS, TLS-ALPN)
4. Validation: CA validates challenge
5. Certificate Issuance: CA issues certificate
6. Renewal: Automated renewal before expiration

ACME uses JSON Web Signature (JWS) for secure communication. Each request is signed with the account's private key.

Basic ACME flow:

Client                                ACME Server (Let's Encrypt)
  |                                          |
  +---- POST newAccount --------------------->
  |                                          |
  <------ Account ID (kid) ----------------+
  |                                          |
  +---- POST newOrder ----------------------->
  | (domains: example.com, *.example.com)   |
  |                                          |
  <------ Order ID, challenges -------+
  |                                          |
  +---- POST challenge response ------------->
  |                                          |
  <------ Processing... ---------------+
  |                                          |
  +---- POST finalize (CSR) ----------------->
  |                                          |
  <------ Certificate ready ---------------+

Challenge Types

ACME supports multiple challenge types for domain ownership proof:

HTTP-01 Challenge:

• Place verification file at .well-known/acme-challenge/{token}
• Let's Encrypt fetches: http://example.com/.well-known/acme-challenge/{token}
• Requires HTTP access on port 80
• Ideal for simple websites

DNS-01 Challenge:

• Create TXT record: _acme-challenge.example.com
• Contains verification token
• Let's Encrypt queries DNS for the record
• Works for wildcard certificates
• Ideal for DNS-based validation

TLS-ALPN-01 Challenge:

• Use TLS with ALPN extension
• Application listens on port 443
• Supports bare domain validation
• Newer, less common

Example DNS-01 challenge:

# Let's Encrypt requests token
_acme-challenge.example.com TXT "verification-token-string"

# DNS query validates
dig _acme-challenge.example.com TXT

Certbot ACME Client

Certbot is the official Let's Encrypt client, supporting multiple plugins.

Install Certbot:

sudo apt-get update
sudo apt-get install -y certbot

Basic certificate request (HTTP challenge):

sudo certbot certonly --standalone \
  -d example.com \
  -d www.example.com \
  --email [email protected] \
  --agree-tos

Certificate with web server integration (Nginx):

sudo certbot --nginx \
  -d example.com \
  -d www.example.com \
  --email [email protected]

Certificate with web server integration (Apache):

sudo certbot --apache \
  -d example.com \
  -d www.example.com \
  --email [email protected]

DNS challenge with Cloudflare plugin:

pip3 install certbot-dns-cloudflare

# Create credentials file
echo "dns_cloudflare_api_token = YOUR_TOKEN" > ~/.cloudflare/cloudflare.ini
chmod 600 ~/.cloudflare/cloudflare.ini

# Request certificate
sudo certbot certonly \
  --dns-cloudflare \
  --dns-cloudflare-credentials ~/.cloudflare/cloudflare.ini \
  -d "*.example.com" \
  -d example.com

Renew certificates:

# Test renewal
sudo certbot renew --dry-run

# Manual renewal
sudo certbot renew --force-renewal

# Check renewal status
sudo certbot certificates

Automatic renewal with systemd timer:

# Check if timer is enabled
sudo systemctl list-timers | grep certbot

# Enable auto-renewal
sudo systemctl enable certbot.timer
sudo systemctl start certbot.timer

acme.sh Client

acme.sh is a lightweight shell script-based ACME client.

Install acme.sh:

curl https://get.acme.sh | sh

# Or
git clone https://github.com/acmesh-official/acme.sh.git
cd acme.sh
./acme.sh --install

Basic certificate issuance:

# Standalone mode (HTTP challenge)
~/.acme.sh/acme.sh --issue -d example.com -w /var/www/html

# With webroot
~/.acme.sh/acme.sh --issue -d example.com -w /var/www/example.com

Multiple domains:

~/.acme.sh/acme.sh --issue \
  -d example.com \
  -d www.example.com \
  -d api.example.com \
  -w /var/www/html

DNS challenge with Cloudflare:

# Export API credentials
export CF_Key="your-cloudflare-api-key"
export CF_Email="your-cloudflare-email"

# Issue certificate
~/.acme.sh/acme.sh --issue \
  -d "*.example.com" \
  -d example.com \
  --dns dns_cf

Copy certificate to destination with renewal hook:

~/.acme.sh/acme.sh --issue -d example.com -w /var/www/html \
  --install-cert -d example.com \
  --cert-file /etc/letsencrypt/live/example.com/cert.pem \
  --key-file /etc/letsencrypt/live/example.com/privkey.pem \
  --fullchain-file /etc/letsencrypt/live/example.com/fullchain.pem \
  --reloadcmd "systemctl reload nginx"

Update certificate:

~/.acme.sh/acme.sh --renew -d example.com

List certificates:

~/.acme.sh/acme.sh --list

lego Client

lego is a Go-based ACME client supporting many DNS providers.

Install lego:

# Download from releases
wget https://github.com/go-acme/lego/releases/download/v4.9.1/lego_v4.9.1_linux_amd64.tar.gz
tar -xzf lego_v4.9.1_linux_amd64.tar.gz
sudo mv lego /usr/local/bin/

Basic certificate issuance:

lego [email protected] \
  --domains=example.com \
  --domains=www.example.com \
  --http \
  run

DNS challenge with Route53:

# Set AWS credentials
export AWS_ACCESS_KEY_ID="your-access-key"
export AWS_SECRET_ACCESS_KEY="your-secret-key"

lego [email protected] \
  --domains="*.example.com" \
  --domains=example.com \
  --dns=route53 \
  run

Certificate renewal:

lego [email protected] \
  --domains=example.com \
  --http \
  renew

With renewal hook:

lego [email protected] \
  --domains=example.com \
  --hook=./hook.sh \
  renew

Automatic Renewal

Implement automated certificate renewal across all clients.

Certbot automatic renewal with cron:

# Cron job at 2 AM daily
crontab -e

# Add line
0 2 * * * /usr/bin/certbot renew --quiet --renew-hook="systemctl reload nginx"

acme.sh auto-update:

# Enable auto-upgrade
~/.acme.sh/acme.sh --upgrade --auto-upgrade

# Manual upgrade
~/.acme.sh/acme.sh --upgrade

Renewal hooks for service restart:

#!/bin/bash
# renewal-hook.sh

# Restart web server
systemctl reload nginx

# Reload application config
systemctl reload myapp

# Send notification
mail -s "Certificate renewed" [email protected] < /dev/null

Make executable and configure in cron:

chmod +x /opt/renewal-hook.sh

crontab -e
# 0 2 * * * certbot renew --quiet --renew-hook="/opt/renewal-hook.sh"

Multi-Domain Certificates

Request certificates covering multiple domains and subdomains.

Single certificate for multiple domains:

sudo certbot certonly \
  -d example.com \
  -d www.example.com \
  -d api.example.com \
  -d mail.example.com \
  --email [email protected]

Wildcard certificate with base domain:

sudo certbot certonly \
  -d "*.example.com" \
  -d example.com \
  --dns-cloudflare \
  --email [email protected]

Update existing certificate with additional domains:

sudo certbot --expand \
  -d "existing.example.com" \
  -d "new-domain.example.com"

View certificate details:

sudo certbot certificates

Staging Environment Testing

Always test with Let's Encrypt staging before production.

Staging URL has no rate limits and doesn't issue valid certificates (useful for testing).

Certbot with staging:

sudo certbot certonly \
  --staging \
  -d example.com \
  --email [email protected]

Staging certificates are self-signed but server responds correctly. Perfect for testing automation.

Request rate limits:

Production:
- 50 certificates per domain per week
- 5 duplicate certificates per week
- 5 failures per account per hour

Staging:
- 30,000 certificates per domain per week
- Unlimited failures

Test renewal process with staging:

# Test with staging first
certbot renew --staging --dry-run

# If successful, renew in production
certbot renew --force-renewal

Rate Limiting

Understand and avoid Let's Encrypt rate limits:

# Check rate limit status
curl -I https://acme-v02.api.letsencrypt.org/directory | grep Rate-Limit

# Check existing certificates for domain
curl https://crt.sh/?q=example.com&output=json | jq length

Best practices to avoid rate limits:

# Use exact same domains if certificate already exists
# Don't request new certificate if existing one covers domains

# Test with staging first
certbot certonly --staging -d example.com

# Request only what you need
# Don't use --force-renewal unless necessary

# Space out requests
# Add delays between multiple certificate requests

Troubleshooting

Common ACME issues and solutions:

DNS challenge failure:

# Verify DNS record created
dig _acme-challenge.example.com TXT

# Check propagation time
nslookup -type=TXT _acme-challenge.example.com

HTTP challenge failure:

# Verify .well-known directory exists
mkdir -p /var/www/html/.well-known/acme-challenge

# Check permissions
chmod 755 /var/www/html/.well-known

# Test with curl
curl http://example.com/.well-known/acme-challenge/

Certificate not issued:

# Check certbot logs
sudo tail -f /var/log/letsencrypt/letsencrypt.log

# Verbose mode
sudo certbot certonly -v -d example.com

Renewal failures:

# Test renewal dry run
sudo certbot renew --dry-run

# Check renewal status
sudo certbot certificates

# Manual renewal with debug
sudo certbot renew -v --cert-name example.com

Conclusion

ACME protocol automation eliminates manual certificate management overhead. This guide covered protocol fundamentals, challenge types, and three major ACME clients: Certbot, acme.sh, and lego. Implement automatic renewal with scheduled tasks, monitor certificate expiration, test with staging environment before production, and respect rate limits. ACME automation enables secure, zero-touch certificate management for modern infrastructure at scale.