SSL Certificate Chain Troubleshooting

SSL certificate chains link an end-entity certificate to a trusted root Certificate Authority through intermediate certificates. Misconfigured chains cause browser warnings, connection failures, and security issues. This guide covers understanding certificate chains, common problems, verification techniques, and troubleshooting solutions.

Table of Contents

Prerequisites

Before troubleshooting certificate chains, ensure you have:

  • OpenSSL installed
  • Certificate files (PEM format)
  • Root access to web servers
  • Basic understanding of X.509 certificates
  • Familiarity with certificate validation process

Understanding Certificate Chains

A certificate chain consists of:

  1. End-Entity Certificate: The certificate for your domain (issued to yoursite.com)
  2. Intermediate Certificate(s): Bridges between end-entity and root CA
  3. Root Certificate: The trusted anchor (self-signed)

Typical chain structure:

Your Certificate (yoursite.com)
          ↓ (signed by)
Intermediate Certificate (Let's Encrypt Authority X3)
          ↓ (signed by)
Root Certificate (DST Root CA X3)
          ↓ (self-signed and trusted by browsers)
Trusted CA Store

Browsers trust the root certificate, which validates the intermediate, which validates your certificate. This chain must be unbroken and correctly ordered.

Examining Certificate Chain Order

Inspect certificate details to understand the chain:

openssl x509 -in certificate.pem -text -noout

This displays:

  • Subject: Who the certificate is issued to
  • Issuer: Who signed this certificate
  • Validity dates
  • Public key information

For the chain, look at the Issuer field. It should match the Subject of the next certificate up the chain.

View certificate in fullchain format (server + intermediate):

openssl x509 -in fullchain.pem -text -noout | grep -A2 "Subject:\|Issuer:"

Print all certificates in a file:

openssl crl2pkcs7 -nocrl -certfile fullchain.pem | openssl pkcs7 -print_certs -text -noout

Better approach - parse individual certificates:

# Extract certificates from bundle
awk 'BEGIN {c=0} /BEGIN CERT/{c++} {print > "cert" c ".pem"}' fullchain.pem

# View each certificate
for i in cert*.pem; do
  echo "=== $i ==="
  openssl x509 -in $i -subject -issuer -noout
done

Common Chain Problems

Problem 1: Missing Intermediate Certificate

  • Symptom: "Intermediate certificate missing" browser warning
  • Cause: Only server certificate sent, not intermediate
  • Solution: Include intermediate in fullchain configuration

Problem 2: Incorrect Certificate Order

  • Symptom: "Self-signed certificate in certificate path"
  • Cause: Root certificate included or certificates in wrong order
  • Solution: Ensure order is server → intermediate → root (if included)

Problem 3: Duplicate Certificates

  • Symptom: Unnecessary data in chain
  • Cause: Server certificate listed twice or intermediates duplicated
  • Solution: Remove duplicates from chain file

Problem 4: Line Ending Issues

  • Symptom: Certificate parsing errors
  • Cause: Mixed line endings (CRLF vs LF)
  • Solution: Convert to LF only
dos2unix certificate.pem

Problem 5: Wrong Certificate Format

  • Symptom: "Unable to read certificate" errors
  • Cause: Base64 encoding or PEM format issues
  • Solution: Verify PEM format with proper headers/footers

OpenSSL Verification Commands

Verify certificate authenticity and chain integrity:

# Verify certificate is valid
openssl x509 -in certificate.pem -text -noout

# Verify certificate matches private key
openssl pkey -in private.pem -pubout -outform PEM | openssl pkey -pubin -outform DER > /tmp/pub1.der
openssl x509 -in certificate.pem -pubkey -noout -outform PEM | openssl pkey -pubin -outform DER > /tmp/pub2.der
cmp /tmp/pub1.der /tmp/pub2.der && echo "Keys match" || echo "Keys don't match"

Verify the complete certificate chain:

openssl verify -CAfile fullchain.pem certificate.pem

For intermediate verification:

# Using explicit chain
openssl verify -untrusted intermediate.pem \
  -CAfile root.pem \
  server-certificate.pem

Verify certificate against system CA store:

# Linux
openssl verify -CApath /etc/ssl/certs certificate.pem

# macOS
openssl verify -CAfile /etc/ssl/cert.pem certificate.pem

Check certificate dates (expiration):

openssl x509 -in certificate.pem -noout -dates
openssl x509 -in certificate.pem -noout -startdate -enddate

Check certificate fingerprint:

# SHA256 fingerprint
openssl x509 -in certificate.pem -noout -fingerprint -sha256

# MD5 fingerprint (for older systems)
openssl x509 -in certificate.pem -noout -fingerprint -md5

Nginx Chain Configuration

Configure Nginx to serve the complete certificate chain correctly.

Proper Nginx SSL configuration:

server {
    listen 443 ssl http2;
    server_name example.com;
    
    # CRITICAL: Use fullchain.pem (server + intermediate)
    # DO NOT use cert.pem alone
    ssl_certificate /etc/letsencrypt/live/example.com/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/example.com/privkey.pem;
    
    # Additional SSL settings
    ssl_protocols TLSv1.2 TLSv1.3;
    ssl_ciphers HIGH:!aNULL:!MD5;
    ssl_prefer_server_ciphers on;
}

If manually combining certificates:

# Correct order: server certificate first, then intermediate(s)
cat server-cert.pem intermediate-cert.pem > fullchain.pem

# Do NOT include root certificate

Verify Nginx certificate configuration:

# Test syntax
nginx -t

# Check which certificate is served
openssl s_client -connect example.com:443 -showcerts

Apache Chain Configuration

Configure Apache VirtualHost with complete certificate chain.

Proper Apache configuration:

<VirtualHost *:443>
    ServerName example.com
    
    SSLEngine on
    
    # Use fullchain for SSLCertificateFile (server + intermediate)
    SSLCertificateFile /etc/letsencrypt/live/example.com/fullchain.pem
    
    # Private key
    SSLCertificateKeyFile /etc/letsencrypt/live/example.com/privkey.pem
    
    # Optional: Chain file (deprecated but may still be needed)
    # SSLCertificateChainFile /etc/letsencrypt/live/example.com/chain.pem
    
    # SSL protocol settings
    SSLProtocol TLSv1.2 TLSv1.3
    SSLCipherSuite HIGH:!aNULL:!MD5
</VirtualHost>

Modern Apache uses only SSLCertificateFile with fullchain. The SSLCertificateChainFile directive is deprecated.

Test Apache configuration:

apache2ctl configtest

# If OK, reload
systemctl reload apache2

Java Keystore Chain Management

Import certificate chains into Java keystores.

Import certificate with chain:

keytool -import \
  -alias example.com \
  -file fullchain.pem \
  -keystore keystore.jks \
  -storepass password123

Import server certificate and intermediate separately:

# First import root (if not already trusted)
keytool -import -alias root \
  -file root.pem \
  -keystore keystore.jks \
  -storepass password123

# Then import intermediate
keytool -import -alias intermediate \
  -file intermediate.pem \
  -keystore keystore.jks \
  -storepass password123

# Finally import server certificate
keytool -import -alias example.com \
  -file server-cert.pem \
  -keystore keystore.jks \
  -storepass password123

Convert PKCS12 format with chain:

openssl pkcs12 -export \
  -in fullchain.pem \
  -inkey private.pem \
  -out keystore.p12 \
  -name example.com \
  -passout pass:password123

Import PKCS12 to Java keystore:

keytool -importkeystore \
  -srckeystore keystore.p12 \
  -srcstoretype PKCS12 \
  -srcstorepass password123 \
  -destkeystore keystore.jks \
  -deststoretype JKS \
  -deststorepass password123

List keystore contents:

keytool -list -v -keystore keystore.jks -storepass password123

Testing and Debugging

Test certificate chain with various tools:

Using OpenSSL s_client:

openssl s_client -connect example.com:443 -showcerts

This displays:

  • Server certificate
  • Intermediate certificate(s)
  • Connection details
  • Certificate verification result

Using curl with verbose output:

curl -v https://example.com
curl --cacert root.pem https://example.com  # With specific CA

Using online SSL checker:

# SSL Labs API
curl https://api.ssllabs.com/api/v3/analyze?host=example.com

# testssl.sh tool
./testssl.sh https://example.com

Manual chain verification:

# Check that each certificate is signed by the next
openssl verify -partial_chain -CAfile intermediate.pem server-cert.pem
openssl verify -CAfile root.pem intermediate.pem

Troubleshooting Guide

Certificate not trusted by browsers:

# Check if root is in system CA store
openssl verify -CApath /etc/ssl/certs certificate.pem

# If failed, verify chain order
openssl verify -untrusted intermediate.pem -CAfile root.pem server-cert.pem

# Check certificate alternative names
openssl x509 -in certificate.pem -text | grep -A1 "Subject Alternative Name"

Nginx shows wrong certificate:

# Check what Nginx is actually serving
openssl s_client -connect localhost:443 -servername example.com

# Verify configuration is reloaded
systemctl reload nginx

# Check for certificate duplication
grep -c "BEGIN CERTIFICATE" /etc/letsencrypt/live/example.com/fullchain.pem

Apache certificate chain errors:

# Test configuration thoroughly
apache2ctl configtest -D DUMP_CONFIG | grep -A5 "example.com"

# Check mod_ssl loading
apache2ctl -M | grep ssl

# Verify file permissions
ls -la /etc/letsencrypt/live/example.com/

Java application certificate errors:

# Debug Java SSL connections
-Djavax.net.debug=ssl:handshake

# Check keystore
keytool -list -keystore keystore.jks

# Verify chain in keystore
keytool -list -v -keystore keystore.jks | grep -A10 "Owner:"

Conclusion

Understanding and properly configuring certificate chains is essential for maintaining secure, trusted connections. This guide covered chain structure, common problems, verification techniques, and web server configuration. For production deployments, always use fullchain.pem (server + intermediate) in web server configurations, never include the root certificate in the served chain, verify certificates with OpenSSL regularly, monitor certificate expiration, and test configurations with multiple tools. Proper chain configuration prevents browser warnings, security issues, and connection failures.