Browsers and API clients trust your certificate only if they can build a path from it to a root certificate they already know. The server is responsible for sending the intermediate certificates in that path; when it does not, some clients work (because they cache or download intermediates) and others fail with errors such as unable to verify the first certificate. In this tutorial you will use OpenSSL on Ubuntu 24.04 to see exactly which chain a server sends, check certificate files locally, rebuild a correct chain file, and fix the Nginx and Apache configuration.

Prerequisites

To follow this guide you need:

  • A server running Ubuntu 24.04 LTS, for example a CubePath VPS, with a non-root user with sudo privileges. The openssl command is installed by default.
  • A site served over HTTPS by Nginx or Apache, reachable as your_domain.
  • The certificate files for that site, typically your certificate, the intermediate bundle and the private key.

Step 1 - Understanding how a chain is built

A typical chain has three levels:

LevelExample subjectWho sends it
Leaf (end-entity)CN=your_domainYour server
Intermediate CACN=R12, O=Let's EncryptYour server
Root CACN=ISRG Root X1Nobody: it is already in the client's trust store

Each certificate's Issuer must equal the Subject of the next one. The server must send the leaf first, followed by the intermediates in order. Sending the root is unnecessary; clients ignore it because they only trust their own copy.

Almost all chain problems fall into four groups:

  • Missing intermediate: the server sends only the leaf. Desktop browsers often still work, while curl, Java, Python, Android apps and payment gateways fail.
  • Wrong order: the intermediate comes before the leaf, or the chain file belongs to a different CA.
  • Wrong certificate: the private key does not match the certificate, or the server presents a certificate for another name.
  • Expired or distrusted certificates: an expired leaf or intermediate, or an old client trust store that lacks a newer root.

Step 2 - Inspecting the chain a server sends

Start from the outside, looking at what clients actually receive. -servername sends SNI, which is required when several sites share an IP address:

echo | openssl s_client -connect your_domain:443 -servername your_domain -showcerts 2>/dev/null | grep -E '^ *[0-9] s:|^ *i:|Verify return code'

A healthy server shows the leaf at depth 0, an intermediate at depth 1 whose subject matches the leaf's issuer, and a return code of 0:

 0 s:CN = your_domain
   i:C = US, O = Let's Encrypt, CN = R12
 1 s:C = US, O = Let's Encrypt, CN = R12
   i:C = US, O = Internet Security Research Group, CN = ISRG Root X1
Verify return code: 0 (ok)

A server missing its intermediate shows only depth 0 and fails verification:

 0 s:CN = your_domain
   i:C = US, O = Let's Encrypt, CN = R12
Verify return code: 21 (unable to verify the first certificate)

The most useful return codes are:

CodeMessageUsual cause
0okThe chain is complete and trusted
10certificate has expiredLeaf or intermediate past its notAfter date
18self-signed certificateThe server presents a self-signed leaf
19self-signed certificate in certificate chainA private or untrusted root is included in the chain
20unable to get local issuer certificateIntermediate missing or wrong, or the root is not in the local trust store
21unable to verify the first certificateThe server sends only the leaf
62hostname mismatchCertificate does not cover the name (only with -verify_hostname)

To check the hostname at the same time, add -verify_hostname your_domain to the s_client command.

Also check what curl reports, since it uses the system trust store in the same way as most command line tools and many applications:

curl -sS -o /dev/null https://your_domain && echo "chain OK"

With a broken chain, curl prints SSL certificate problem: unable to get local issuer certificate instead.

Step 3 - Checking certificate files locally

Once you know what the server sends, look at the files it is configured with. Print subject and issuer for every certificate in a bundle, in the order they appear:

openssl crl2pkcs7 -nocrl -certfile /path/to/fullchain.pem | openssl pkcs7 -print_certs -noout
subject=CN = your_domain
issuer=C = US, O = Let's Encrypt, CN = R12

subject=C = US, O = Let's Encrypt, CN = R12
issuer=C = US, O = Internet Security Research Group, CN = ISRG Root X1

The first certificate must be your leaf, and each issuer must match the subject of the certificate below it. Count how many certificates the file contains, which helps spot duplicates:

grep -c 'BEGIN CERTIFICATE' /path/to/fullchain.pem

Verify the leaf against the intermediates, using the system trust store (/etc/ssl/certs) for the root. -untrusted marks the intermediates as candidates for building the chain, not as trust anchors:

openssl verify -untrusted /path/to/chain.pem /path/to/cert.pem
/path/to/cert.pem: OK

If this prints error 20 at 0 depth lookup: unable to get local issuer certificate, the intermediate file does not contain the issuer of your certificate.

Check the validity dates of the leaf and of each intermediate:

openssl x509 -in /path/to/cert.pem -noout -subject -dates
subject=CN = your_domain
notBefore=Sep  1 08:14:22 2026 GMT
notAfter=Nov 30 08:14:21 2026 GMT

Finally, confirm the private key belongs to the certificate by comparing the hash of their public keys. The two lines must be identical:

openssl x509 -in /path/to/cert.pem -noout -pubkey | openssl sha256
sudo openssl pkey -in /path/to/privkey.pem -pubout | openssl sha256
SHA2-256(stdin)= 3f7a9c...e21b
SHA2-256(stdin)= 3f7a9c...e21b

This works for both RSA and ECDSA keys. If the hashes differ, Nginx refuses to start with key values mismatch and Apache with certificate and private key do not match.

Step 4 - Building a correct chain file

If you use Certbot, you do not need to build anything: /etc/letsencrypt/live/your_domain/fullchain.pem already contains the leaf followed by the intermediate. Use it and skip to Step 5.

For certificates from a commercial CA, you normally receive the leaf (your_domain.crt) and an intermediate bundle (often ca-bundle.crt or intermediate.crt). If you have lost the intermediate, the leaf itself tells you where to download it, in the Authority Information Access extension:

openssl x509 -in your_domain.crt -noout -ext authorityInfoAccess
Authority Information Access:
    CA Issuers - URI:http://ca.example.com/intermediate.crt

Download it and convert it to PEM. The file is usually in binary DER format:

curl -fsSL -o intermediate.der http://ca.example.com/intermediate.crt
openssl x509 -inform DER -in intermediate.der -out intermediate.pem

If the conversion fails with Could not read certificate, the file is already PEM; rename it to intermediate.pem instead. A .p7c file is a PKCS#7 bundle and is converted with openssl pkcs7 -inform DER -in file.p7c -print_certs -out intermediate.pem.

Build the chain file with the leaf first, then the intermediates, and without the root:

cat your_domain.crt intermediate.pem > fullchain.pem

Files from Windows systems sometimes have CRLF line endings, which some tools reject. Strip them in place:

sed -i 's/\r$//' fullchain.pem

Verify the result as in Step 3 before deploying it:

openssl verify -untrusted fullchain.pem your_domain.crt
your_domain.crt: OK

Copy the chain and the key to a proper location, for example /etc/ssl/your_domain/, with the key readable only by root:

sudo mkdir -p /etc/ssl/your_domain
sudo cp fullchain.pem /etc/ssl/your_domain/fullchain.pem
sudo cp your_domain.key /etc/ssl/your_domain/privkey.pem
sudo chmod 600 /etc/ssl/your_domain/privkey.pem

Step 5 - Fixing the web server configuration

The single most common cause of an incomplete chain is pointing the server at the leaf certificate (cert.pem or your_domain.crt) instead of the full chain.

Nginx

Nginx reads the leaf and the intermediates from one file. Open your server block, for example:

sudo nano /etc/nginx/sites-available/your_domain

Make sure ssl_certificate points to the full chain:

server {
    listen 443 ssl http2;
    server_name your_domain;

    ssl_certificate     /etc/ssl/your_domain/fullchain.pem;
    ssl_certificate_key /etc/ssl/your_domain/privkey.pem;

    # ... rest of the configuration
}

With Certbot the paths are /etc/letsencrypt/live/your_domain/fullchain.pem and /etc/letsencrypt/live/your_domain/privkey.pem. Test and reload:

sudo nginx -t
sudo systemctl reload nginx

Apache

Since Apache 2.4.8, SSLCertificateFile accepts the leaf followed by the intermediates, and SSLCertificateChainFile is obsolete. Open the virtual host:

sudo nano /etc/apache2/sites-available/your_domain-ssl.conf
<VirtualHost *:443>
    ServerName your_domain

    SSLEngine on
    SSLCertificateFile    /etc/ssl/your_domain/fullchain.pem
    SSLCertificateKeyFile /etc/ssl/your_domain/privkey.pem

    # ... rest of the configuration
</VirtualHost>

Remove any leftover SSLCertificateChainFile line that points to an old intermediate, then test and reload:

sudo apache2ctl configtest
sudo systemctl reload apache2
Syntax OK

Step 6 - Verifying the fix

Repeat the check from Step 2. You should now see the intermediate at depth 1 and return code 0:

echo | openssl s_client -connect your_domain:443 -servername your_domain -verify_hostname your_domain -showcerts 2>/dev/null | grep -E '^ *[0-9] s:|Verify return code'
 0 s:CN = your_domain
 1 s:C = US, O = Let's Encrypt, CN = R12
Verify return code: 0 (ok)

Run the curl check as well:

curl -sS -o /dev/null https://your_domain && echo "chain OK"
chain OK

If the domain resolves to several servers or sits behind a load balancer, run the check against each backend IP with -connect backend_ip:443 -servername your_domain, since each one may have its own copy of the files.

Troubleshooting

  • The server still sends the old chain after the fix. Another server block or virtual host matches first, often the default one. Find every certificate directive with sudo nginx -T | grep ssl_certificate or sudo apache2ctl -S, and remember the configuration only applies after a reload.
  • Only some clients fail and the chain looks correct. The client trust store is out of date. On Debian and Ubuntu clients, update it with sudo apt install --only-upgrade ca-certificates; for Java, update the JDK, which ships its own cacerts.
  • Java applications and Tomcat. Java keystores need the full chain too. Create a PKCS#12 keystore, which current Java versions read directly, including the intermediates with -certfile: openssl pkcs12 -export -in your_domain.crt -inkey your_domain.key -certfile intermediate.pem -name your_domain -out keystore.p12. OpenSSL asks for an export password; use a strong one and configure it in the application.
  • Return code 19 on an internal service. The chain includes a private root that clients do not trust. Distribute the private root to the clients (on Ubuntu, copy it to /usr/local/share/ca-certificates/ with a .crt extension and run sudo update-ca-certificates) rather than disabling verification.

Conclusion

You can now read the chain a server sends with openssl s_client, verify certificate files and key pairs locally, rebuild a chain file in the right order from the leaf and its intermediates, and point Nginx or Apache at it. Add the s_client or curl check to your deployment process or monitoring so a missing intermediate is caught before clients report it, and prefer ACME clients such as Certbot, which always produce a correct fullchain.pem.