Varnish Cache Configuration

Varnish Cache is a powerful HTTP accelerator and reverse caching proxy that dramatically improves web application performance by caching content and serving it at lightning speed. As one of the most effective caching solutions available, Varnish can handle hundreds of thousands of requests per second while reducing backend server load by up to 90%. This comprehensive guide will walk you through installing, configuring, and optimizing Varnish Cache to supercharge your web infrastructure.

Whether you're running a high-traffic news website, e-commerce platform, content management system, or API service, implementing Varnish Cache can transform your application's performance profile. By intelligently caching HTTP responses and serving them from memory, Varnish eliminates redundant backend processing and database queries, delivering exceptional speed improvements that directly impact user experience and SEO rankings.

Prerequisites
Understanding Varnish Cache
Installation
Basic Configuration
VCL Configuration Language
Cache Policies
Backend Configuration
Advanced Caching Strategies
Verification and Testing
Troubleshooting
Best Practices
Conclusion

Prerequisites

Before installing and configuring Varnish Cache, ensure you have:

Operating System: Ubuntu 20.04/22.04, Debian 10/11, CentOS 8/Rocky Linux 8, or similar
Backend Web Server: Apache, Nginx, or any HTTP server already configured
RAM: At least 1GB RAM (more recommended for production; Varnish stores cache in memory)
Root or sudo access: Required for installation and configuration
Backend Application: Running web application or website to cache
Basic understanding: HTTP protocol, caching concepts, and web server administration
Backup: Current backup of your web server configuration

Varnish operates as a reverse proxy cache, sitting between clients and your backend web server, so ensure your backend is properly configured and running before proceeding.

Understanding Varnish Cache

What is Varnish Cache?

Varnish Cache is an HTTP accelerator and caching reverse proxy designed for content-heavy dynamic websites and APIs. Unlike traditional web servers, Varnish is specifically built for caching and serving cached content at exceptional speeds.

Key characteristics:

In-memory caching: Stores cached objects in RAM for ultra-fast access
Reverse proxy architecture: Sits in front of backend web servers
VCL scripting: Powerful configuration language for fine-tuned cache control
Edge Side Includes (ESI): Allows partial page caching
Load balancing: Can distribute requests across multiple backends

How Varnish Works

Request Reception: Client sends HTTP request to Varnish
Cache Lookup: Varnish checks if content exists in cache
Cache Hit: If found, Varnish serves cached content immediately
Cache Miss: If not found, Varnish forwards request to backend
Backend Response: Backend processes request and returns response
Cache Storage: Varnish stores response in cache and serves to client
Subsequent Requests: Future requests for same content served from cache

Benefits of Varnish Cache

Performance Gains:

50-1000x faster response times compared to backend processing
Reduces backend load by 80-95% for cacheable content
Handles high traffic spikes without backend overload
Improved user experience through faster page loads

Scalability:

Handle thousands of concurrent connections efficiently
Serve millions of requests per day from moderate hardware
Scale horizontally with multiple Varnish instances

Flexibility:

Powerful VCL scripting for custom caching logic
Support for different backend servers
Advanced cache invalidation strategies

Cost Efficiency:

Reduce infrastructure costs by handling more traffic with fewer servers
Lower bandwidth usage through efficient caching

Installation

Installing Varnish on Ubuntu/Debian

Using official Varnish repository (recommended for latest version):

# Update package index
sudo apt update

# Install prerequisites
sudo apt install -y debian-archive-keyring curl gnupg apt-transport-https

# Add Varnish GPG key
curl -fsSL https://packagecloud.io/varnishcache/varnish70/gpgkey | sudo gpg --dearmor -o /usr/share/keyrings/varnishcache-archive-keyring.gpg

# Add Varnish repository (Varnish 7.0 LTS)
echo "deb [signed-by=/usr/share/keyrings/varnishcache-archive-keyring.gpg] https://packagecloud.io/varnishcache/varnish70/ubuntu/ $(lsb_release -sc) main" | sudo tee /etc/apt/sources.list.d/varnishcache-varnish70.list

# Update package index
sudo apt update

# Install Varnish
sudo apt install -y varnish

# Verify installation
varnishd -V

Using default Ubuntu repository (older version):

# Install Varnish from default repository
sudo apt update
sudo apt install -y varnish

# Check version
varnishd -V

Installing Varnish on CentOS/Rocky Linux

# Install EPEL repository
sudo dnf install -y epel-release

# Add Varnish repository (Varnish 7.0)
sudo curl -s https://packagecloud.io/install/repositories/varnishcache/varnish70/script.rpm.sh | sudo bash

# Install Varnish
sudo dnf install -y varnish

# Verify installation
varnishd -V

# Enable and start Varnish
sudo systemctl enable varnish
sudo systemctl start varnish

Initial Configuration Files

After installation, key configuration files are located at:

/etc/varnish/default.vcl: Main VCL configuration file
/etc/default/varnish (Ubuntu/Debian) or /etc/varnish/varnish.params (CentOS): Service parameters
/etc/systemd/system/varnish.service: Systemd service file
/var/lib/varnish: Cache storage directory

Basic Configuration

Configuring Varnish Port

By default, Varnish listens on port 6081. For production use, configure it to listen on port 80.

Step 1: Reconfigure backend web server

First, change your backend web server (Apache/Nginx) to listen on a different port:

For Apache (/etc/apache2/ports.conf):

# Change from:
Listen 80

# To:
Listen 8080

Update virtual hosts:

# Ubuntu/Debian
sudo sed -i 's/:80/:8080/g' /etc/apache2/sites-available/*.conf

# Restart Apache
sudo systemctl restart apache2

For Nginx (/etc/nginx/sites-available/default):

# Change from:
listen 80;

# To:
listen 8080;

Restart Nginx:

sudo systemctl restart nginx

Step 2: Configure Varnish to listen on port 80

Edit Varnish service parameters:

# Ubuntu/Debian
sudo nano /etc/default/varnish

# CentOS/Rocky Linux
sudo nano /etc/varnish/varnish.params

Configure the following parameters:

# Varnish listening address and port
VARNISH_LISTEN_ADDRESS=0.0.0.0
VARNISH_LISTEN_PORT=80

# Cache storage
# malloc: RAM-based storage
# file: Disk-based storage
VARNISH_STORAGE="malloc,256M"

# VCL configuration file
VARNISH_VCL_CONF=/etc/varnish/default.vcl

# Admin interface
VARNISH_ADMIN_LISTEN_ADDRESS=127.0.0.1
VARNISH_ADMIN_LISTEN_PORT=6082

# Secret file for admin authentication
VARNISH_SECRET_FILE=/etc/varnish/secret

Step 3: Update systemd service file

Create or edit the systemd override:

# Create systemd override directory
sudo mkdir -p /etc/systemd/system/varnish.service.d/

# Create override configuration
sudo nano /etc/systemd/system/varnish.service.d/override.conf

Add the following content:

[Service]
ExecStart=
ExecStart=/usr/sbin/varnishd \
  -a :80 \
  -T localhost:6082 \
  -f /etc/varnish/default.vcl \
  -S /etc/varnish/secret \
  -s malloc,256m

Reload systemd and restart Varnish:

# Reload systemd daemon
sudo systemctl daemon-reload

# Restart Varnish
sudo systemctl restart varnish

# Verify Varnish is running on port 80
sudo ss -tlnp | grep :80
# Should show varnishd listening on port 80

Basic VCL Configuration

Edit the main VCL file to configure your backend:

sudo nano /etc/varnish/default.vcl

Basic VCL configuration:

vcl 4.1;

# Backend definition
backend default {
    .host = "127.0.0.1";
    .port = "8080";
    .connect_timeout = 600s;
    .first_byte_timeout = 600s;
    .between_bytes_timeout = 600s;
}

# Receive request from client
sub vcl_recv {
    # Remove any existing X-Forwarded-For headers
    unset req.http.X-Forwarded-For;
    set req.http.X-Forwarded-For = client.ip;

    # Normalize Accept-Encoding header
    if (req.http.Accept-Encoding) {
        if (req.url ~ "\.(jpg|jpeg|png|gif|gz|tgz|bz2|tbz|mp3|ogg|swf|woff|woff2|ttf|eot)$") {
            # Already compressed formats, don't modify
            unset req.http.Accept-Encoding;
        } elsif (req.http.Accept-Encoding ~ "gzip") {
            set req.http.Accept-Encoding = "gzip";
        } elsif (req.http.Accept-Encoding ~ "deflate") {
            set req.http.Accept-Encoding = "deflate";
        } else {
            unset req.http.Accept-Encoding;
        }
    }

    # Don't cache POST, PUT, DELETE requests
    if (req.method != "GET" && req.method != "HEAD") {
        return (pass);
    }

    # Don't cache requests with authorization
    if (req.http.Authorization) {
        return (pass);
    }

    # Don't cache admin areas
    if (req.url ~ "^/admin" || req.url ~ "^/wp-admin") {
        return (pass);
    }

    # Cache everything else
    return (hash);
}

# Handle backend response
sub vcl_backend_response {
    # Cache static files for 1 hour
    if (bereq.url ~ "\.(jpg|jpeg|png|gif|css|js|ico|woff|woff2|ttf|eot|svg)$") {
        set beresp.ttl = 1h;
        unset beresp.http.Set-Cookie;
    }

    # Don't cache cookies
    if (beresp.http.Set-Cookie) {
        set beresp.ttl = 0s;
        set beresp.uncacheable = true;
        return (deliver);
    }

    # Default cache time
    set beresp.ttl = 5m;

    return (deliver);
}

# Deliver response to client
sub vcl_deliver {
    # Add cache status header for debugging
    if (obj.hits > 0) {
        set resp.http.X-Cache = "HIT";
        set resp.http.X-Cache-Hits = obj.hits;
    } else {
        set resp.http.X-Cache = "MISS";
    }

    # Remove backend headers for security
    unset resp.http.Server;
    unset resp.http.X-Powered-By;
    unset resp.http.X-Varnish;
    unset resp.http.Via;

    return (deliver);
}

Test and reload VCL:

# Test VCL syntax
sudo varnishd -C -f /etc/varnish/default.vcl

# Reload Varnish with new configuration
sudo systemctl reload varnish

# Verify Varnish is working
curl -I http://localhost

VCL Configuration Language

VCL Basics

VCL (Varnish Configuration Language) is a domain-specific language for defining caching policies. It consists of subroutines that are executed at different stages of request processing.

Key VCL subroutines:

vcl_recv: Called when request is received from client
vcl_backend_fetch: Called before sending request to backend
vcl_backend_response: Called when response is received from backend
vcl_deliver: Called before delivering response to client
vcl_hash: Defines cache key components
vcl_hit: Called when cached object is found
vcl_miss: Called when cached object is not found

Conditional Logic in VCL

sub vcl_recv {
    # If-else conditions
    if (req.url ~ "^/api/") {
        # API requests
        set req.backend_hint = api_backend;
    } elsif (req.url ~ "^/static/") {
        # Static content
        set req.backend_hint = static_backend;
    } else {
        # Default backend
        set req.backend_hint = default;
    }

    # Regular expression matching
    if (req.url ~ "\.(css|js|jpg|png|gif|ico|svg)$") {
        # Remove cookies for static content
        unset req.http.Cookie;
    }

    # Multiple conditions with logical operators
    if (req.http.Cookie && req.url !~ "^/admin") {
        # Handle cookies except for admin area
        return (hash);
    }
}

Variable Manipulation

sub vcl_recv {
    # Set custom headers
    set req.http.X-Custom-Header = "value";

    # Concatenate strings
    set req.http.X-Full-URL = req.http.host + req.url;

    # Remove headers
    unset req.http.Cookie;

    # Access client information
    set req.http.X-Client-IP = client.ip;
    set req.http.X-Client-Identity = client.identity;
}

sub vcl_backend_response {
    # Modify TTL
    set beresp.ttl = 1h;

    # Set cache control
    set beresp.http.Cache-Control = "public, max-age=3600";
}

Cache Policies

Cache Duration Configuration

Define different cache durations based on content type:

sub vcl_backend_response {
    # Remove cookies for cacheable content
    if (bereq.url ~ "^/static/" || bereq.url ~ "\.(css|js|jpg|png|gif|ico|svg|woff|woff2)$") {
        unset beresp.http.Set-Cookie;
    }

    # Static assets: 1 week
    if (bereq.url ~ "\.(css|js|jpg|jpeg|png|gif|ico|svg|woff|woff2|ttf|eot)$") {
        set beresp.ttl = 7d;
        set beresp.http.Cache-Control = "public, max-age=604800";
    }
    # HTML pages: 10 minutes
    elsif (beresp.http.Content-Type ~ "text/html") {
        set beresp.ttl = 10m;
        set beresp.http.Cache-Control = "public, max-age=600";
    }
    # JSON API responses: 5 minutes
    elsif (beresp.http.Content-Type ~ "application/json") {
        set beresp.ttl = 5m;
        set beresp.http.Cache-Control = "public, max-age=300";
    }
    # Default: 1 hour
    else {
        set beresp.ttl = 1h;
        set beresp.http.Cache-Control = "public, max-age=3600";
    }

    # Grace mode: serve stale content if backend is down
    set beresp.grace = 6h;

    return (deliver);
}

Cache Bypass Rules

Configure when to bypass cache:

sub vcl_recv {
    # Bypass cache for specific URL patterns
    if (req.url ~ "^/admin" ||
        req.url ~ "^/login" ||
        req.url ~ "^/checkout" ||
        req.url ~ "^/cart") {
        return (pass);
    }

    # Bypass cache for authenticated users
    if (req.http.Authorization || req.http.Cookie ~ "sessionid") {
        return (pass);
    }

    # Bypass cache for POST, PUT, DELETE methods
    if (req.method != "GET" && req.method != "HEAD") {
        return (pass);
    }

    # Bypass cache when specific header is present
    if (req.http.Cache-Control ~ "no-cache") {
        return (pass);
    }

    # Force refresh with Ctrl+F5
    if (req.http.Cache-Control ~ "no-cache" || req.http.Pragma ~ "no-cache") {
        set req.hash_always_miss = true;
    }
}

Cookie Handling

Properly handle cookies for optimal caching:

sub vcl_recv {
    # Remove specific cookies while keeping others
    if (req.http.Cookie) {
        # Remove Google Analytics and other tracking cookies
        set req.http.Cookie = regsuball(req.http.Cookie, "(^|;\s*)(_ga|_gid|_gat|__utm[a-z]+)=[^;]*", "");
        set req.http.Cookie = regsuball(req.http.Cookie, "^;\s*", "");

        # If no cookies remain, remove the header
        if (req.http.Cookie == "") {
            unset req.http.Cookie;
        }
    }

    # Remove cookies for static content
    if (req.url ~ "^/static/" || req.url ~ "\.(css|js|jpg|png|gif|ico|svg|woff|woff2)$") {
        unset req.http.Cookie;
    }
}

sub vcl_backend_response {
    # Remove Set-Cookie header for cacheable content
    if (bereq.url ~ "^/static/" || bereq.url ~ "\.(css|js|jpg|png|gif|ico|svg)$") {
        unset beresp.http.Set-Cookie;
    }
}

Backend Configuration

Multiple Backend Servers

Configure multiple backend servers for load balancing:

vcl 4.1;

import directors;

# Backend servers
backend web1 {
    .host = "192.168.1.10";
    .port = "8080";
    .connect_timeout = 5s;
    .first_byte_timeout = 30s;
    .between_bytes_timeout = 5s;
    .probe = {
        .url = "/health";
        .timeout = 2s;
        .interval = 5s;
        .window = 5;
        .threshold = 3;
    }
}

backend web2 {
    .host = "192.168.1.11";
    .port = "8080";
    .connect_timeout = 5s;
    .first_byte_timeout = 30s;
    .between_bytes_timeout = 5s;
    .probe = {
        .url = "/health";
        .timeout = 2s;
        .interval = 5s;
        .window = 5;
        .threshold = 3;
    }
}

backend web3 {
    .host = "192.168.1.12";
    .port = "8080";
    .connect_timeout = 5s;
    .first_byte_timeout = 30s;
    .between_bytes_timeout = 5s;
    .probe = {
        .url = "/health";
        .timeout = 2s;
        .interval = 5s;
        .window = 5;
        .threshold = 3;
    }
}

# Initialize director for load balancing
sub vcl_init {
    new cluster = directors.round_robin();
    cluster.add_backend(web1);
    cluster.add_backend(web2);
    cluster.add_backend(web3);
}

sub vcl_recv {
    # Set backend hint to load balancer
    set req.backend_hint = cluster.backend();
}

Health Checks

Configure backend health checks:

backend default {
    .host = "127.0.0.1";
    .port = "8080";

    # Health check configuration
    .probe = {
        .url = "/health";              # Health check endpoint
        .request =
            "GET /health HTTP/1.1"
            "Host: localhost"
            "Connection: close"
            "User-Agent: Varnish Health Check";
        .timeout = 2s;                 # Probe timeout
        .interval = 5s;                # Check every 5 seconds
        .window = 5;                   # Last 5 probes
        .threshold = 3;                # 3 out of 5 must succeed
        .initial = 3;                  # Initial healthy threshold
    }
}

Failover Configuration

Implement backend failover:

backend primary {
    .host = "primary.example.com";
    .port = "8080";
    .probe = {
        .url = "/health";
        .interval = 5s;
        .timeout = 2s;
        .window = 5;
        .threshold = 3;
    }
}

backend fallback {
    .host = "fallback.example.com";
    .port = "8080";
}

sub vcl_recv {
    # Try primary, fallback to secondary if unhealthy
    if (std.healthy(req.backend_hint)) {
        set req.backend_hint = primary;
    } else {
        set req.backend_hint = fallback;
    }
}

Advanced Caching Strategies

Edge Side Includes (ESI)

Use ESI for partial page caching:

sub vcl_backend_response {
    # Enable ESI processing
    if (beresp.http.Content-Type ~ "text/html") {
        set beresp.do_esi = true;
    }
}

Backend HTML with ESI tags:

<!DOCTYPE html>
<html>
<head>
    <title>My Website</title>
</head>
<body>
    <!-- Cached header (1 day) -->
    <header>
        <esi:include src="/header.html" />
    </header>

    <!-- Dynamic content (not cached) -->
    <main>
        <esi:include src="/dynamic-content" />
    </main>

    <!-- Cached footer (1 day) -->
    <footer>
        <esi:include src="/footer.html" />
    </footer>
</body>
</html>

Cache Invalidation

Implement cache purging:

# ACL for allowed purge IPs
acl purge_allowed {
    "localhost";
    "127.0.0.1";
    "192.168.1.0"/24;
}

sub vcl_recv {
    # Handle PURGE method
    if (req.method == "PURGE") {
        if (!client.ip ~ purge_allowed) {
            return (synth(405, "Not allowed"));
        }
        return (purge);
    }

    # Handle BAN method (purge by pattern)
    if (req.method == "BAN") {
        if (!client.ip ~ purge_allowed) {
            return (synth(405, "Not allowed"));
        }
        ban("req.url ~ " + req.url);
        return (synth(200, "Banned"));
    }
}

Purge cache from command line:

# Purge specific URL
curl -X PURGE http://example.com/path/to/page

# Ban by pattern
curl -X BAN http://example.com/category/*

# Purge entire cache (use with caution!)
sudo varnishadm "ban req.url ~ /"

Grace Mode

Serve stale content when backend is unavailable:

sub vcl_backend_response {
    # Set grace period
    set beresp.grace = 6h;  # Serve stale content for up to 6 hours

    # Set keep period
    set beresp.keep = 24h;  # Keep objects in cache for 24 hours
}

sub vcl_hit {
    # If backend is unhealthy, serve stale content
    if (obj.ttl >= 0s) {
        # Normal hit
        return (deliver);
    } elsif (std.healthy(req.backend_hint)) {
        # Backend is healthy, refresh content
        if (obj.ttl + 10s > 0s) {
            set req.http.grace = "normal";
            return (deliver);
        } else {
            return (restart);
        }
    } else {
        # Backend unhealthy, serve stale content
        return (deliver);
    }
}

Caching POST Requests

Cache POST requests with specific criteria:

sub vcl_recv {
    # Hash POST requests for caching GraphQL queries
    if (req.method == "POST" && req.url ~ "^/graphql") {
        # Only cache if query is marked as cacheable
        if (req.http.X-Cache-This == "true") {
            return (hash);
        }
    }
}

sub vcl_hash {
    # Include POST body in cache key
    hash_data(req.url);
    if (req.http.host) {
        hash_data(req.http.host);
    } else {
        hash_data(server.ip);
    }
    # Add request body to cache key for POST
    if (req.method == "POST") {
        hash_data(req.http.X-Body-Hash);
    }
    return (lookup);
}

Verification and Testing

Check Varnish Status

Verify Varnish is running correctly:

# Check Varnish service status
sudo systemctl status varnish

# Check which port Varnish is listening on
sudo ss -tlnp | grep varnish

# Check Varnish statistics
varnishstat

# View Varnish log
varnishlog

# View only cache hits and misses
varnishlog -g request -q "VCL_call eq 'HIT' or VCL_call eq 'MISS'"

Test Cache Functionality

Test if caching is working:

# First request (cache MISS)
curl -I http://example.com/
# Look for: X-Cache: MISS

# Second request (cache HIT)
curl -I http://example.com/
# Look for: X-Cache: HIT

# Check cache hits count
curl -I http://example.com/ | grep X-Cache-Hits

Monitor Cache Performance

# Real-time statistics
varnishstat

# Key metrics to watch:
# - cache_hit: Number of cache hits
# - cache_miss: Number of cache misses
# - n_object: Number of cached objects
# - n_expired: Number of expired objects

# Calculate hit rate
varnishstat -1 | grep -E 'cache_hit|cache_miss'

# Monitor specific counters
watch -n 1 'varnishstat -1 | grep -E "cache_hit|cache_miss|n_object"'

Debug Cache Behavior

# View detailed request flow
varnishlog -g request

# Filter by URL
varnishlog -q "ReqURL ~ '/api/'"

# View backend requests only
varnishlog -g request -i BeginFetch

# Check why cache was bypassed
varnishlog -q "VCL_call eq 'PASS'"

# View all headers
varnishlog -i ReqHeader,RespHeader

Troubleshooting

Cache Not Working

Issue: Content is not being cached

Diagnosis:

# Check cache status in response headers
curl -I http://example.com/ | grep X-Cache

# View Varnish log for specific request
varnishlog -q "ReqURL eq '/'"

# Check VCL configuration
sudo varnishd -C -f /etc/varnish/default.vcl

Common causes:

Cookies preventing caching:

# Solution: Remove cookies for cacheable content
sub vcl_recv {
    if (req.url ~ "\.(css|js|jpg|png)$") {
        unset req.http.Cookie;
    }
}

Cache-Control headers preventing caching:

# Solution: Override backend cache control
sub vcl_backend_response {
    set beresp.ttl = 1h;
    set beresp.http.Cache-Control = "public, max-age=3600";
}

Authorization headers:

# Solution: Bypass cache for authenticated requests
sub vcl_recv {
    if (req.http.Authorization) {
        return (pass);
    }
}

Backend Connection Failures

Issue: 503 Backend Fetch Failed

Diagnosis:

# Check backend health
varnishadm backend.list

# Check Varnish error log
sudo journalctl -u varnish -f

# Test backend directly
curl -I http://localhost:8080

Solutions:

# Verify backend is running
sudo systemctl status apache2  # or nginx

# Check firewall rules
sudo iptables -L | grep 8080

# Test backend connectivity
telnet localhost 8080

# Increase timeout in VCL
backend default {
    .connect_timeout = 10s;
    .first_byte_timeout = 60s;
}

Memory Issues

Issue: Varnish running out of memory

Diagnosis:

# Check Varnish memory usage
varnishstat -1 | grep -E 'SMA|storage'

# View cache size
du -sh /var/lib/varnish/

Solutions:

# Increase malloc size
# Edit /etc/default/varnish or /etc/varnish/varnish.params
VARNISH_STORAGE="malloc,1G"  # Increase from 256M to 1G

# Or use file-based storage
VARNISH_STORAGE="file,/var/lib/varnish/cache,2G"

# Restart Varnish
sudo systemctl restart varnish

VCL Syntax Errors

Issue: VCL fails to compile

Diagnosis:

# Check VCL syntax
sudo varnishd -C -f /etc/varnish/default.vcl

# View detailed error
sudo journalctl -u varnish | tail -20

Common errors:

Missing semicolon:

# Wrong:
set req.http.X-Custom = "value"

# Correct:
set req.http.X-Custom = "value";

Invalid return statement:

# Wrong:
sub vcl_recv {
    return (cache);  # 'cache' is not valid
}

# Correct:
sub vcl_recv {
    return (hash);  # Use 'hash' for caching
}

Best Practices

Performance Optimization

Use sufficient memory for cache:

# Allocate at least 1GB per 100,000 objects
VARNISH_STORAGE="malloc,2G"

Optimize TTL values:

# Balance freshness vs. performance
# Static assets: days/weeks
# Dynamic content: minutes/hours
# API responses: seconds/minutes

Enable compression:

sub vcl_backend_response {
    if (beresp.http.content-type ~ "text|javascript|json|css|xml") {
        set beresp.do_gzip = true;
    }
}

Use grace mode for high availability:

set beresp.grace = 6h;  # Serve stale if backend fails

Security Hardening

Restrict purge access:

acl purge_allowed {
    "localhost";
    "127.0.0.1";
    # Don't allow public access
}

Hide internal headers:

sub vcl_deliver {
    unset resp.http.X-Varnish;
    unset resp.http.Via;
    unset resp.http.X-Powered-By;
}

Implement rate limiting:

import std;
import vsthrottle;

sub vcl_recv {
    if (vsthrottle.is_denied(client.identity, 100, 10s)) {
        return (synth(429, "Too Many Requests"));
    }
}

Monitoring and Maintenance

Regular monitoring:

# Monitor cache hit rate (should be > 80%)
varnishstat -1 | grep cache_hit

# Watch for errors
varnishlog -q "RespStatus >= 500"

# Monitor memory usage
varnishstat -1 | grep -E 'SMA|storage'

Log rotation:

# Ensure Varnish logs are rotated
sudo nano /etc/logrotate.d/varnish

Regular cache warming:

# Script to warm cache after restarts
for url in /page1 /page2 /page3; do
    curl -s http://example.com$url > /dev/null
done

Caching Strategy

Cache selectively: Not everything should be cached
Set appropriate TTLs: Balance freshness and performance
Use grace mode: Improve availability during backend issues
Implement cache invalidation: Keep content fresh
Monitor hit rates: Optimize based on real data

Conclusion

Varnish Cache is an exceptionally powerful HTTP accelerator that can transform your web application's performance profile. By intelligently caching content in memory and serving it at blazing speeds, Varnish reduces backend load, improves response times, and enables your infrastructure to handle significantly more traffic.

Key takeaways from this guide:

Proper installation and configuration: Set up Varnish to listen on port 80 with backend on alternate port
VCL mastery: Use VCL to implement sophisticated caching policies
Backend configuration: Configure health checks and load balancing for reliability
Cache policies: Define appropriate TTLs and bypass rules for different content types
Advanced features: Leverage ESI, grace mode, and cache invalidation
Monitoring and optimization: Continuously monitor performance and optimize configurations

Varnish Cache is particularly effective for content-heavy websites, high-traffic applications, and APIs where read operations significantly outnumber writes. Combined with proper cache invalidation strategies, intelligent TTL configuration, and comprehensive monitoring, Varnish provides the performance acceleration needed for modern web applications.

Remember that caching is a balancing act between performance and freshness. Start with conservative TTL values, monitor cache hit rates, and gradually optimize based on your specific application requirements and traffic patterns. With proper configuration and monitoring, Varnish Cache can reduce response times from hundreds of milliseconds to single-digit milliseconds while reducing backend load by 80-95%.

Continue learning by exploring advanced topics like Varnish Modules (VMODs), custom VCL functions, integration with CDNs, and advanced load balancing strategies to further enhance your caching infrastructure.

Varnish Cache Configuration

Varnish Cache Configuration

Table of Contents

Prerequisites

Understanding Varnish Cache

What is Varnish Cache?

How Varnish Works

Benefits of Varnish Cache

Installation

Installing Varnish on Ubuntu/Debian

Installing Varnish on CentOS/Rocky Linux

Initial Configuration Files

Basic Configuration

Configuring Varnish Port

Basic VCL Configuration

VCL Configuration Language

VCL Basics

Conditional Logic in VCL

Variable Manipulation

Cache Policies

Cache Duration Configuration

Cache Bypass Rules

Cookie Handling

Backend Configuration

Multiple Backend Servers

Health Checks

Failover Configuration

Advanced Caching Strategies

Edge Side Includes (ESI)

Cache Invalidation

Grace Mode

Caching POST Requests

Verification and Testing

Check Varnish Status

Test Cache Functionality

Monitor Cache Performance

Debug Cache Behavior

Troubleshooting

Cache Not Working

Backend Connection Failures

Memory Issues

VCL Syntax Errors

Best Practices

Performance Optimization

Security Hardening

Monitoring and Maintenance

Caching Strategy

Conclusion

Latest Video

Get $20 Free Credit