Redis Sentinel High Availability

Redis Sentinel is a distributed system that monitors Redis instances and automatically handles failover when the primary server becomes unavailable. It proporciona high availability without relying on external dependencies, using consensus algorithms to elect a new primary and redirect client connections. Esta guía completa cubre Sentinel deployment, configuration, monitoring, automatic failover behavior, client integration, and operational procedures for production Redis deployments.

Tabla de Contenidos

Architecture and Concepts

Redis Sentinel is a separate system running alongside Redis instances that monitors their health and coordinates failover. Multiple Sentinel instances form a consensus group that detects primary failure and promotes a réplica without administrator intervention. Sentinel watches Redis instances continuously, detecting failures through a combination of health checks and heartbeat monitoring.

The Sentinel cluster itself requires a quorum (typically 3 instances) for voting on failover decisions. If the primary becomes unreachable and a quorum of Sentinels agrees, one of the réplicas is promoted to primary and other Sentinels reconfigure the remaining réplicas to replicate from the new primary.

Instalación

Instala Redis Server and Sentinel. First install Redis:

# Ubuntu/Debian
sudo apt-get update
sudo apt-get install -y redis-server redis-tools

# CentOS/RHEL
sudo dnf install -y redis

# Verifica installation
redis-server --version
redis-cli --version
redis-sentinel --version

Build from source for latest version:

# Download Redis
cd /tmp
wget http://download.redis.io/redis-stable.tar.gz
tar -xzf redis-stable.tar.gz
cd redis-stable

# Compile
make
make test

# Instala
sudo make install PREFIX=/usr/local

# Crea symlinks
sudo ln -s /usr/local/bin/redis-server /usr/bin/redis-server
sudo ln -s /usr/local/bin/redis-cli /usr/bin/redis-cli
sudo ln -s /usr/local/bin/redis-sentinel /usr/bin/redis-sentinel

# Verifica
redis-server --version

Crea dedicated user and directories:

# Crea redis user
sudo useradd -r -s /bin/false redis 2>/dev/null || true

# Crea directories
sudo mkdir -p /var/lib/redis /var/log/redis /etc/redis
sudo chown -R redis:redis /var/lib/redis /var/log/redis
sudo chmod 755 /var/lib/redis /var/log/redis

Sentinel Configuración

Configura Redis Sentinel. Crea sentinel configuration file:

sudo nano /etc/redis/sentinel.conf

Add comprehensive Sentinel configuration:

# Listening puerto
puerto 26379

# Sentinel monitor configuration
# sentinel monitor <master-name> <master-ip> <master-puerto> <quorum>
sentinel monitor mymaster 192.168.1.10 6379 2

# Timeout for detecting master as down
sentinel down-after-milliseconds mymaster 30000

# Timeout for failover
sentinel failover-timeout mymaster 180000

# Number of réplicas to reconfigure in parallel
sentinel parallel-syncs mymaster 1

# Notification script (optional)
# sentinel notification-script mymaster /path/to/notification.sh

# Client reconfiguration script (optional)
# sentinel client-reconfig-script mymaster /path/to/reconfig.sh

# Logging
logfile /var/log/redis/sentinel.log
loglevel notice
syslog-enabled yes
syslog-ident redis-sentinel

# Directory for state files
dir /var/lib/redis

# Sentinel authentication (if Redis requires password)
# sentinel auth-pass mymaster your-redis-password

# Quorum and failover policy
sentinel deny-scripts-reconfig yes
sentinel announce-ip 192.168.1.20
sentinel announce-puerto 26379

# Connection settings
sentinel tcp-backlog 511
sentinel tcp-keepalive 300

# Default réplica priority (lower = higher priority for promotion)
sentinel réplica-priority mymaster 100

Set permissions on configuration:

sudo chown redis:redis /etc/redis/sentinel.conf
sudo chmod 600 /etc/redis/sentinel.conf

Crea systemd servicio for Sentinel:

sudo nano /etc/systemd/system/redis-sentinel.servicio

Add:

[Unit]
Description=Redis Sentinel
After=red.target

[Servicio]
Type=simple
User=redis
Group=redis

ExecStart=/usr/local/bin/redis-sentinel /etc/redis/sentinel.conf
ExecStop=/bin/kill -TERM $MAINPID
Reinicia=always
RestartSec=10

StandardOutput=journal
StandardError=journal

[Instala]
WantedBy=multi-user.target

Habilita and start Sentinel:

sudo systemctl daemon-reload
sudo systemctl enable redis-sentinel
sudo systemctl start redis-sentinel
sudo systemctl status redis-sentinel

# Monitorea logs
sudo journalctl -u redis-sentinel -f

Multi-Sentinel Despliegue

Despliega multiple Sentinel instances for redundancy. Configura first Sentinel instance (on different server):

# SSH to sentinel1 (192.168.1.20)
ssh [email protected]

# Copy sentinel configuration from primary setup
sudo nano /etc/redis/sentinel.conf

Add configuration:

puerto 26379
sentinel monitor mymaster 192.168.1.10 6379 2
sentinel down-after-milliseconds mymaster 30000
sentinel failover-timeout mymaster 180000
sentinel parallel-syncs mymaster 1
logfile /var/log/redis/sentinel.log
dir /var/lib/redis
announce-ip 192.168.1.20
announce-puerto 26379

Configura second Sentinel instance (192.168.1.21):

# SSH to sentinel2
ssh [email protected]

# Configura Sentinel
sudo nano /etc/redis/sentinel.conf

Add:

puerto 26379
sentinel monitor mymaster 192.168.1.10 6379 2
sentinel down-after-milliseconds mymaster 30000
sentinel failover-timeout mymaster 180000
sentinel parallel-syncs mymaster 1
logfile /var/log/redis/sentinel.log
dir /var/lib/redis
announce-ip 192.168.1.21
announce-puerto 26379

Configura third Sentinel instance (192.168.1.22):

# SSH to sentinel3
ssh [email protected]

# Configura Sentinel
sudo nano /etc/redis/sentinel.conf

Add:

puerto 26379
sentinel monitor mymaster 192.168.1.10 6379 2
sentinel down-after-milliseconds mymaster 30000
sentinel failover-timeout mymaster 180000
sentinel parallel-syncs mymaster 1
logfile /var/log/redis/sentinel.log
dir /var/lib/redis
announce-ip 192.168.1.22
announce-puerto 26379

Inicia all Sentinel instances and verifica cluster:

# On each Sentinel host
sudo systemctl start redis-sentinel
sudo systemctl status redis-sentinel

# From any Sentinel, verifica cluster status
redis-cli -p 26379 SENTINEL masters

# Get current master info
redis-cli -p 26379 SENTINEL get-master-addr-by-name mymaster

# Check Sentinel nodos
redis-cli -p 26379 SENTINEL sentinels mymaster

# Check réplicas
redis-cli -p 26379 SENTINEL réplicas mymaster

Monitoreo and Health Checks

Monitorea Sentinel and Redis health:

# Conecta to Sentinel
redis-cli -p 26379

# View all monitored masters
SENTINEL masters

# Get detailed master information
SENTINEL master mymaster

# Check current leader
SENTINEL is-master-down-by-addr 192.168.1.10 6379 0

# Get master status
SENTINEL get-master-addr-by-name mymaster

# Check réplica status
SENTINEL réplicas mymaster

# View Sentinel info
INFO sentinel

# Exit
EXIT

Monitorea from multiple Sentinels:

#!/bin/bash
# Check Sentinel status across cluster

SENTINELS=("192.168.1.20" "192.168.1.21" "192.168.1.22")

for sentinel in "${SENTINELS[@]}"; do
  echo "=== Sentinel at $sentinel ==="
  redis-cli -h $sentinel -p 26379 SENTINEL get-master-addr-by-name mymaster
  redis-cli -h $sentinel -p 26379 SENTINEL sentinel-state mymaster | head -5
done

Crea monitoring script:

#!/bin/bash
# /usr/local/bin/redis-sentinel-monitor.sh

MASTER_HOST="192.168.1.10"
MASTER_PORT="6379"
SENTINELS=("192.168.1.20" "192.168.1.21" "192.168.1.22")

echo "=== Redis Sentinel Status ==="
echo "Timestamp: $(date)"

# Check master
echo ""
echo "Master Status:"
redis-cli -h $MASTER_HOST -p $MASTER_PORT ping > /dev/null 2>&1 && echo "Master: ONLINE" || echo "Master: OFFLINE"

# Check all Sentinels
echo ""
echo "Sentinel Status:"
for sentinel in "${SENTINELS[@]}"; do
  result=$(redis-cli -h $sentinel -p 26379 PING 2>/dev/null)
  [ "$result" = "PONG" ] && echo "Sentinel $sentinel: OK" || echo "Sentinel $sentinel: FAILED"
done

# Get master address from Sentinel
echo ""
echo "Current Master:"
redis-cli -h ${SENTINELS[0]} -p 26379 SENTINEL get-master-addr-by-name mymaster

# Check réplica count
echo ""
echo "Réplica Status:"
redis-cli -h $MASTER_HOST -p $MASTER_PORT INFO replication | grep -E "role|connected_slaves"

Automatic Failover

Understand and monitor automatic failover:

# Simulate master failure
redis-cli -h 192.168.1.10 -p 6379 shutdown

# Monitorea Sentinel detection
redis-cli -p 26379 SENTINEL get-master-addr-by-name mymaster
# Result will show new master after failover

# Watch logs during failover
tail -f /var/log/redis/sentinel.log | grep -E "failover|switch"

# After failover, verifica new master
redis-cli -h 192.168.1.11 -p 6379 INFO replication

# Rejoin original master (after recovery)
# Inicia Redis on original master
redis-server /etc/redis/redis.conf

# Monitorea Sentinel reconfiguring it as réplica
tail -f /var/log/redis/sentinel.log

Monitorea failover events:

#!/bin/bash
# Monitorea and log failover events

redis-cli -p 26379 SUBSCRIBE +master-failover-began \
                              +master-failover-end \
                              +convert-to-réplica \
                              +role-change |
while read line; do
  echo "$(date): $line" >> /var/log/redis/failover-events.log
done

Client Configuración

Configura Redis client libraries for Sentinel support:

# Python example with redis-py
from redis.sentinel import Sentinel

# Crea Sentinel connection
sentinel = Sentinel(
    sentinels=[
        ('192.168.1.20', 26379),
        ('192.168.1.21', 26379),
        ('192.168.1.22', 26379)
    ],
    socket_timeout=0.1
)

# Get master connection
master = sentinel.master_for('mymaster', socket_timeout=0.1)
master.set('key', 'value')

# Get slave connection for reads
slave = sentinel.slave_for('mymaster', socket_timeout=0.1)
value = slave.get('key')

Nodo.js example:

// Using nodo-redis with Sentinel
const redis = require('redis');

const sentinel = redis.createClient({
  sentinels: [
    { host: '192.168.1.20', puerto: 26379 },
    { host: '192.168.1.21', puerto: 26379 },
    { host: '192.168.1.22', puerto: 26379 }
  ],
  name: 'mymaster',
  sentinelRetryStrategy: (times) => {
    return Math.min(times * 10, 1000);
  }
});

sentinel.on('connect', () => {
  console.log('Connected to Redis via Sentinel');
});

Java example with Jedis:

// Using Jedis with Sentinel
import redis.clients.jedis.Jedis;
import redis.clients.jedis.JedisSentinelPool;
import redis.clients.util.Pool;

Set<String> sentinels = new HashSet<>(Arrays.asList(
    "192.168.1.20:26379",
    "192.168.1.21:26379",
    "192.168.1.22:26379"
));

JedisSentinelPool sentinelPool = new JedisSentinelPool(
    "mymaster",
    sentinels
);

try (Jedis jedis = sentinelPool.getResource()) {
    jedis.set("key", "value");
    String value = jedis.get("key");
}

Notification Scripts

Configura scripts to execute on failover events:

# Crea notification script
sudo nano /usr/local/bin/redis-failover-notification.sh

Add:

#!/bin/bash
# Redis Sentinel notification script

MASTER_NAME=$1
ROLE=$2
STATE=$3

# Log the event
echo "$(date): Sentinel event - Master: $MASTER_NAME, Role: $ROLE, State: $STATE" >> /var/log/redis/failover-notifications.log

# Send email notification
SUBJECT="Redis Failover Event: $MASTER_NAME"
BODY="Time: $(date)\nMaster: $MASTER_NAME\nRole: $ROLE\nState: $STATE"

# Uncomment to enable email
# echo -e "$BODY" | mail -s "$SUBJECT" [email protected]

# Actualiza load balancer or DNS
case $STATE in
    "failover-end")
        echo "Failover completed for $MASTER_NAME"
        # Actualiza load balancer to point to new master
        # curl -X POST http://load-balancer:8080/update-redis-endpoint
        ;;
esac

exit 0

Set permissions and configure:

sudo chmod +x /usr/local/bin/redis-failover-notification.sh
sudo chown redis:redis /usr/local/bin/redis-failover-notification.sh

# Actualiza sentinel.conf to use the script
sudo nano /etc/redis/sentinel.conf

# Add line
sentinel notification-script mymaster /usr/local/bin/redis-failover-notification.sh

Operational Procedures

Perform maintenance operations safely:

# Graceful master failover (no downtime)
redis-cli -p 26379 SENTINEL failover mymaster

# Monitorea failover progress
redis-cli -p 26379 SENTINEL master mymaster | grep failover

# Resync after maintenance
redis-cli -h 192.168.1.10 -p 6379 SLAVEOF 192.168.1.11 6379

# Restaura original master role
redis-cli -h 192.168.1.10 -p 6379 SLAVEOF NO ONE

Reset Sentinel state:

# Clear Sentinel state files (requires restart)
sudo rm /var/lib/redis/sentinel-*.conf
sudo systemctl restart redis-sentinel

# Reconfigure Sentinel if needed
redis-cli -p 26379 SENTINEL reset \*

Solución de Problemas

Depura common Sentinel issues:

# Check if Sentinel can see master
redis-cli -p 26379 SENTINEL is-master-down-by-addr 192.168.1.10 6379 0

# Get detailed master state
redis-cli -p 26379 SENTINEL master mymaster

# Check Sentinel logs
sudo tail -100 /var/log/redis/sentinel.log

# Verifica red connectivity
ping 192.168.1.10
redis-cli -h 192.168.1.10 -p 6379 ping

# Check Sentinel quorum
redis-cli -p 26379 SENTINEL sentinels mymaster | wc -l

# Monitorea failover attempt
strace -e trace=red redis-sentinel /etc/redis/sentinel.conf

Conclusión

Redis Sentinel proporciona automated failover and high availability for Redis deployments without requiring external tools. Its lightweight nature, built-in consensus mechanism, and automatic client notification make it ideal for production environments where Redis availability is critical. By properly configuring multiple Sentinel instances, monitoring health continuously, and testing failover scenarios, you asegúrate de que your Redis infrastructure maintains availability through hardware failures and planned maintenance. Sentinel's simplicity combined with its reliability makes it the recommended approach for Redis high availability in most production deployments.