Logrotate Configuration: Complete Guide

Introduction

Log files are essential for monitoring system health, debugging applications, and maintaining security, but without proper management, they can quickly consume valuable disk space and make log analysis difficult. Logrotate is a powerful utility that automates log file rotation, compression, and removal, ensuring your logs remain manageable while preserving historical data when needed.

Logrotate handles the entire lifecycle of log files by automatically rotating them based on size or time intervals, compressing old logs to save space, and removing ancient logs according to retention policies. This automation prevents disk space exhaustion, maintains system performance, and ensures compliance with data retention requirements.

In this comprehensive guide, we'll explore logrotate configuration from basic concepts to advanced implementations. You'll learn how to configure rotation policies for system logs, application logs, and custom services, implement compression strategies, manage retention periods, and troubleshoot common issues. Whether you're managing a single server or a fleet of systems, mastering logrotate is essential for maintaining a healthy logging infrastructure.

Prerequisites

Before configuring logrotate, ensure you have:

• A Linux server (Ubuntu 20.04/22.04, Debian 10/11, CentOS 7/8, Rocky Linux 8/9, or similar)
• Root or sudo access to modify system configurations
• Basic understanding of Linux file system and permissions
• Familiarity with log file locations (typically /var/log/)
• Text editor (nano, vim, or vi)

System Requirements:

• Logrotate package installed (usually pre-installed on most distributions)
• Sufficient disk space for temporary log files during rotation
• Cron daemon running for scheduled rotation

Installing and Verifying Logrotate

Most Linux distributions include logrotate by default, but let's verify and install if needed.

Installation

On Ubuntu/Debian:

# Check if logrotate is installed
dpkg -l | grep logrotate

# Install if not present
sudo apt update
sudo apt install logrotate -y

# Verify installation
logrotate --version

On CentOS/Rocky Linux/AlmaLinux:

# Check if logrotate is installed
rpm -qa | grep logrotate

# Install if not present
sudo yum install logrotate -y

# Verify installation
logrotate --version

Verify Logrotate Service

# Check logrotate systemd timer (if using systemd)
systemctl list-timers | grep logrotate

# View logrotate cron job
cat /etc/cron.daily/logrotate

# Check logrotate status file
cat /var/lib/logrotate/status
# or on older systems
cat /var/lib/logrotate.status

Understanding Logrotate Architecture

Main Configuration File

The primary configuration file is /etc/logrotate.conf:

# View main configuration
sudo cat /etc/logrotate.conf

Typical logrotate.conf structure:

# Global options
weekly
rotate 4
create
dateext
compress

# Package-specific configuration files
include /etc/logrotate.d

# System-specific logs
/var/log/wtmp {
    monthly
    create 0664 root utmp
    minsize 1M
    rotate 1
}

/var/log/btmp {
    missingok
    monthly
    create 0600 root utmp
    rotate 1
}

Configuration Directory

Application-specific configurations are stored in /etc/logrotate.d/:

# List all logrotate configurations
ls -l /etc/logrotate.d/

# Example configurations
/etc/logrotate.d/apache2
/etc/logrotate.d/nginx
/etc/logrotate.d/mysql-server
/etc/logrotate.d/rsyslog

Status File

Logrotate tracks rotation history in a status file:

# View current rotation status
sudo cat /var/lib/logrotate/status

# Example output:
# "/var/log/syslog" 2024-1-11-3:0:0
# "/var/log/auth.log" 2024-1-11-3:0:0
# "/var/log/nginx/access.log" 2024-1-10-3:0:0

Logrotate Configuration Syntax

Basic Configuration Structure

A logrotate configuration file consists of log file patterns and directives:

# Basic structure
/path/to/logfile {
    # Rotation directives
    directive1
    directive2 value

    # Scripts
    prerotate
        # commands before rotation
    endscript

    postrotate
        # commands after rotation
    endscript
}

Common Directives

Rotation Frequency:

daily           # Rotate logs daily
weekly          # Rotate logs weekly (on Sunday by default)
monthly         # Rotate logs monthly (on first day of month)
yearly          # Rotate logs yearly (on January 1st)

Rotation Conditions:

size 100M       # Rotate when file reaches 100MB
minsize 10M     # Rotate only if file is at least 10MB
maxsize 500M    # Force rotation if file exceeds 500MB
rotate 7        # Keep 7 rotated logs
maxage 30       # Remove logs older than 30 days

File Handling:

compress        # Compress rotated logs with gzip
nocompress      # Don't compress logs
delaycompress   # Compress on next rotation (keep one uncompressed)
compresscmd gzip        # Specify compression command
compressext .gz         # Specify compression extension
compressoptions -9      # Maximum compression

create 0640 user group  # Create new log with permissions
nocreate                # Don't create new log file
copy                    # Copy log instead of moving
copytruncate            # Copy then truncate original log

Missing Files:

missingok       # Don't error if log file is missing
notifempty      # Don't rotate if log is empty
ifempty         # Rotate even if log is empty

Old Log Handling:

dateext         # Add date extension (e.g., .20240111)
dateformat -%Y%m%d      # Custom date format
extension .log  # Extension to preserve
olddir /path/to/old     # Move old logs to directory
noolddir                # Keep old logs in same directory

Mailing:

mail [email protected]  # Email log before deletion
nomail                  # Don't email logs
mailfirst               # Mail newest log
maillast                # Mail oldest log (default)

Shared Logs:

sharedscripts   # Run scripts once for all matching logs
nosharedscripts # Run scripts for each log separately

Basic Logrotate Configurations

Simple Daily Rotation

# Create configuration: /etc/logrotate.d/myapp
sudo nano /etc/logrotate.d/myapp

/var/log/myapp/*.log {
    daily
    rotate 7
    compress
    delaycompress
    missingok
    notifempty
    create 0640 myapp myapp
}

Explanation:

• Rotate all .log files in /var/log/myapp/ daily
• Keep 7 days of logs
• Compress old logs (but delay compression by one rotation)
• Don't error if logs are missing
• Don't rotate empty logs
• Create new logs with specific permissions and ownership

Size-Based Rotation

# Configuration for large application logs
/var/log/application/app.log {
    size 100M
    rotate 5
    compress
    missingok
    notifempty
    create 0644 appuser appgroup
    dateext
}

Explanation:

• Rotate when log reaches 100MB
• Keep 5 rotated copies
• Add date extension to rotated files
• Compress rotated logs immediately

Weekly Rotation with Old Directory

/var/log/backup/*.log {
    weekly
    rotate 4
    compress
    delaycompress
    missingok
    notifempty
    create 0640 root root
    olddir /var/log/backup/old
    dateext
    dateformat -%Y%m%d
}

Explanation:

• Rotate weekly (every Sunday)
• Keep 4 weeks of logs
• Move old logs to separate directory
• Add custom date format to filenames

Advanced Logrotate Configurations

Web Server Logs (Nginx)

# /etc/logrotate.d/nginx
/var/log/nginx/*.log {
    daily
    rotate 14
    compress
    delaycompress
    missingok
    notifempty
    create 0640 www-data adm
    sharedscripts
    prerotate
        if [ -d /etc/logrotate.d/httpd-prerotate ]; then \
            run-parts /etc/logrotate.d/httpd-prerotate; \
        fi
    endscript
    postrotate
        [ -s /run/nginx.pid ] && kill -USR1 `cat /run/nginx.pid`
    endscript
}

Key features:

• Rotates daily, keeps 2 weeks
• Uses sharedscripts to run postrotate once
• Signals nginx to reopen log files after rotation

Apache Web Server

# /etc/logrotate.d/apache2
/var/log/apache2/*.log {
    daily
    rotate 14
    compress
    delaycompress
    missingok
    notifempty
    create 0640 root adm
    sharedscripts
    postrotate
        /usr/sbin/apache2ctl graceful > /dev/null 2>&1 || true
    endscript
}

MySQL/MariaDB Logs

# /etc/logrotate.d/mysql-server
/var/log/mysql/*.log {
    daily
    rotate 7
    compress
    delaycompress
    missingok
    notifempty
    create 0640 mysql adm
    sharedscripts
    postrotate
        if test -x /usr/bin/mysqladmin && \
           /usr/bin/mysqladmin ping &>/dev/null
        then
           /usr/bin/mysqladmin --local flush-error-log \
             flush-engine-log flush-general-log flush-slow-log
        fi
    endscript
}

Application Logs with Custom Scripts

# /etc/logrotate.d/custom-app
/var/log/custom-app/*.log {
    daily
    rotate 30
    compress
    delaycompress
    missingok
    notifempty
    create 0644 appuser appgroup
    dateext
    dateformat -%Y%m%d-%s
    maxsize 500M

    prerotate
        # Notify monitoring system
        echo "Rotating logs for custom-app" | \
            logger -t logrotate

        # Backup critical logs before rotation
        if [ -f /var/log/custom-app/critical.log ]; then
            cp /var/log/custom-app/critical.log \
               /backup/logs/critical-$(date +%Y%m%d).log
        fi
    endscript

    postrotate
        # Restart application to use new log file
        systemctl reload custom-app.service

        # Update log index
        /usr/local/bin/update-log-index.sh
    endscript

    firstaction
        # Run before any logs are rotated
        mkdir -p /backup/logs
    endfirstaction

    lastaction
        # Run after all logs are rotated
        /usr/local/bin/cleanup-old-backups.sh
    endlastaction
}

Multiple Log Files with Different Policies

# /etc/logrotate.d/multi-app
# Access logs - high volume, short retention
/var/log/multi-app/access.log {
    daily
    rotate 7
    compress
    delaycompress
    missingok
    notifempty
    create 0644 appuser appgroup
    size 100M
}

# Error logs - keep longer
/var/log/multi-app/error.log {
    weekly
    rotate 12
    compress
    delaycompress
    missingok
    notifempty
    create 0644 appuser appgroup
    maxsize 50M
}

# Audit logs - long retention, no compression
/var/log/multi-app/audit.log {
    monthly
    rotate 24
    nocompress
    missingok
    notifempty
    create 0600 appuser appgroup
    dateext
    dateformat -%Y%m
}

Wildcard Patterns

# Rotate all logs in multiple directories
/var/log/app1/*.log /var/log/app2/*.log /var/log/app3/*.log {
    daily
    rotate 7
    compress
    missingok
    notifempty
    sharedscripts
    postrotate
        systemctl reload app-logger.service
    endscript
}

# Exclude specific files
/var/log/application/*.log {
    # This will rotate all .log files except debug.log
    daily
    rotate 5
    compress
    missingok
    notifempty

    # Use tabooext to exclude files
    tabooext + .debug
}

Copy and Truncate for Open Files

# For applications that don't support log reopening
/var/log/legacy-app/app.log {
    daily
    rotate 7
    compress
    copytruncate
    missingok
    notifempty

    # Note: copytruncate may lose some log entries
    # during the copy operation
}

Compression Strategies

Using Different Compression Tools

Gzip (default):

/var/log/app/*.log {
    compress
    compresscmd /usr/bin/gzip
    compressoptions -9
    compressext .gz
}

Bzip2 (better compression, slower):

/var/log/app/*.log {
    compress
    compresscmd /usr/bin/bzip2
    uncompresscmd /usr/bin/bunzip2
    compressext .bz2
    compressoptions -9
}

XZ (best compression, slowest):

/var/log/app/*.log {
    compress
    compresscmd /usr/bin/xz
    uncompresscmd /usr/bin/unxz
    compressext .xz
    compressoptions -9e
}

Zstd (fast compression with good ratio):

/var/log/app/*.log {
    compress
    compresscmd /usr/bin/zstd
    uncompresscmd /usr/bin/unzstd
    compressext .zst
    compressoptions -19
}

Delayed Compression Strategy

# Keep most recent rotated log uncompressed
/var/log/app/*.log {
    daily
    rotate 7
    compress
    delaycompress
    missingok
    notifempty
}

# Result:
# app.log           (current, uncompressed)
# app.log.1         (yesterday, uncompressed)
# app.log.2.gz      (2 days ago, compressed)
# app.log.3.gz      (3 days ago, compressed)

Testing and Debugging Logrotate

Test Configuration Syntax

# Test specific configuration file
sudo logrotate -d /etc/logrotate.d/nginx

# Test main configuration
sudo logrotate -d /etc/logrotate.conf

# Verbose debug output
sudo logrotate -dv /etc/logrotate.d/myapp

Force Rotation

# Force rotation even if conditions aren't met
sudo logrotate -f /etc/logrotate.conf

# Force specific configuration
sudo logrotate -f /etc/logrotate.d/nginx

# Force with verbose output
sudo logrotate -fv /etc/logrotate.d/myapp

Manual Rotation Test

# Test rotation without updating status file
sudo logrotate -d --state /tmp/logrotate-test.status /etc/logrotate.d/myapp

# Perform actual rotation with temporary status
sudo logrotate --state /tmp/logrotate-test.status /etc/logrotate.d/myapp

Verify Rotation Results

# Check status file
sudo cat /var/lib/logrotate/status | grep myapp

# List rotated logs
ls -lht /var/log/myapp/

# Verify new log creation
ls -l /var/log/myapp/app.log

# Check log permissions
stat /var/log/myapp/app.log

Enable Logrotate Logging

# Add to /etc/logrotate.conf or specific config
/var/log/myapp/*.log {
    daily
    rotate 7

    # Log rotation events to syslog
    prerotate
        logger -t logrotate "Rotating myapp logs"
    endscript

    postrotate
        logger -t logrotate "Completed myapp log rotation"
    endscript
}

# View rotation logs
sudo grep logrotate /var/log/syslog

Scheduling and Automation

Cron-Based Execution

Logrotate typically runs via cron:

# View daily cron job
cat /etc/cron.daily/logrotate

# Typical content:
#!/bin/sh
test -x /usr/sbin/logrotate || exit 0
/usr/sbin/logrotate /etc/logrotate.conf

Custom Scheduling

Run logrotate hourly:

# Create hourly script
sudo nano /etc/cron.hourly/logrotate-hourly

#!/bin/bash
/usr/sbin/logrotate /etc/logrotate.d/high-volume-app

# Make executable
sudo chmod +x /etc/cron.hourly/logrotate-hourly

Custom cron schedule:

# Edit crontab
sudo crontab -e

# Run at 3 AM daily
0 3 * * * /usr/sbin/logrotate /etc/logrotate.conf

# Run every 6 hours
0 */6 * * * /usr/sbin/logrotate /etc/logrotate.d/frequent-app

Systemd Timer

Create systemd timer for logrotate:

# Create timer unit
sudo nano /etc/systemd/system/logrotate.timer

[Unit]
Description=Daily rotation of log files
Documentation=man:logrotate(8) man:logrotate.conf(5)

[Timer]
OnCalendar=daily
AccuracySec=1h
Persistent=true

[Install]
WantedBy=timers.target

Create service unit:

sudo nano /etc/systemd/system/logrotate.service

[Unit]
Description=Rotate log files
Documentation=man:logrotate(8) man:logrotate.conf(5)
ConditionACPower=true

[Service]
Type=oneshot
ExecStart=/usr/sbin/logrotate /etc/logrotate.conf

Nice=19
IOSchedulingClass=best-effort
IOSchedulingPriority=7

PrivateTmp=true

Enable and start timer:

sudo systemctl daemon-reload
sudo systemctl enable logrotate.timer
sudo systemctl start logrotate.timer
sudo systemctl status logrotate.timer

# List timer schedule
systemctl list-timers logrotate.timer

Monitoring and Alerting

Monitor Rotation Success

#!/bin/bash
# check-logrotate.sh - Monitor logrotate execution

STATUS_FILE="/var/lib/logrotate/status"
LOG_DIR="/var/log/myapp"
EMAIL="[email protected]"

# Check if rotation happened today
TODAY=$(date +%Y-%m-%d)
LAST_ROTATION=$(grep "$LOG_DIR" "$STATUS_FILE" | awk '{print $2}' | cut -d'-' -f1-3)

if [ "$LAST_ROTATION" != "$TODAY" ]; then
    echo "WARNING: Logrotate may not have run today for $LOG_DIR" | \
        mail -s "Logrotate Alert" "$EMAIL"
fi

# Check for old uncompressed logs
OLD_LOGS=$(find "$LOG_DIR" -name "*.log.*" -not -name "*.gz" -mtime +2)
if [ -n "$OLD_LOGS" ]; then
    echo "Found old uncompressed logs: $OLD_LOGS" | \
        mail -s "Logrotate Compression Issue" "$EMAIL"
fi

Disk Space Monitoring

#!/bin/bash
# monitor-log-space.sh - Alert on log directory size

LOG_DIR="/var/log"
THRESHOLD=80  # Percentage
EMAIL="[email protected]"

USAGE=$(df "$LOG_DIR" | tail -1 | awk '{print $5}' | sed 's/%//')

if [ "$USAGE" -gt "$THRESHOLD" ]; then
    {
        echo "WARNING: $LOG_DIR is ${USAGE}% full"
        echo ""
        echo "Largest log files:"
        du -sh "$LOG_DIR"/* | sort -rh | head -10
        echo ""
        echo "Logrotate status:"
        tail -20 /var/lib/logrotate/status
    } | mail -s "Log Directory Space Alert" "$EMAIL"
fi

Rotation Failure Detection

#!/bin/bash
# detect-rotation-failures.sh

# Check for failed rotations in syslog
ERRORS=$(grep logrotate /var/log/syslog | grep -i "error\|fail" | tail -10)

if [ -n "$ERRORS" ]; then
    echo "Logrotate errors detected:"
    echo "$ERRORS"
    echo "$ERRORS" | mail -s "Logrotate Errors Detected" [email protected]
fi

# Check for very large log files (may indicate rotation failure)
find /var/log -type f -size +1G -exec ls -lh {} \; | while read line; do
    echo "Large log file detected: $line"
    echo "$line" | mail -s "Large Log File Alert" [email protected]
done

Troubleshooting Common Issues

Issue 1: Logrotate Not Running

Diagnosis:

# Check if logrotate is installed
which logrotate

# Verify cron daemon is running
systemctl status cron  # Debian/Ubuntu
systemctl status crond # CentOS/Rocky

# Check cron job
ls -l /etc/cron.daily/logrotate

# Test manual execution
sudo /etc/cron.daily/logrotate

# Check for errors
sudo logrotate -v /etc/logrotate.conf

Solution:

# Reinstall logrotate
sudo apt install --reinstall logrotate

# Fix cron permissions
sudo chmod +x /etc/cron.daily/logrotate

# Restart cron daemon
sudo systemctl restart cron

Issue 2: Logs Not Rotating

Diagnosis:

# Check configuration syntax
sudo logrotate -d /etc/logrotate.d/myapp

# Verify log file exists
ls -l /var/log/myapp/app.log

# Check status file
sudo grep "myapp" /var/lib/logrotate/status

# Test force rotation
sudo logrotate -f /etc/logrotate.d/myapp

Common causes:

• notifempty directive with empty log
• Size threshold not met
• Incorrect file path or permissions
• Syntax errors in configuration

Solution:

# Remove notifempty if log is empty
# Or ensure log has content

# Fix permissions
sudo chown root:root /etc/logrotate.d/myapp
sudo chmod 644 /etc/logrotate.d/myapp

# Correct syntax errors
sudo logrotate -d /etc/logrotate.d/myapp

Issue 3: Permission Denied Errors

Diagnosis:

# Run with verbose output
sudo logrotate -dv /etc/logrotate.d/myapp

# Check log file permissions
ls -l /var/log/myapp/

# Check logrotate process user
ps aux | grep logrotate

Solution:

# Ensure logrotate runs as root
sudo logrotate /etc/logrotate.conf

# Fix log directory permissions
sudo chmod 755 /var/log/myapp/
sudo chown myapp:myapp /var/log/myapp/*.log

# Update create directive
# In /etc/logrotate.d/myapp:
create 0644 myapp myapp

Issue 4: Compression Failing

Diagnosis:

# Check if compression tool is installed
which gzip

# Verify compression settings
sudo logrotate -dv /etc/logrotate.d/myapp | grep compress

# Check for compressed files
ls -l /var/log/myapp/*.gz

Solution:

# Install compression tool
sudo apt install gzip

# Check disk space
df -h /var/log

# Test compression manually
gzip -t /var/log/myapp/app.log.1.gz

# Remove failed compression
sudo rm /var/log/myapp/*.gz
sudo logrotate -f /etc/logrotate.d/myapp

Issue 5: Application Won't Write to New Log

Diagnosis:

# Check if new log file was created
ls -l /var/log/myapp/app.log

# Verify permissions
stat /var/log/myapp/app.log

# Check application process
lsof | grep app.log

Solution:

# Use copytruncate for applications that can't reopen logs
/var/log/myapp/app.log {
    copytruncate
    daily
    rotate 7
}

# Or add postrotate script to reload application
postrotate
    systemctl reload myapp.service
endscript

# Or send signal to reopen logs
postrotate
    kill -HUP $(cat /var/run/myapp.pid)
endscript

Best Practices

Configuration Organization

# 1. Use separate config files per application
/etc/logrotate.d/nginx
/etc/logrotate.d/mysql
/etc/logrotate.d/myapp

# 2. Use descriptive filenames
/etc/logrotate.d/application-access-logs
/etc/logrotate.d/application-error-logs

# 3. Document unusual configurations
# /etc/logrotate.d/myapp
# Special rotation for high-volume application
# Rotates hourly via cron.hourly instead of daily
/var/log/myapp/*.log {
    size 100M
    rotate 24
    compress
}

Security Considerations

# Secure configuration files
sudo chmod 644 /etc/logrotate.d/*
sudo chown root:root /etc/logrotate.d/*

# Restrict log access
create 0640 appuser appgroup

# Don't email sensitive logs
nomail

# Use secure old directory
olddir /var/log/secure-archive
create 0600 root root

Performance Optimization

# Use delayed compression for frequently accessed logs
delaycompress

# Compress in background for large files
compress
compressoptions -1  # Fast compression

# Use sharedscripts to reduce overhead
sharedscripts
postrotate
    systemctl reload nginx
endscript

# Limit rotation frequency for high-volume logs
size 500M
maxsize 1G

Retention Policies

# Short-term high-volume logs
/var/log/app/access.log {
    daily
    rotate 7
    maxage 7
}

# Medium-term error logs
/var/log/app/error.log {
    weekly
    rotate 12
    maxage 90
}

# Long-term audit logs
/var/log/app/audit.log {
    monthly
    rotate 24
    maxage 730
    nocompress
}

Conclusion

Logrotate is an essential tool for maintaining a healthy logging infrastructure on Linux systems. Proper configuration ensures that logs are rotated efficiently, disk space is managed effectively, and historical data is retained according to your requirements.

Key takeaways:

1. Automated management - Logrotate handles the complete lifecycle of log files without manual intervention
2. Flexible policies - Configure rotation based on time, size, or both to match your application needs
3. Compression strategies - Balance disk space savings with access requirements using appropriate compression
4. Application integration - Use pre/postrotate scripts to ensure applications handle log rotation correctly
5. Monitoring - Regularly verify that rotation is working as expected and disk space is under control

By implementing the configurations and practices outlined in this guide, you'll maintain manageable log files, prevent disk space issues, ensure compliance with retention requirements, and facilitate efficient log analysis when troubleshooting issues or investigating security incidents.

Remember to test your logrotate configurations thoroughly before deploying to production, monitor rotation execution regularly, and adjust retention policies as your storage capacity and compliance requirements evolve.