Email Logs: Analysis and Diagnostics Complete Guide

Introduction

Email server logs are your most valuable diagnostic tool, containing detailed information about every message, connection, authentication attempt, and error that occurs on your system. However, the sheer volume of data in mail logs can be overwhelming—a busy server might generate thousands of log entries per hour. Without effective analysis techniques, critical information gets buried in noise.

Understanding how to read, analyze, and extract meaningful insights from email logs is essential for maintaining a reliable mail server. Logs reveal delivery problems, authentication failures, spam patterns, performance bottlenecks, security threats, and configuration issues. They provide the evidence needed to diagnose problems, verify successful delivery, track message paths, and optimize server performance.

This comprehensive guide teaches you how to effectively analyze email logs using command-line tools, identify common patterns and issues, extract specific information, create automated monitoring scripts, and maintain logs for optimal performance and compliance. You'll learn to quickly find the information you need, spot problems before they become critical, and maintain complete visibility into your email infrastructure.

Whether you're tracking down a missing email, investigating authentication failures, monitoring for attacks, or simply verifying your server is operating correctly, this guide provides the knowledge and commands needed to master email log analysis.

Prerequisites

Before diving into log analysis, ensure you have:

System Access

  • Root or sudo access to the mail server
  • SSH access to the server
  • Understanding of your log rotation schedule
  • Access to backup logs if needed

Knowledge Requirements

  • Basic Linux command-line proficiency
  • Understanding of email protocols (SMTP, IMAP, POP3)
  • Familiarity with Postfix and Dovecot
  • Basic regex knowledge (helpful but not required)

Tools Required

Standard Linux tools should be available:

# Verify tools are available
which grep tail cat less awk sed sort uniq wc head

# Install if missing
sudo apt install coreutils grep gawk sed -y

Understanding Log File Locations

Postfix Logs

Ubuntu/Debian:

/var/log/mail.log        # Main mail log
/var/log/mail.err        # Mail errors only
/var/log/mail.warn       # Warnings

CentOS/Rocky Linux:

/var/log/maillog         # Main mail log

Alternative/systemd systems:

# View via journalctl
journalctl -u postfix

Dovecot Logs

Configured location (check dovecot.conf):

/var/log/dovecot.log     # Main Dovecot log
/var/log/dovecot-info.log

Or in system logs:

/var/log/syslog          # Ubuntu/Debian
/var/log/messages        # CentOS/Rocky

Other Related Logs

/var/log/auth.log        # Authentication attempts
/var/log/syslog          # System messages
/var/log/spamassassin/spamd.log  # SpamAssassin
/var/log/opendkim.log    # OpenDKIM (if configured)

Understanding Log Entry Format

Postfix Log Format

Standard Postfix log entry:

Jan 11 10:30:45 mail postfix/smtp[12345]: ABCD1234: to=<[email protected]>, relay=mx.example.com[203.0.113.10]:25, delay=0.52, delays=0.01/0.02/0.15/0.34, dsn=2.0.0, status=sent (250 OK)

Components:

  • Jan 11 10:30:45 - Timestamp
  • mail - Hostname
  • postfix/smtp - Service/daemon
  • [12345] - Process ID
  • ABCD1234 - Queue ID (unique message identifier)
  • to=<[email protected]> - Recipient address
  • relay=mx.example.com[203.0.113.10]:25 - Destination server
  • delay=0.52 - Total delivery time (seconds)
  • delays=0.01/0.02/0.15/0.34 - before queue/in queue/connection/transmission
  • dsn=2.0.0 - Delivery Status Notification code
  • status=sent - Final delivery status
  • (250 OK) - SMTP response

Dovecot Log Format

Standard Dovecot log entry:

Jan 11 10:30:45 mail dovecot: imap-login: Login: user=<[email protected]>, method=PLAIN, rip=203.0.113.20, lip=203.0.113.10, mpid=12346, secured, session=<abc123>

Components:

  • Jan 11 10:30:45 - Timestamp
  • mail - Hostname
  • dovecot: - Service
  • imap-login: - Component
  • Login: - Action
  • user=<[email protected]> - Username
  • method=PLAIN - Authentication method
  • rip=203.0.113.20 - Remote IP
  • lip=203.0.113.10 - Local IP
  • mpid=12346 - Master process ID
  • secured - Connection encrypted
  • session=<abc123> - Session ID

Basic Log Viewing Commands

View Recent Logs

# Last 50 lines
sudo tail -50 /var/log/mail.log

# Last 100 lines
sudo tail -100 /var/log/mail.log

# Real-time monitoring (follow mode)
sudo tail -f /var/log/mail.log

# Exit tail -f: Press Ctrl+C

View Entire Log File

# View with less (scrollable)
sudo less /var/log/mail.log

# Navigation in less:
# - Space: Next page
# - b: Previous page
# - /pattern: Search forward
# - ?pattern: Search backward
# - q: Quit

# View entire file with cat
sudo cat /var/log/mail.log

View Older Logs

# List available log files
ls -lh /var/log/mail.log*

# View yesterday's log (if rotated)
sudo cat /var/log/mail.log.1

# View compressed older logs
sudo zcat /var/log/mail.log.2.gz
sudo zless /var/log/mail.log.3.gz

Searching Logs with Grep

Basic Grep Usage

# Search for specific email address
sudo grep "[email protected]" /var/log/mail.log

# Case-insensitive search
sudo grep -i "error" /var/log/mail.log

# Show line numbers
sudo grep -n "bounced" /var/log/mail.log

# Show 3 lines of context (before and after)
sudo grep -C 3 "authentication failed" /var/log/mail.log

# Show only 5 lines after match
sudo grep -A 5 "connection timeout" /var/log/mail.log

# Count matches
sudo grep -c "status=sent" /var/log/mail.log

Advanced Grep Patterns

# Multiple patterns (OR)
sudo grep -E "bounced|deferred" /var/log/mail.log

# Pattern from file
echo "[email protected]" > /tmp/patterns.txt
echo "[email protected]" >> /tmp/patterns.txt
sudo grep -f /tmp/patterns.txt /var/log/mail.log

# Invert match (show lines NOT matching)
sudo grep -v "status=sent" /var/log/mail.log

# Recursive search in directory
sudo grep -r "specific-queue-id" /var/log/

# Search compressed logs
sudo zgrep "pattern" /var/log/mail.log.*.gz

Regular Expression Patterns

# Match email addresses
sudo grep -E "[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,}" /var/log/mail.log

# Match IPv4 addresses
sudo grep -E "[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}" /var/log/mail.log

# Match queue IDs (Postfix format)
sudo grep -E "[A-F0-9]{10}" /var/log/mail.log

# Match specific time range
sudo grep "Jan 11 10:[0-5][0-9]:" /var/log/mail.log

Common Log Analysis Tasks

Task 1: Track Specific Email

Track an email by recipient:

# Find all entries for recipient
sudo grep "to=<[email protected]>" /var/log/mail.log

# Get queue ID for email
QUEUE_ID=$(sudo grep "to=<[email protected]>" /var/log/mail.log | tail -1 | grep -oE "[A-F0-9]{10}")

# Track entire message flow by queue ID
sudo grep "$QUEUE_ID" /var/log/mail.log

Task 2: Count Daily Email Volume

# Count emails sent today
sudo grep "status=sent" /var/log/mail.log | grep "$(date '+%b %d')" | wc -l

# Count by hour today
for hour in {00..23}; do
    count=$(sudo grep "status=sent" /var/log/mail.log | grep "$(date '+%b %d') $hour:" | wc -l)
    echo "Hour $hour: $count emails"
done

# Total sent this month
sudo grep "status=sent" /var/log/mail.log* | grep "$(date '+%b')" | wc -l

Task 3: Find Bounced Emails

# All bounced emails
sudo grep "status=bounced" /var/log/mail.log

# Bounced emails today
sudo grep "status=bounced" /var/log/mail.log | grep "$(date '+%b %d')"

# Bounced emails with reasons
sudo grep "status=bounced" /var/log/mail.log | grep -A 1 "status=bounced"

# Count bounces by reason
sudo grep "status=bounced" /var/log/mail.log | awk -F'[()]' '{print $2}' | sort | uniq -c | sort -rn

Task 4: Find Authentication Failures

# All authentication failures
sudo grep "authentication failed" /var/log/mail.log

# Failed auth with usernames
sudo grep "authentication failed" /var/log/mail.log | grep -oE "sasl_username=[^ ]*"

# Failed auth by IP
sudo grep "authentication failed" /var/log/mail.log | grep -oE "rip=[0-9.]+" | sort | uniq -c | sort -rn

# Failed IMAP/POP3 logins (Dovecot)
sudo grep "auth failed" /var/log/dovecot.log

Task 5: Find Rejected Connections

# All rejected connections
sudo grep "reject:" /var/log/mail.log

# Rejections today
sudo grep "reject:" /var/log/mail.log | grep "$(date '+%b %d')"

# Count rejections by reason
sudo grep "reject:" /var/log/mail.log | awk -F'reject: ' '{print $2}' | cut -d: -f1 | sort | uniq -c | sort -rn

# Rejected by IP
sudo grep "reject:" /var/log/mail.log | grep -oE "\[[0-9.]+\]" | sort | uniq -c | sort -rn

Task 6: Find Deferred Emails

# All deferred emails
sudo grep "status=deferred" /var/log/mail.log

# Deferred with reasons
sudo grep "status=deferred" /var/log/mail.log | tail -20

# Count deferrals by reason
sudo grep "status=deferred" /var/log/mail.log | awk -F'[()]' '{print $2}' | sort | uniq -c | sort -rn

# Emails deferred to specific domain
sudo grep "status=deferred" /var/log/mail.log | grep "relay=mx.example.com"

Task 7: Find Spam Detection

# SpamAssassin detections
sudo grep "X-Spam-Status: Yes" /var/log/mail.log

# High spam scores
sudo grep "X-Spam-Status: Yes, score=" /var/log/mail.log | grep -oE "score=[0-9.]+" | sort -t= -k2 -rn | head -20

# Spam from specific sender
sudo grep "X-Spam-Status: Yes" /var/log/mail.log | grep "from=<[email protected]>"

Advanced Log Analysis with AWK

AWK is powerful for structured log analysis.

Extract Specific Fields

# Extract queue IDs
sudo grep "status=sent" /var/log/mail.log | awk '{print $6}' | cut -d: -f1

# Extract recipient addresses
sudo grep "to=<" /var/log/mail.log | grep -oE "to=<[^>]+>" | cut -d= -f2 | tr -d '<>'

# Extract sender addresses
sudo grep "from=<" /var/log/mail.log | grep -oE "from=<[^>]+>" | cut -d= -f2 | tr -d '<>'

Calculate Statistics

# Average delay time
sudo grep "status=sent" /var/log/mail.log | grep -oE "delay=[0-9.]+" | cut -d= -f2 | awk '{sum+=$1; count++} END {print "Average delay:", sum/count, "seconds"}'

# Maximum delay
sudo grep "status=sent" /var/log/mail.log | grep -oE "delay=[0-9.]+" | cut -d= -f2 | sort -rn | head -1

# Message size statistics
sudo grep "size=" /var/log/mail.log | grep -oE "size=[0-9]+" | cut -d= -f2 | awk '{sum+=$1; count++} END {print "Average size:", sum/count/1024, "KB"; print "Total:", sum/1024/1024, "MB"}'

Top Senders/Recipients

# Top 10 senders
sudo grep "from=<" /var/log/mail.log | grep -oE "from=<[^>]+>" | sort | uniq -c | sort -rn | head -10

# Top 10 recipients
sudo grep "to=<" /var/log/mail.log | grep -oE "to=<[^>]+>" | sort | uniq -c | sort -rn | head -10

# Top sending domains
sudo grep "from=<" /var/log/mail.log | grep -oE "@[a-zA-Z0-9.-]+" | sort | uniq -c | sort -rn | head -10

# Top receiving domains
sudo grep "to=<" /var/log/mail.log | grep -oE "@[a-zA-Z0-9.-]+" | sort | uniq -c | sort -rn | head -10

Connection Analysis

# Top connecting IPs
sudo grep "connect from" /var/log/mail.log | grep -oE "\[[0-9.]+\]" | sort | uniq -c | sort -rn | head -10

# Connections per hour today
for hour in {00..23}; do
    count=$(sudo grep "connect from" /var/log/mail.log | grep "$(date '+%b %d') $hour:" | wc -l)
    echo "$hour:00 - $count connections"
done

Creating Automated Analysis Scripts

Daily Email Report Script

sudo nano /usr/local/bin/daily-mail-report.sh

Add:

#!/bin/bash

DATE=$(date '+%b %d')
LOG="/var/log/mail.log"

echo "=== Daily Mail Report for $(date '+%Y-%m-%d') ==="
echo ""

echo "Email Volume:"
echo "  Sent: $(grep "status=sent" $LOG | grep "$DATE" | wc -l)"
echo "  Bounced: $(grep "status=bounced" $LOG | grep "$DATE" | wc -l)"
echo "  Deferred: $(grep "status=deferred" $LOG | grep "$DATE" | wc -l)"
echo "  Rejected: $(grep "reject:" $LOG | grep "$DATE" | wc -l)"
echo ""

echo "Authentication:"
echo "  Failed attempts: $(grep "authentication failed" $LOG | grep "$DATE" | wc -l)"
echo "  Successful logins: $(grep "Login:" /var/log/dovecot.log | grep "$DATE" | wc -l)"
echo ""

echo "Top 5 Senders:"
grep "from=<" $LOG | grep "$DATE" | grep -oE "from=<[^>]+>" | sort | uniq -c | sort -rn | head -5
echo ""

echo "Top 5 Recipients:"
grep "to=<" $LOG | grep "$DATE" | grep -oE "to=<[^>]+>" | sort | uniq -c | sort -rn | head -5
echo ""

echo "Recent Errors:"
grep -i "error" $LOG | grep "$DATE" | tail -10

Make executable:

sudo chmod +x /usr/local/bin/daily-mail-report.sh

Schedule daily:

sudo crontab -e

Add:

0 8 * * * /usr/local/bin/daily-mail-report.sh | mail -s "Daily Mail Report" [email protected]

Real-Time Alert Script

sudo nano /usr/local/bin/mail-alert.sh

Add:

#!/bin/bash

LOG="/var/log/mail.log"
ALERT_EMAIL="[email protected]"
THRESHOLD=10

# Check authentication failures in last 5 minutes
AUTH_FAILURES=$(grep "authentication failed" $LOG | tail -100 | wc -l)

if [ $AUTH_FAILURES -gt $THRESHOLD ]; then
    echo "WARNING: $AUTH_FAILURES authentication failures detected" | \
        mail -s "ALERT: High Authentication Failures" $ALERT_EMAIL
fi

# Check queue size
QUEUE_SIZE=$(mailq | tail -1 | awk '{print $5}')

if [ "$QUEUE_SIZE" != "empty" ] && [ $QUEUE_SIZE -gt 100 ]; then
    echo "WARNING: Mail queue has $QUEUE_SIZE messages" | \
        mail -s "ALERT: Large Mail Queue" $ALERT_EMAIL
fi

Make executable and schedule:

sudo chmod +x /usr/local/bin/mail-alert.sh
sudo crontab -e

Add:

*/5 * * * * /usr/local/bin/mail-alert.sh

Log Analysis Dashboard Script

sudo nano /usr/local/bin/mail-dashboard.sh

Add:

#!/bin/bash

clear
echo "╔════════════════════════════════════════════════╗"
echo "║       MAIL SERVER DASHBOARD                    ║"
echo "╚════════════════════════════════════════════════╝"
echo ""

LOG="/var/log/mail.log"
TODAY=$(date '+%b %d')

# Real-time stats
echo "Real-time Statistics (today):"
echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━"

SENT=$(grep "status=sent" $LOG | grep "$TODAY" | wc -l)
BOUNCED=$(grep "status=bounced" $LOG | grep "$TODAY" | wc -l)
DEFERRED=$(grep "status=deferred" $LOG | grep "$TODAY" | wc -l)
REJECTED=$(grep "reject:" $LOG | grep "$TODAY" | wc -l)

printf "  %-20s %10s\n" "Sent:" "$SENT"
printf "  %-20s %10s\n" "Bounced:" "$BOUNCED"
printf "  %-20s %10s\n" "Deferred:" "$DEFERRED"
printf "  %-20s %10s\n" "Rejected:" "$REJECTED"
echo ""

# Queue status
echo "Mail Queue:"
echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━"
mailq | tail -1
echo ""

# Recent activity
echo "Last 5 Sent Emails:"
echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━"
grep "status=sent" $LOG | tail -5 | awk '{print $1, $2, $3, $7}' | sed 's/to=<//g' | sed 's/>.*//g'
echo ""

# Errors
echo "Recent Errors:"
echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━"
grep -i "error\|fatal" $LOG | tail -3
echo ""

echo "Last updated: $(date)"

Make executable and run:

sudo chmod +x /usr/local/bin/mail-dashboard.sh
sudo /usr/local/bin/mail-dashboard.sh

Log Rotation and Management

Configure Logrotate

sudo nano /etc/logrotate.d/rsyslog

Configure:

/var/log/mail.log
{
    rotate 7
    daily
    missingok
    notifempty
    compress
    delaycompress
    sharedscripts
    postrotate
        /usr/lib/rsyslog/rsyslog-rotate
    endscript
}

Options explained:

  • rotate 7: Keep 7 days of logs
  • daily: Rotate daily
  • compress: Compress old logs
  • delaycompress: Don't compress yesterday's log (still being written)

Manual Log Rotation

# Manually trigger rotation
sudo logrotate -f /etc/logrotate.d/rsyslog

# Check rotation status
sudo cat /var/lib/logrotate/status

Archive Old Logs

# Create archive directory
sudo mkdir -p /var/log/mail-archive

# Archive logs older than 30 days
sudo find /var/log -name "mail.log.*.gz" -mtime +30 -exec mv {} /var/log/mail-archive/ \;

# Create monthly archive
sudo tar czf /var/log/mail-archive/mail-$(date +%Y-%m).tar.gz /var/log/mail.log.*.gz

Clear Logs (Caution!)

# Clear log file (keeps file, removes content)
sudo truncate -s 0 /var/log/mail.log

# Or
sudo > /var/log/mail.log

# NEVER delete log files while service is running!
# Always truncate or rotate instead

Performance Optimization

Limit Log Verbosity

If logs are too large:

# Reduce Postfix verbosity
sudo postconf -e "smtpd_tls_loglevel = 0"
sudo postconf -e "smtp_tls_loglevel = 0"
sudo systemctl reload postfix

# Reduce Dovecot verbosity
sudo nano /etc/dovecot/conf.d/10-logging.conf

Set:

auth_verbose = no
auth_debug = no
mail_debug = no

Reload:

sudo systemctl reload dovecot

Exclude Noisy Entries

Use rsyslog to filter:

sudo nano /etc/rsyslog.d/50-mail-filter.conf

Add:

# Exclude noisy entries
:msg, contains, "statistics:" stop
:msg, contains, "connect from localhost" stop

Restart rsyslog:

sudo systemctl restart rsyslog

Security and Compliance

Protect Log Files

# Set restrictive permissions
sudo chmod 640 /var/log/mail.log
sudo chown syslog:adm /var/log/mail.log

# Protect log directory
sudo chmod 755 /var/log

Log Retention for Compliance

Different regulations require different retention:

  • GDPR: Minimal necessary period
  • HIPAA: 6 years
  • SOX: 7 years
  • PCI-DSS: 1 year

Configure accordingly:

sudo nano /etc/logrotate.d/rsyslog

For 1 year retention:

/var/log/mail.log
{
    rotate 365
    daily
    compress
    delaycompress
}

Remove Sensitive Data

Before sharing logs:

# Redact email addresses
sed 's/[a-zA-Z0-9._%+-]*@[a-zA-Z0-9.-]*/***@***/g' /var/log/mail.log

# Redact IP addresses
sed 's/[0-9]\{1,3\}\.[0-9]\{1,3\}\.[0-9]\{1,3\}\.[0-9]\{1,3\}/XXX.XXX.XXX.XXX/g' /var/log/mail.log

Troubleshooting with Logs

Common Log Patterns

Success:

status=sent (250 2.0.0 OK)

User unknown:

status=bounced (User unknown)

Connection timeout:

status=deferred (Connection timed out)

Authentication failure:

authentication failed
auth failed

TLS error:

TLS library problem
SSL_connect error

Disk full:

No space left on device
Mail queue file write error

Error Code Reference

2xx codes: Success

  • 250: OK
  • 251: User not local; will forward

4xx codes: Temporary failure (try again)

  • 421: Service not available
  • 450: Mailbox unavailable
  • 451: Local error
  • 452: Insufficient storage

5xx codes: Permanent failure (don't retry)

  • 550: User unknown
  • 551: User not local
  • 552: Exceeded storage
  • 553: Mailbox name invalid
  • 554: Transaction failed

Conclusion

You now have comprehensive knowledge of email log analysis and diagnostics. These skills enable you to quickly identify issues, track message flow, monitor server health, and maintain optimal email operations.

Key Takeaways

  1. Understand log formats: Know how to read Postfix and Dovecot logs
  2. Use appropriate tools: grep, awk, sed for efficient analysis
  3. Create automated reports: Regular monitoring catches issues early
  4. Maintain logs properly: Rotation, archival, and retention
  5. Secure sensitive data: Protect logs and redact when sharing

Best Practices

  • Monitor daily: Review logs regularly for anomalies
  • Automate alerts: Catch critical issues immediately
  • Document patterns: Build knowledge base of common issues
  • Retain appropriately: Balance compliance with storage costs
  • Secure access: Logs contain sensitive information

Recommended Schedule

Daily:

  • Quick review of errors and warnings
  • Check for authentication failures
  • Verify normal email volume

Weekly:

  • Detailed analysis of trends
  • Review top senders/recipients
  • Check deferred/bounced statistics

Monthly:

  • Archive old logs
  • Update analysis scripts
  • Review retention policies

With these log analysis skills and tools, you can maintain complete visibility into your email infrastructure, quickly diagnose problems, and ensure reliable, secure email operations.