Fail2Ban Advanced Configuration and Custom Jails

Fail2Ban is a powerful intrusion prevention framework that monitors log files, identifies failed authentication attempts, and dynamically blocks offending IP addresses through firewall rules. Unlike static firewall rules, Fail2Ban responds in real-time to ongoing attacks by reading logs, detecting patterns, and automatically updating iptables/nftables rules. This guide covers advanced configuration, creating custom filters and actions, implementing rate limiting, managing multi-service setups, and maintaining persistent bans.

Table of Contents

System Requirements

Fail2Ban requires minimal system resources and Python interpreter:

  • Python 3.6 or later
  • 100 MB disk space
  • 50 MB RAM
  • iptables or nftables for firewall rule management
  • Log files accessible by Fail2Ban user
  • Root access for firewall rule modification

Verify system capabilities:

python3 --version
iptables --version
nftables --version

Installation

Install Fail2Ban from official repositories or source code.

For Ubuntu/Debian:

sudo apt-get update
sudo apt-get install -y fail2ban fail2ban-doc

For CentOS/RHEL:

sudo yum install -y fail2ban fail2ban-systemd

For Fedora:

sudo dnf install -y fail2ban

Verify installation:

fail2ban-client --version

Enable and start Fail2Ban:

sudo systemctl enable fail2ban
sudo systemctl start fail2ban

Check status:

sudo systemctl status fail2ban
sudo fail2ban-client status

Basic Configuration

Configure Fail2Ban for your environment.

The main configuration file is /etc/fail2ban/fail2ban.conf. Create a local override:

sudo nano /etc/fail2ban/fail2ban.local

Essential settings:

[DEFAULT]
# Logging
loglevel = INFO
logtarget = /var/log/fail2ban/fail2ban.log
socket = /var/run/fail2ban/fail2ban.sock
pidfile = /var/run/fail2ban/fail2ban.pid

# Backend for monitoring log files
backend = auto

# Port range for Fail2Ban communication
port = 5000

Configure jail defaults in /etc/fail2ban/jail.local:

sudo nano /etc/fail2ban/jail.local

Default settings for all jails:

[DEFAULT]
# Time between checks in seconds
bantime = 3600

# Failed attempts before banning
maxretry = 5

# Time window for counting failures
findtime = 600

# Deban action
debanaction = %(banaction)s

# Enable by default
enabled = true

# Action: block through iptables
banaction = iptables-multiport

# Whitelist IPs never to ban
ignoreip = 127.0.0.1/8 ::1 192.168.1.0/24 10.0.0.0/8

View active jails:

sudo fail2ban-client status

Check specific jail status:

sudo fail2ban-client status sshd

Custom Filters

Create custom filters for specific applications and log formats.

Filter files are located in /etc/fail2ban/filter.d/. Create a custom filter:

sudo nano /etc/fail2ban/filter.d/custom-app.conf

Basic filter structure:

[Definition]
# Filter name and description
failregex = ^<HOST> .* authentication failed.*$
            ^<HOST> .* invalid user.*$
            ^<HOST> .* connection closed by authenticating user <USER>.*$

# Regular expressions for account lockouts
            ^<HOST> .* account locked.*$

# Regex for connection attempts
ignoreregex = ^<HOST> .* successful login.*$

Example custom filter for web application:

[Definition]
failregex = ^<HOST> - - \[.*\] ".*HTTP/1\.[01]" 401 .*$
            ^<HOST> - .* \[.*\] ".*" 403 .*$
            ^<HOST> - .* \[.*\] ".*login.*" 404 .*$

ignoreregex = ^127\.0\.0\.1 - .* \[.*\] ".*" 401 .*$

Example filter for database failures:

[Definition]
failregex = ^.*<HOST>.*\[ERROR\].*Access denied for user.*$
            ^.*<HOST>.*\[ERROR\].*Password mismatch.*$
            ^.*<HOST>.*\[WARN\].*Failed login attempt.*$

ignoreregex = ^.*127\.0\.0\.1.*success.*$

Test filter against log files:

sudo fail2ban-regex /var/log/auth.log /etc/fail2ban/filter.d/sshd.conf

Output shows:

Lines: 1234
Regex matches: 45
% matched: 3.6%

Regular Expression Patterns

Master regex patterns for accurate log matching.

Common regex elements in Fail2Ban filters:

<HOST>       - Matches the IP address
<UNIT>       - Matches unit/service name
<USER>       - Matches username
[\s\S]*      - Matches any character including newlines
.*           - Matches any character except newline
\d+          - Matches one or more digits
\S+          - Matches non-whitespace
^            - Matches start of line
$            - Matches end of line
[A-Za-z0-9]  - Character class

Example patterns for SSH:

# Invalid user attempt
^.*Invalid user .* from <HOST> port \d+ ssh2$

# Failed password
^.*Failed password for .* from <HOST> port \d+ ssh2$

# Failed publickey
^.*Failed publickey for .* from <HOST> port \d+ ssh2$

# Authentication refused
^.*Authentication refused: bad packet type from <HOST>$

Example patterns for HTTP:

# 401 Unauthorized
^<HOST> -.*GET.*HTTP/1\.[01]" 401 .*$

# 403 Forbidden (repeated attempts)
^<HOST> -.*POST.*HTTP/1\.[01]" 403 .*$

# 404 Not Found (repeated attempts)
^<HOST> -.*\.(php|jsp|aspx).*HTTP/1\.[01]" 404 .*$

# SQL injection attempts
^<HOST> -.*union.*select.*HTTP/1\.[01]".*$

Test regex patterns:

sudo fail2ban-regex --verbose /path/to/logfile /etc/fail2ban/filter.d/custom.conf

Custom Actions

Create custom actions for automated responses to attacks.

Action files are in /etc/fail2ban/action.d/. Create custom action:

sudo nano /etc/fail2ban/action.d/custom-action.conf

Basic action structure:

[Definition]
# Description of action
name = Custom Ban Action
port = all
protocol = all

# Actions executed on ban
actionstart = <action_on_start>
actionstop = <action_on_stop>
actionban = <action_on_ban>
actionunban = <action_on_unban>

# Action variables
_cmd = /usr/bin/
_args = -

Example action that creates firewall rules:

[Definition]
name = iptables-custom
port = all
protocol = all

actionstart = /usr/bin/iptables -t filter -N fail2ban-custom
              /usr/bin/iptables -t filter -I INPUT -p all -j fail2ban-custom

actionstop = /usr/bin/iptables -t filter -D INPUT -p all -j fail2ban-custom
             /usr/bin/iptables -t filter -X fail2ban-custom

actionban = /usr/bin/iptables -t filter -I fail2ban-custom -s <ip> -j DROP

actionunban = /usr/bin/iptables -t filter -D fail2ban-custom -s <ip> -j DROP

Example action that sends email notifications:

[Definition]
name = send-email-ban
actionban = /usr/local/bin/fail2ban-notify.sh <ip> <port> BAN
actionunban = /usr/local/bin/fail2ban-notify.sh <ip> <port> UNBAN

Create notification script:

sudo nano /usr/local/bin/fail2ban-notify.sh

Content:

#!/bin/bash
IP=$1
PORT=$2
ACTION=$3
TIMESTAMP=$(date)

echo "Fail2Ban Alert: IP $IP action $ACTION at $TIMESTAMP" | \
  mail -s "Fail2Ban: $ACTION for $IP" [email protected]

Make script executable:

sudo chmod +x /usr/local/bin/fail2ban-notify.sh

Example action for logging to syslog:

[Definition]
name = syslog-notification
actionban = /bin/echo "Fail2Ban: Banned <ip> for <name>" | nc -w 0 127.0.0.1 514

Jail Configuration

Configure jails to protect specific services.

SSH jail configuration:

sudo nano /etc/fail2ban/jail.d/sshd.local

Content:

[sshd]
enabled = true
port = ssh,2222
filter = sshd
logpath = /var/log/auth.log
maxretry = 3
findtime = 600
bantime = 3600
action = iptables-multiport[name=SSH, port="ssh,2222"]
         sendmail-whois[name=SSH, [email protected]]

Nginx jail for HTTP:

sudo nano /etc/fail2ban/jail.d/nginx-http-auth.local

Content:

[nginx-http-auth]
enabled = true
port = http,https
filter = nginx-http-auth
logpath = /var/log/nginx/error.log
maxretry = 5
findtime = 600
bantime = 1800
action = iptables-multiport[name=Nginx-Auth, port="http,https"]
         sendmail-whois[name=Nginx-Auth]

Apache jail:

sudo nano /etc/fail2ban/jail.d/apache-auth.local

Content:

[apache-auth]
enabled = true
port = http,https
filter = apache-auth
logpath = /var/log/apache2/error.log
maxretry = 5
findtime = 600
bantime = 1800
action = iptables-multiport[name=Apache-Auth]

FTP jail:

sudo nano /etc/fail2ban/jail.d/proftpd.local

Content:

[proftpd]
enabled = true
port = ftp
filter = proftpd
logpath = /var/log/proftpd/proftpd.log
maxretry = 3
findtime = 600
bantime = 3600
action = iptables-multiport[name=ProFTPD, port="ftp"]

Multi-Service Jails

Configure multiple jails for comprehensive service protection.

Create combined jail configuration:

sudo nano /etc/fail2ban/jail.local

Content protecting multiple services:

[DEFAULT]
bantime = 3600
maxretry = 5
findtime = 600
debanaction = %(banaction)s
ignoreip = 127.0.0.1/8 192.168.1.0/24

[sshd]
enabled = true
port = ssh
filter = sshd
logpath = /var/log/auth.log
action = iptables-multiport[name=SSH, port="ssh"]

[nginx-http-auth]
enabled = true
port = http,https
filter = nginx-http-auth
logpath = /var/log/nginx/error.log
action = iptables-multiport[name=Nginx-Auth, port="http,https"]

[apache-auth]
enabled = true
port = http,https
filter = apache-auth
logpath = /var/log/apache2/error.log
action = iptables-multiport[name=Apache-Auth, port="http,https"]

[mysql-auth]
enabled = true
filter = mysql-auth
logpath = /var/log/mysql/error.log
maxretry = 3
action = iptables[name=MySQL]

[dovecot]
enabled = true
port = pop3,pop3s,imap,imaps
filter = dovecot
logpath = /var/log/mail.log
action = iptables-multiport[name=Dovecot]

Reload Fail2Ban to apply changes:

sudo fail2ban-client reload

Check all active jails:

sudo fail2ban-client status

Rate Limiting

Implement rate limiting without permanent bans.

Create rate-limit filter:

sudo nano /etc/fail2ban/filter.d/rate-limit.conf

Content:

[Definition]
failregex = ^<HOST> .* \[.*\] "GET .* HTTP/1\.[01]" \d+ .*$
ignoreregex =

Configure rate-limit jail:

sudo nano /etc/fail2ban/jail.d/rate-limit.local

Content:

[nginx-rate-limit]
enabled = true
port = http,https
filter = rate-limit
logpath = /var/log/nginx/access.log
# More failures allowed but shorter bantime
maxretry = 30
findtime = 60
bantime = 300
action = iptables-multiport[name=Rate-Limit, port="http,https"]

Reload to activate:

sudo fail2ban-client reload

Persistent Bans

Maintain bans across system restarts.

Create persistent ban database:

sudo nano /etc/fail2ban/jail.local

Add to DEFAULT section:

# Persistent bans
banlistdir = /var/lib/fail2ban

Create directory:

sudo mkdir -p /var/lib/fail2ban
sudo chown fail2ban:fail2ban /var/lib/fail2ban

Save currently banned IPs:

sudo fail2ban-client get sshd banlist > /var/lib/fail2ban/banned-ips.txt

Restore bans on startup. Create systemd service:

sudo nano /etc/systemd/system/fail2ban-restore-bans.service

Content:

[Unit]
Description=Restore Fail2Ban persistent bans
After=fail2ban.service
Wants=fail2ban.service

[Service]
Type=oneshot
ExecStart=/usr/local/bin/fail2ban-restore.sh
RemainAfterExit=yes

[Install]
WantedBy=multi-user.target

Create restore script:

sudo nano /usr/local/bin/fail2ban-restore.sh

Content:

#!/bin/bash
FAIL2BAN_CLIENT=/usr/bin/fail2ban-client
BANNED_FILE=/var/lib/fail2ban/banned-ips.txt

if [ -f "$BANNED_FILE" ]; then
    while read IP; do
        # Re-ban IPs from persistent list
        $FAIL2BAN_CLIENT set sshd banip "$IP"
    done < "$BANNED_FILE"
fi

Make executable:

sudo chmod +x /usr/local/bin/fail2ban-restore.sh

Enable service:

sudo systemctl enable fail2ban-restore-bans.service
sudo systemctl daemon-reload

Monitoring and Maintenance

Monitor Fail2Ban activity and maintain the system.

View currently banned IPs:

sudo fail2ban-client get sshd banlist
sudo fail2ban-client get nginx-http-auth banlist

Check jail status:

sudo fail2ban-client status sshd

View unban stats:

sudo fail2ban-client get sshd unbanlist

Manually ban/unban IP:

sudo fail2ban-client set sshd banip 192.0.2.5
sudo fail2ban-client set sshd unbanip 192.0.2.5

View Fail2Ban logs:

sudo tail -f /var/log/fail2ban/fail2ban.log

Monitor real-time activity:

sudo tail -f /var/log/fail2ban/fail2ban.log | grep "Ban\|Unban"

Generate statistics:

grep "Ban" /var/log/fail2ban/fail2ban.log | grep -o "[0-9]\{1,3\}\.[0-9]\{1,3\}\.[0-9]\{1,3\}\.[0-9]\{1,3\}" | sort | uniq -c | sort -rn

Rotate logs:

sudo logrotate -f /etc/logrotate.d/fail2ban

Check iptables rules created by Fail2Ban:

sudo iptables -L fail2ban-sshd -n
sudo iptables -L fail2ban-nginx-http-auth -n

Conclusion

Fail2Ban provides dynamic intrusion prevention through automated log analysis and firewall rule management. By following this guide, you've installed and configured Fail2Ban, created custom filters for application-specific threats, developed regular expression patterns for accurate detection, implemented custom actions for automated responses, configured multi-service jails for comprehensive protection, implemented rate limiting for traffic control, and maintained persistent bans across system restarts. Regular monitoring and filter tuning ensure Fail2Ban effectively protects against brute force attacks while minimizing false positives. Whether protecting single servers or large deployments, Fail2Ban scales with flexible configuration and powerful automation capabilities.