Auditing with auditd on Linux

Introduction

System auditing provides essential visibility into user actions, system events, and security-relevant activities on Linux systems. The Linux Audit framework (auditd) is a powerful, kernel-level auditing system that records detailed information about system calls, file access, authentication attempts, and administrative actions, creating a comprehensive audit trail for security monitoring, forensics, and compliance.

Proper audit logging is not just a compliance requirement—it's a critical security control that enables detection of unauthorized access, tracks privilege escalation attempts, provides evidence for incident response, and ensures accountability for system changes. Whether you're managing a single server or an enterprise infrastructure, understanding and implementing auditd is essential for maintaining security and meeting regulatory requirements.

This comprehensive guide covers everything from basic auditd configuration to advanced audit rule creation, log analysis, and integration with security monitoring systems. You'll learn how to implement effective auditing that balances security visibility with system performance, enabling you to detect threats, investigate incidents, and demonstrate compliance with various security frameworks.

Understanding auditd and Security Context

What Is auditd?

The Linux Audit system (auditd) is a comprehensive auditing framework that operates at the kernel level to monitor and record system events. It consists of:

• Kernel audit subsystem: Captures events at the kernel level
• auditd daemon: Manages audit records and writes to logs
• auditctl: Command-line utility for managing audit rules
• ausearch/aureport: Tools for searching and reporting on audit logs
• audispd: Audit event dispatcher for real-time processing

How auditd Works

auditd operates through several key mechanisms:

1. Kernel-Level Monitoring

Audit hooks in the kernel capture events before they complete, ensuring comprehensive and tamper-resistant logging.

2. Rule-Based Filtering

Three types of audit rules:

• Control rules: Configure audit system behavior
• File system rules: Monitor file access (watches)
• System call rules: Monitor specific system calls

3. Event Recording

Each audit event includes:

• Timestamp with millisecond precision
• Event type and result (success/failure)
• Subject (user/process) performing action
• Object (file/resource) being accessed
• Additional context (arguments, environment)

4. Log Management

Events are written to /var/log/audit/audit.log with rotation and compression managed by auditd configuration.

Why Auditing Matters

auditd provides critical security and compliance capabilities:

• Intrusion detection: Detect unauthorized access and suspicious activities
• Forensic analysis: Investigate security incidents with detailed event logs
• Compliance: Meet requirements for PCI DSS, HIPAA, SOX, NIST, ISO 27001
• Accountability: Track who did what, when, and where
• Change management: Monitor configuration and administrative changes
• Privilege monitoring: Detect privilege escalation attempts
• Data access tracking: Monitor access to sensitive files

Common Audit Requirements

Regulatory frameworks often require auditing of:

• Authentication events: Logins, logouts, failed attempts
• Privilege escalation: sudo usage, su commands
• File access: Reading/modifying sensitive files
• Administrative actions: System configuration changes
• User management: Creating/deleting users and groups
• Network activities: Network connections and configurations
• Time changes: System clock modifications
• Audit system: Changes to audit configuration itself

Performance Considerations

Auditing has performance implications:

• CPU overhead: 5-15% depending on rule complexity
• Disk I/O: Continuous writing to audit logs
• Storage: Audit logs can grow quickly (MB to GB per day)
• Network: If forwarding logs to central server

Best practices minimize impact:

• Audit only what's necessary
• Use efficient rule design
• Implement log rotation
• Consider centralized logging

Prerequisites

Before configuring auditd, ensure you have:

System Requirements

• Operating System: Any modern Linux distribution
• Kernel: Linux kernel 2.6.32 or later with audit support
• Disk Space: Adequate space for audit logs (varies by activity)
• Root Access: Administrative privileges required
• Memory: Sufficient RAM for audit daemon (typically <100MB)

Required Knowledge

• Linux system administration
• Command-line proficiency
• Understanding of system calls and file operations
• Log analysis basics
• Security concepts and terminology

Software Requirements

Ubuntu/Debian:

sudo apt-get update
sudo apt-get install auditd audispd-plugins

CentOS/RHEL:

sudo yum install audit audit-libs

Verify installation:

sudo systemctl status auditd
auditctl --version

Enable and Start auditd

Start auditd:

sudo systemctl start auditd

Enable at boot:

sudo systemctl enable auditd

Check status:

sudo systemctl status auditd

IMPORTANT: Use service auditd commands instead of systemctl on some systems, as auditd may require special handling.

Step-by-Step auditd Configuration

Step 1: Configure auditd Daemon

Edit main configuration:

sudo nano /etc/audit/auditd.conf

Key configuration options:

# Directory for audit logs
log_file = /var/log/audit/audit.log

# Maximum log file size (MB)
max_log_file = 100

# Action when max size reached (ROTATE, SUSPEND, KEEP_LOGS, etc.)
max_log_file_action = ROTATE

# Number of rotated log files to keep
num_logs = 10

# Disk space action when running low
space_left_action = EMAIL
space_left = 500

# Action when disk is full
admin_space_left_action = HALT
admin_space_left = 50

# Disk full action
disk_full_action = SUSPEND

# Email for notifications
action_mail_acct = root

# Format (RAW or ENRICHED)
log_format = RAW

# Flush frequency (INCREMENTAL_ASYNC, INCREMENTAL, DATA, SYNC, NONE)
flush = INCREMENTAL_ASYNC

# Priority boost for auditd
priority_boost = 4

Save and restart auditd:

sudo service auditd restart

Step 2: Understand Audit Rules Structure

Audit rules follow this format:

auditctl -w /path/to/file -p permissions -k key_name

Components:

• -w: Watch (file system rule)
• /path/to/file: File or directory to monitor
• -p: Permissions to monitor (r=read, w=write, x=execute, a=attribute change)
• -k: Key name for filtering logs

For system call rules:

auditctl -a action,filter -S system_call -F field=value -k key_name

Components:

• -a: Add rule
• action,filter: always,exit (most common)
• -S: System call to monitor
• -F: Additional filters
• -k: Key name

Step 3: Create Basic File System Watches

Monitor sensitive files:

# Monitor /etc/passwd for any access
sudo auditctl -w /etc/passwd -p wa -k passwd_changes

# Monitor /etc/group
sudo auditctl -w /etc/group -p wa -k group_changes

# Monitor /etc/shadow
sudo auditctl -w /etc/shadow -p wa -k shadow_changes

# Monitor /etc/sudoers
sudo auditctl -w /etc/sudoers -p wa -k sudoers_changes

# Monitor SSH configuration
sudo auditctl -w /etc/ssh/sshd_config -p wa -k sshd_config_changes

Monitor important directories:

# Monitor /etc directory
sudo auditctl -w /etc/ -p wa -k etc_changes

# Monitor /boot directory (kernel files)
sudo auditctl -w /boot/ -p wa -k boot_changes

# Monitor /usr/bin and /usr/sbin
sudo auditctl -w /usr/bin/ -p wa -k bin_changes
sudo auditctl -w /usr/sbin/ -p wa -k sbin_changes

Monitor security-relevant files:

# Monitor PAM configuration
sudo auditctl -w /etc/pam.d/ -p wa -k pam_changes

# Monitor audit configuration
sudo auditctl -w /etc/audit/ -p wa -k audit_config_changes
sudo auditctl -w /etc/audit/auditd.conf -p wa -k auditd_config_changes
sudo auditctl -w /etc/audit/audit.rules -p wa -k audit_rules_changes

# Monitor firewall rules
sudo auditctl -w /etc/iptables/ -p wa -k iptables_changes
sudo auditctl -w /etc/firewalld/ -p wa -k firewalld_changes

Step 4: Create System Call Rules

Monitor privilege escalation:

# Monitor sudo usage
sudo auditctl -a always,exit -F arch=b64 -S execve -F path=/usr/bin/sudo -k sudo_usage
sudo auditctl -a always,exit -F arch=b32 -S execve -F path=/usr/bin/sudo -k sudo_usage

# Monitor su usage
sudo auditctl -a always,exit -F arch=b64 -S execve -F path=/bin/su -k su_usage
sudo auditctl -a always,exit -F arch=b32 -S execve -F path=/bin/su -k su_usage

Monitor file deletion:

# Monitor file deletion
sudo auditctl -a always,exit -F arch=b64 -S unlink -S unlinkat -S rename -S renameat -k file_deletion
sudo auditctl -a always,exit -F arch=b32 -S unlink -S unlinkat -S rename -S renameat -k file_deletion

Monitor user/group modifications:

# Monitor user additions/deletions
sudo auditctl -a always,exit -F arch=b64 -S execve -F path=/usr/sbin/useradd -k user_management
sudo auditctl -a always,exit -F arch=b64 -S execve -F path=/usr/sbin/userdel -k user_management
sudo auditctl -a always,exit -F arch=b64 -S execve -F path=/usr/sbin/usermod -k user_management

# Monitor group modifications
sudo auditctl -a always,exit -F arch=b64 -S execve -F path=/usr/sbin/groupadd -k group_management
sudo auditctl -a always,exit -F arch=b64 -S execve -F path=/usr/sbin/groupdel -k group_management

Monitor network activities:

# Monitor socket creation
sudo auditctl -a always,exit -F arch=b64 -S socket -k network_socket
sudo auditctl -a always,exit -F arch=b32 -S socket -k network_socket

# Monitor network configuration changes
sudo auditctl -a always,exit -F arch=b64 -S sethostname -S setdomainname -k network_config

Monitor time changes:

# Monitor system time modifications
sudo auditctl -a always,exit -F arch=b64 -S adjtimex -S settimeofday -S clock_settime -k time_changes
sudo auditctl -a always,exit -F arch=b32 -S adjtimex -S settimeofday -S clock_settime -k time_changes

# Monitor timezone changes
sudo auditctl -w /etc/localtime -p wa -k time_zone_changes

Step 5: Make Rules Persistent

Current rules are temporary (lost on reboot). Make them persistent:

Save current rules:

sudo auditctl -l > /tmp/audit.rules.backup

Edit persistent rules file:

sudo nano /etc/audit/rules.d/audit.rules

Or on some systems:

sudo nano /etc/audit/audit.rules

Add rules (one per line):

# Delete all existing rules
-D

# Set buffer size
-b 8192

# Set failure mode (0=silent, 1=printk, 2=panic)
-f 1

# Monitor authentication files
-w /etc/passwd -p wa -k passwd_changes
-w /etc/group -p wa -k group_changes
-w /etc/shadow -p wa -k shadow_changes
-w /etc/sudoers -p wa -k sudoers_changes

# Monitor system calls
-a always,exit -F arch=b64 -S execve -F path=/usr/bin/sudo -k sudo_usage
-a always,exit -F arch=b32 -S execve -F path=/usr/bin/sudo -k sudo_usage

# Monitor file deletions
-a always,exit -F arch=b64 -S unlink -S unlinkat -k file_deletion
-a always,exit -F arch=b32 -S unlink -S unlinkat -k file_deletion

# Make configuration immutable (reboot required to change)
-e 2

Load rules:

sudo augenrules --load

Or:

sudo service auditd restart

Verify rules are loaded:

sudo auditctl -l

Step 6: Search Audit Logs

Basic search with ausearch:

# Search by key
sudo ausearch -k passwd_changes

# Search by message type
sudo ausearch -m USER_AUTH

# Search for failed events
sudo ausearch --success no

# Search for specific user
sudo ausearch -ua username

# Search by date/time
sudo ausearch -ts today
sudo ausearch -ts 10:00 -te 11:00
sudo ausearch -ts 2024-01-01 -te 2024-01-31

# Search by event ID
sudo ausearch -a 12345

# Interpret results (human readable)
sudo ausearch -k passwd_changes -i

Common message types:

• USER_AUTH: Authentication events
• USER_LOGIN: User login events
• USER_LOGOUT: User logout events
• ADD_USER: User creation
• DEL_USER: User deletion
• USER_CMD: Commands executed by users
• SYSCALL: System call events
• PATH: File path information
• EXECVE: Program execution

Step 7: Generate Audit Reports

Generate summary report:

sudo aureport

Authentication report:

sudo aureport -au

Login report:

sudo aureport -l

Failed events report:

sudo aureport --failed

User command report:

sudo aureport -x

File access report:

sudo aureport -f

Report by specific key:

sudo aureport -k passwd_changes

Time range report:

sudo aureport -ts today
sudo aureport -ts 2024-01-01 -te 2024-01-31

Summary with statistics:

sudo aureport --summary

Step 8: Monitor Specific Users

Track all actions by user:

# Add rule to monitor specific user
sudo auditctl -a always,exit -F auid=1000 -k user_1000_actions

# Replace 1000 with actual user ID
id username

Monitor specific user group:

sudo auditctl -a always,exit -F gid=wheel -k admin_group_actions

Search user's audit events:

sudo ausearch -ua username -i

Step 9: Set Up Centralized Logging

Configure audisp (audit dispatcher):

sudo nano /etc/audit/plugins.d/syslog.conf

Enable syslog plugin:

active = yes
direction = out
path = builtin_syslog
type = builtin
args = LOG_INFO
format = string

Configure remote logging with audisp:

sudo nano /etc/audit/plugins.d/au-remote.conf

active = yes
direction = out
path = /sbin/audisp-remote
type = always
format = string

Configure remote server:

sudo nano /etc/audit/audisp-remote.conf

remote_server = log-server.example.com
port = 60

Restart auditd:

sudo service auditd restart

Advanced auditd Hardening Tips

1. Implement Immutable Configuration

Make audit configuration immutable (requires reboot to change):

Add to /etc/audit/rules.d/audit.rules:

# Must be last line
-e 2

This prevents:

• Disabling audit system
• Modifying audit rules
• Tampering with audit configuration

To modify immutable configuration:

• Reboot system
• Rules load before -e 2 during boot
• Make changes before immutability flag

2. Implement Comprehensive User Monitoring

Track all executed commands:

-a always,exit -F arch=b64 -S execve -k command_execution
-a always,exit -F arch=b32 -S execve -k command_execution

Warning: Generates significant log volume.

3. Monitor Container Activities

For Docker:

-w /usr/bin/docker -k docker_commands
-w /var/lib/docker -p wa -k docker_data
-w /etc/docker -p wa -k docker_config

For Kubernetes:

-w /usr/bin/kubectl -k kubectl_commands
-w /etc/kubernetes -p wa -k kubernetes_config

4. Implement Database Monitoring

For MySQL:

-w /var/lib/mysql -p wa -k mysql_data
-w /etc/mysql -p wa -k mysql_config

For PostgreSQL:

-w /var/lib/postgresql -p wa -k postgres_data
-w /etc/postgresql -p wa -k postgres_config

5. Monitor Module Loading

Track kernel module operations:

-a always,exit -F arch=b64 -S init_module -S delete_module -k kernel_modules
-a always,exit -F arch=b32 -S init_module -S delete_module -k kernel_modules

6. Implement Performance Tuning

Increase audit buffer:

-b 16384

Set appropriate failure mode:

# 0 = silent failures
# 1 = print to kernel log
# 2 = panic (system halt)
-f 1

Use rate limiting for noisy rules:

-a always,exit -F arch=b64 -S open -F success=0 -F rate=100/s -k failed_file_access

7. Implement Alert Mechanisms

Create alert script:

#!/bin/bash
# /usr/local/bin/audit-alert.sh

KEY="$1"
MESSAGE=$(sudo ausearch -k "$KEY" -ts recent --raw | tail -20)

echo "Audit Alert: $KEY" | \
mail -s "Security Alert: $KEY" [email protected] -A <(echo "$MESSAGE")

Add to cron for specific keys:

*/5 * * * * /usr/local/bin/audit-alert.sh passwd_changes

8. Implement Log Integrity

Sign audit logs with aide or rsyslog:

Configure rsyslog to sign logs:

sudo nano /etc/rsyslog.d/50-audit-sign.conf

$ModLoad imfile
$InputFileName /var/log/audit/audit.log
$InputFileTag audit:
$InputFileStateFile audit-state
$InputFileSeverity info
$InputFileFacility local6
$InputRunFileMonitor

local6.* @@log-server.example.com:514

Verification and Testing

Verify auditd is Running

Check service status:

sudo systemctl status auditd
sudo service auditd status

Check audit rules:

sudo auditctl -l

Should show all your configured rules.

Test File Watches

Create test action:

sudo touch /etc/test-audit-file
echo "test" | sudo tee /etc/test-audit-file
sudo rm /etc/test-audit-file

Search for events:

sudo ausearch -f /etc/test-audit-file -i

Should show creation, modification, and deletion events.

Test System Call Rules

Test sudo monitoring:

sudo ls /root

Search for event:

sudo ausearch -k sudo_usage -i

Test User Monitoring

Execute some commands:

whoami
ls /tmp
cat /etc/hosts

Search user's actions:

sudo ausearch -ua $(whoami) -ts recent -i

Verify Log Rotation

Check log files:

ls -lh /var/log/audit/

Check rotation configuration:

cat /etc/logrotate.d/auditd

Performance Testing

Check audit daemon resource usage:

ps aux | grep auditd
top -p $(pidof auditd)

Check audit queue:

auditctl -s

Look for:

• lost events (should be 0)
• backlog queue size

Troubleshooting Common Issues

Issue 1: Audit Rules Not Loading

Symptoms: Rules don't appear in auditctl -l

Solutions:

1. Check syntax in rules file:

   sudo auditctl -R /etc/audit/rules.d/audit.rules
   

2. Check for errors:

   sudo tail -f /var/log/audit/audit.log | grep -i error
   

3. Verify file location:

   ls -l /etc/audit/rules.d/
   ls -l /etc/audit/audit.rules
   

4. Reload rules:

   sudo augenrules --load
   sudo service auditd restart
   

Issue 2: High CPU Usage

Symptoms: auditd consuming excessive CPU

Solutions:

1. Identify noisy rules:

   sudo aureport -k --summary
   

2. Remove or modify high-volume rules

3. Increase buffer size:

   -b 16384
   

4. Use rate limiting:

   -a always,exit -F arch=b64 -S open -F rate=100/s
   

Issue 3: Disk Space Filling Up

Symptoms: /var/log/audit/ consuming too much space

Solutions:

1. Check current log size:

   du -sh /var/log/audit/
   

2. Reduce max_log_file in /etc/audit/auditd.conf:

   max_log_file = 50
   

3. Reduce retention:

   num_logs = 5
   

4. Implement more aggressive compression:

   sudo nano /etc/logrotate.d/auditd
   

5. Consider centralized logging to offload storage

Issue 4: Lost Audit Events

Symptoms: auditctl -s shows lost events

Solutions:

1. Increase buffer:

   -b 32768
   

2. Check disk I/O:

   iostat -x 1
   

3. Reduce rule complexity

4. Increase audit daemon priority: In /etc/audit/auditd.conf:

   priority_boost = 6
   

Issue 5: Cannot Modify Audit Configuration

Symptoms: Changes rejected, "Operation not permitted"

Cause: Audit system is immutable (-e 2)

Solutions:

1. Check immutability:

   sudo auditctl -s | grep enabled
   

2. If enabled=2, must reboot:

   sudo reboot
   

3. Remove -e 2 from rules file before next boot if needed

Issue 6: Too Many False Positives

Symptoms: Too many irrelevant audit events

Solutions:

1. Add exclusions:

   -a never,exit -F path=/usr/bin/update-notifier
   

2. Use more specific filters:

   # Instead of monitoring all /tmp
   -w /tmp/sensitive-app/ -p wa -k sensitive_tmp
   

3. Filter by user:

   -a always,exit -F auid>=1000 -F auid!=4294967295
   

   This excludes system users.

Best Practices for auditd Management

1. Rule Design Strategy

• Start minimal: Begin with essential compliance requirements
• Add gradually: Expand monitoring based on risk assessment
• Test thoroughly: Test rules in non-production first
• Document everything: Maintain clear documentation of all rules
• Regular review: Audit rules quarterly for relevance

2. Performance Management

• Monitor overhead: Track CPU and disk I/O impact
• Tune buffer size: Adjust based on event volume
• Use rate limiting: For high-frequency events
• Filter intelligently: Exclude unnecessary events
• Centralize logs: Offload storage and processing

3. Log Management

• Implement rotation: Prevent disk space exhaustion
• Compress old logs: Save space while retaining history
• Centralized logging: Forward to SIEM or log aggregation
• Retention policy: Define based on compliance requirements
• Backup regularly: Protect audit logs from loss

4. Security Hardening

• Make immutable: Use -e 2 for production systems
• Monitor audit system: Alert on audit failures
• Protect logs: Restrict access to audit logs
• Integrity checking: Implement log signing or hashing
• Redundancy: Forward logs to multiple destinations

5. Operational Practices

• Baseline behavior: Establish normal activity patterns
• Regular analysis: Review logs weekly minimum
• Automate alerts: Notify on critical events immediately
• Incident response: Include audit logs in investigation procedures
• Change management: Update audit rules with system changes

6. Compliance Alignment

• Map to requirements: Align rules with specific compliance needs
• Regular audits: Verify auditing meets regulatory requirements
• Evidence collection: Maintain audit logs per retention policies
• Access controls: Limit who can view and manage audit systems
• Documentation: Maintain audit configuration documentation

7. Integration and Automation

• SIEM integration: Forward events to security monitoring platform
• Automated analysis: Use tools to identify anomalies
• Correlation: Combine with other log sources
• Alerting: Configure real-time alerts for critical events
• Reporting: Generate compliance reports automatically

Conclusion

The Linux Audit system (auditd) provides comprehensive, kernel-level monitoring essential for security, compliance, and forensic analysis. By implementing the practices outlined in this guide, you create a robust auditing infrastructure that provides visibility into system activities, enables detection of security incidents, and demonstrates compliance with regulatory requirements.

Key takeaways:

• Essential visibility: auditd provides detailed logging of security-relevant events
• Compliance requirement: Many frameworks require comprehensive audit logging
• Forensic value: Detailed audit trails enable effective incident investigation
• Balanced approach: Audit what's necessary without overwhelming systems
• Immutable protection: Use immutability for production audit configurations
• Centralized management: Forward logs to SIEM for analysis and retention

auditd is particularly valuable for:

• Meeting compliance requirements (PCI DSS, HIPAA, SOX, ISO 27001)
• Detecting unauthorized access and privilege escalation
• Forensic investigation of security incidents
• Monitoring administrative actions and changes
• Tracking access to sensitive data
• Demonstrating due diligence in security

Remember that effective auditing requires:

• Careful rule design balancing visibility and performance
• Regular log analysis and review
• Integration with broader security monitoring
• Proper log retention and protection
• Documentation and change management

Start with essential compliance requirements, expand based on risk assessment, and continuously refine your audit configuration. With proper implementation and maintenance, auditd becomes an invaluable security control that provides early warning of security incidents, enables effective forensic analysis, and demonstrates your commitment to security and compliance. Combined with other security controls, comprehensive auditing forms a critical component of defense-in-depth security strategy.