Audit logs record who changed what on a system: edits to user accounts, sudo configuration, SSH settings, clock changes and privilege escalation. Compliance frameworks require keeping them for a year or more and proving they were not altered. In this tutorial you will configure the Linux audit daemon (auditd) on Ubuntu 24.04 with a focused rule set, make the rules immutable, size local rotation, and archive every rotated log to S3 compatible object storage with Object Lock and SHA-256 checksums through a systemd timer. You will finish by searching both live and archived logs.

Prerequisites

To follow this tutorial you need:

  • A server running Ubuntu 24.04 LTS, for example a CubePath VPS, and a non-root user with sudo privileges.
  • An S3 compatible bucket (AWS S3, MinIO, Wasabi, Cloudflare R2 or similar) created with versioning and Object Lock enabled, plus an access key that can write to it. On most providers Object Lock can only be enabled when the bucket is created.
  • A few GB of free disk space under /var/log/audit.

How long to keep audit logs

Retention is set by your policy and the regulations you fall under. Common reference points:

FrameworkTypical requirement
PCI DSS12 months, with the last 3 months immediately available
HIPAA6 years for required documentation, commonly applied to audit trails
SOX7 years for audit-related records

A practical layout is a few weeks on local disk for fast searches and the full retention period in object storage.

Step 1 - Installing auditd

Install the audit daemon and its utilities:

sudo apt update
sudo apt install auditd

The service starts automatically. Check its state:

sudo auditctl -s

enabled 1
failure 1
pid 812
rate_limit 0
backlog_limit 8192
lost 0
backlog 0
[...]

enabled 1 means auditing is active. lost 0 means no events were dropped.

Step 2 - Adding audit rules

Rules live in /etc/audit/rules.d/ and are compiled into /etc/audit/audit.rules by augenrules. Files are processed in name order. Ubuntu's audit.rules in that directory already clears old rules and sets the buffer size. augenrules always moves those control lines (-D, -b, -f) to the top and -e to the end, so your own rules can go in a separate numbered file:

sudo nano /etc/audit/rules.d/50-baseline.rules

## Identity and authentication files
-w /etc/passwd -p wa -k identity
-w /etc/group -p wa -k identity
-w /etc/shadow -p wa -k identity
-w /etc/gshadow -p wa -k identity

## Privilege configuration
-w /etc/sudoers -p wa -k sudoers
-w /etc/sudoers.d/ -p wa -k sudoers

## SSH server configuration
-w /etc/ssh/sshd_config -p wa -k sshd
-w /etc/ssh/sshd_config.d/ -p wa -k sshd

## Audit configuration itself
-w /etc/audit/ -p wa -k audit_config

## Clock changes
-a always,exit -F arch=b64 -S adjtimex,settimeofday,clock_settime -k time_change
-w /etc/localtime -p wa -k time_change

## Programs run with root privileges by a non-root user (sudo, setuid binaries)
-a always,exit -F arch=b64 -S execve -C uid!=euid -F euid=0 -k priv_esc

## Kernel module loading and unloading
-a always,exit -F arch=b64 -S init_module,finit_module,delete_module -k modules

Each -w line watches a file or directory for writes and attribute changes (-p wa). Each -a always,exit line audits a system call. The -k key is a label you will search by later. Keep the rule set focused: auditing every file deletion or every system call run by root produces gigabytes per day on a busy server and hides the events that matter.

Now add a separate final file that locks the configuration:

echo "-e 2" | sudo tee /etc/audit/rules.d/99-finalize.rules

-e 2 makes the rules immutable until the next reboot: not even root can add, remove or disable rules at runtime, and any attempt is itself logged. This is what auditors expect, but it also means every rule change from now on requires a reboot.

Load the rules and check them:

sudo augenrules --load
sudo auditctl -l

-w /etc/passwd -p wa -k identity
-w /etc/group -p wa -k identity
[...]
-a always,exit -F arch=b64 -S init_module,delete_module,finit_module -F key=modules

Verify the lock with sudo auditctl -s, which now reports enabled 2. Test a rule by touching a watched file and searching for its key:

sudo touch /etc/sudoers.d/README
sudo ausearch -k sudoers -ts recent -i | tail -n 3

The output contains a SYSCALL record with comm=touch and your user in auid=, the login user who ran the command even through sudo.

Step 3 - Sizing local rotation

auditd rotates its own logs; do not add a logrotate rule for /var/log/audit/audit.log, as the two conflict. The defaults on Ubuntu (8 MB files, 5 files kept) are too small to keep more than a few hours on a busy server.

Open the daemon configuration:

sudo nano /etc/audit/auditd.conf

Find and change these existing keys, leaving the rest of the file as it is:

max_log_file = 100
num_logs = 30
max_log_file_action = ROTATE
space_left = 2048
space_left_action = SYSLOG
admin_space_left = 1024
admin_space_left_action = SUSPEND
disk_full_action = SUSPEND

This keeps up to 30 files of 100 MB (about 3 GB). space_left and admin_space_left are in MB: when free space on the partition drops below 2 GB, a warning goes to syslog; below 1 GB, auditd stops writing. If your policy says the system must not run unaudited, use HALT instead of SUSPEND for the admin and disk-full actions, knowing that a full disk then stops the server.

Tell auditd to reread its configuration:

sudo systemctl kill -s SIGHUP auditd
sudo ausearch -m DAEMON_CONFIG -ts recent

A DAEMON_CONFIG record with res=success confirms the change. For more protection against a full /var, put /var/log/audit on its own partition or logical volume.

Step 4 - Configuring access to object storage

Local files can be deleted by anyone with root on the server. Copies in a bucket with Object Lock in compliance mode cannot be deleted or overwritten by anyone, including the account owner, until their retention period ends. Set the bucket's default retention to your policy (for example 7 years). With the AWS CLI on an admin workstation it looks like this:

aws s3api put-object-lock-configuration \
  --bucket your_bucket \
  --object-lock-configuration '{"ObjectLockEnabled":"Enabled","Rule":{"DefaultRetention":{"Mode":"COMPLIANCE","Years":7}}}'

Other providers expose the same setting in their console or through the same S3 API call. Give the server an access key that can only put and read objects in this bucket, not delete them or change bucket settings.

On the server, install rclone, which works with any S3 compatible provider:

sudo apt install rclone

Create a configuration file readable only by root:

sudo install -d -m 700 /etc/audit-archive
sudo nano /etc/audit-archive/rclone.conf

[archive]
type = s3
provider = Other
access_key_id = your_access_key
secret_access_key = your_secret_key
endpoint = https://your_s3_endpoint
region = your_region
no_check_bucket = true

For AWS, set provider = AWS and remove the endpoint line. no_check_bucket = true stops rclone from trying to create the bucket, which a restricted key cannot do. Protect the file and test access:

sudo chmod 600 /etc/audit-archive/rclone.conf
sudo rclone --config /etc/audit-archive/rclone.conf lsd archive:your_bucket

An empty result without errors means the credentials work.

Step 5 - Writing the archive script

The script closes the current log with a rotation, then uploads every rotated file that has not been archived yet. Because auditd renames files on each rotation (audit.log.1 becomes audit.log.2), the script identifies files by their SHA-256 hash, not by name, and keeps a local manifest of what it already sent. Each archive gets a companion .sha256 file so its integrity can be checked years later.

sudo nano /usr/local/sbin/audit-archive

#!/usr/bin/env bash
set -euo pipefail

BUCKET="your_bucket"
HOST="$(hostname -s)"
LOG_DIR="/var/log/audit"
STATE_DIR="/var/lib/audit-archive"
MANIFEST="$STATE_DIR/archived.sha256"
STAGING="$STATE_DIR/staging"
RCLONE=(rclone --config /etc/audit-archive/rclone.conf)

mkdir -p "$STAGING"
touch "$MANIFEST"

# Close the current log so everything up to now is archived
systemctl kill -s SIGUSR1 auditd.service
sleep 5

shopt -s nullglob
for file in "$LOG_DIR"/audit.log.[0-9]*; do
    sum="$(sha256sum "$file" | cut -d' ' -f1)"
    if grep -q "^$sum " "$MANIFEST"; then
        continue
    fi

    stamp="$(date -u -r "$file" +%Y%m%dT%H%M%SZ)"
    month="$(date -u -r "$file" +%Y/%m)"
    name="audit-$HOST-$stamp.log.gz"

    gzip -c "$file" > "$STAGING/$name"
    (cd "$STAGING" && sha256sum "$name" > "$name.sha256")

    "${RCLONE[@]}" copy "$STAGING" "archive:$BUCKET/$HOST/$month"

    echo "$sum $name" >> "$MANIFEST"
    rm -f "$STAGING/$name" "$STAGING/$name.sha256"
    echo "archived $file as $HOST/$month/$name"
done

Replace your_bucket with your bucket name, then make the script executable:

sudo chmod 750 /usr/local/sbin/audit-archive

Notes on the design:

  • SIGUSR1 is auditd's documented signal for "rotate now".
  • The file's modification time is the time of its last event and does not change when auditd renames it, so it makes a stable archive name.
  • If an upload fails, set -e stops the script before the manifest is updated, and the next run retries.

Run it once by hand:

sudo /usr/local/sbin/audit-archive

archived /var/log/audit/audit.log.1 as web-01/2026/09/audit-web-01-20260925T103512Z.log.gz

Run it a second time: it only archives the file created by the new rotation, confirming that already-uploaded files are skipped. List the bucket:

sudo rclone --config /etc/audit-archive/rclone.conf ls archive:your_bucket

Step 6 - Scheduling the archive with a systemd timer

Create the service unit:

sudo nano /etc/systemd/system/audit-archive.service

[Unit]
Description=Archive rotated audit logs to object storage
Wants=network-online.target
After=network-online.target

[Service]
Type=oneshot
ExecStart=/usr/local/sbin/audit-archive

And the timer, which runs it every night:

sudo nano /etc/systemd/system/audit-archive.timer

[Unit]
Description=Nightly audit log archive

[Timer]
OnCalendar=*-*-* 00:30:00
RandomizedDelaySec=15m
Persistent=true

[Install]
WantedBy=timers.target

Persistent=true runs a missed job after the server was powered off at the scheduled time. Enable the timer and check the next run:

sudo systemctl daemon-reload
sudo systemctl enable --now audit-archive.timer
systemctl list-timers audit-archive.timer

Each run's output ends up in the journal: journalctl -u audit-archive.service. Make sure num_logs in Step 3 covers well over a day of rotations, so files are never deleted locally before the nightly upload.

Step 7 - Searching live and archived logs

ausearch finds events by key, user, type or time, and -i translates numeric IDs into names:

# Changes to users and groups today
sudo ausearch -k identity -ts today -i

# Commands executed with root privileges by non-root users this week
sudo ausearch -k priv_esc -ts this-week -i

# Failed authentication attempts
sudo ausearch -m USER_AUTH -sv no -ts today -i

aureport summarizes:

sudo aureport --summary -ts this-month
sudo aureport -au --failed -ts this-week
sudo aureport -k --summary

For an investigation that goes back beyond local retention, download the archive and its checksum, verify it and search it with ausearch -if:

mkdir -p ~/investigation && cd ~/investigation
sudo rclone --config /etc/audit-archive/rclone.conf copy archive:your_bucket/web-01/2026/09/ . --include "audit-web-01-20260925T103512Z.log.gz*"
sha256sum -c audit-web-01-20260925T103512Z.log.gz.sha256

audit-web-01-20260925T103512Z.log.gz: OK

zcat audit-web-01-20260925T103512Z.log.gz > audit.log
sudo ausearch -if audit.log -k sudoers -i

Because both the archive and its checksum are under Object Lock, a mismatch can only come from corruption in transit, never from someone rewriting both files.

Troubleshooting

augenrules --load reports that rules are locked. The -e 2 flag is active. Edit the files in /etc/audit/rules.d/ and reboot to apply them.

auditctl -s shows lost increasing. Events arrive faster than the kernel buffer drains. Remove noisy rules or raise the -b value in /etc/audit/rules.d/audit.rules, then reboot.

The archive job fails with AccessDenied. The key lacks PutObject on the bucket, or the provider requires a checksum header for Object Lock uploads. Check the exact error with journalctl -u audit-archive.service and test with rclone copy and the -vv flag.

Old audit files disappear before they are archived. Rotation is faster than the archive schedule. Increase num_logs or max_log_file, or run the timer more often, for example OnCalendar=hourly.

Conclusion

Your Ubuntu 24.04 server now records security-relevant changes with a locked auditd rule set, keeps a few weeks of logs locally, and ships every rotated file to write-once object storage with checksums on a nightly timer. As next steps, forward events in real time to a central log host with the audisp-remote plugin so an attacker with root cannot suppress recent events, add rules for your own sensitive application directories, and document a periodic review of aureport output to satisfy the review requirements in PCI DSS and similar frameworks.