How to Analyze Linux Logs in /var/log and journalctl

When a service crashes, a disk fills up or someone tries to break into your server, the explanation is almost always in the logs. On Ubuntu 24.04, logs live in two places: plain text files under /var/log, written by rsyslog and by applications such as Nginx, and the binary systemd journal, read with journalctl. In this tutorial you will learn which file holds what, how to filter logs quickly by time, service and severity, and how to answer common questions such as "why did this service stop?" and "who is trying to log in over SSH?".

Prerequisites

To follow this guide you need:

• A server running Ubuntu 24.04 LTS, for example a CubePath VPS. The commands also work on Debian 12, with the file name differences noted below.
• A non-root user with sudo privileges. Most system logs are readable only by root and the adm group.

On Rocky Linux and other RHEL-based systems, /var/log/syslog is called /var/log/messages and /var/log/auth.log is /var/log/secure. The journalctl commands are identical.

Step 1 - Getting to know /var/log

List the directory, sorted by modification time so the most active logs are at the top:

ls -lt /var/log | head -n 15

These are the files and directories you will use most on Ubuntu 24.04:

Rotated logs sit next to the current file: syslog.1 is the previous period and syslog.2.gz, syslog.3.gz are older compressed copies.

Note

On Debian 12, rsyslog is not installed by default, so /var/log/syslog and /var/log/auth.log do not exist. Use journalctl (Step 3) or install rsyslog with sudo apt install rsyslog.

To read these files without typing sudo every time, you can add your user to the adm group and log in again:

sudo usermod -aG adm your_user

Step 2 - Reading and searching text logs

Show the last 50 lines of a log:

sudo tail -n 50 /var/log/syslog

Follow a log live while you reproduce a problem, for example restarting a service in another terminal. Press Ctrl+C to stop:

sudo tail -f /var/log/syslog

To page through a large file, use less. Press G to jump to the end, / to search forward, n for the next match, F to follow new lines like tail -f and q to quit:

sudo less /var/log/syslog

Search for a term with grep. -i ignores case and -C 2 shows two lines of context before and after each match:

sudo grep -i -C 2 'error' /var/log/syslog

Count matches instead of printing them:

sudo grep -ci 'error' /var/log/syslog

Rotated .gz files cannot be searched with grep directly. zgrep reads compressed and plain files alike, so you can search the whole history at once:

sudo zgrep -h 'Out of memory' /var/log/kern.log*

Step 3 - Querying the systemd journal

Every service managed by systemd sends its output to the journal, even when it writes no text log file. This makes journalctl the fastest way to see why a service failed.

Check that the journal is persistent, so logs survive reboots. On Ubuntu 24.04 the directory exists by default:

ls -d /var/log/journal

/var/log/journal

Show the logs of one service. Service names are the unit names, for example ssh, nginx or cron:

sudo journalctl -u ssh

The most useful filters, which you can combine:

sudo journalctl -u nginx -n 100 --no-pager

Shows the last 100 lines of Nginx without the pager.

sudo journalctl -u nginx -f

Follows new Nginx entries live.

sudo journalctl --since "2026-09-25 09:00" --until "2026-09-25 10:00"

Limits output to a time window. Relative values like --since "1 hour ago" or --since yesterday also work.

sudo journalctl -p err -b

Shows only messages of priority err or more severe (crit, alert, emerg) since the current boot. This is a good first command after an incident.

sudo journalctl -k -b -1

Shows kernel messages from the previous boot, which is where you find the reason for an unexpected reboot or a crash.

List the boots the journal knows about:

sudo journalctl --list-boots

IDX BOOT ID                          FIRST ENTRY                 LAST ENTRY
 -1 5c1e0d3f2b7a4f3e9d7c8a1b2c3d4e5f Mon 2026-09-15 08:12:01 UTC Wed 2026-09-24 22:41:10 UTC
  0 a9b8c7d6e5f4a3b2c1d0e9f8a7b6c5d4 Wed 2026-09-24 22:43:05 UTC Thu 2026-09-25 10:15:44 UTC

When you only need to know why a unit is failing, systemctl status shows its state together with the last log lines:

sudo systemctl status nginx

Step 4 - Investigating SSH login attempts

Any server with a public IP receives constant SSH brute-force attempts. auth.log records each one. Count failed password attempts per source IP:

sudo grep 'Failed password' /var/log/auth.log | grep -oE 'from [0-9a-f.:]+' | awk '{print $2}' | sort | uniq -c | sort -rn | head

    412 198.51.100.77
    288 203.0.113.201
     36 192.0.2.14

See the attempts that used usernames that do not exist on the server:

sudo grep 'Invalid user' /var/log/auth.log | awk '{for (i=1; i<=NF; i++) if ($i == "user") print $(i+1)}' | sort | uniq -c | sort -rn | head

List successful logins, which is what matters most in an investigation:

sudo grep 'Accepted' /var/log/auth.log

2026-09-25T08:03:11.482915+00:00 web01 sshd[1834]: Accepted publickey for your_user from 192.0.2.10 port 51844 ssh2: ED25519 SHA256:...

Every Accepted line should come from a user and IP address you recognize. Also check who ran commands with sudo:

sudo grep 'sudo:' /var/log/auth.log | grep 'COMMAND='

The last command summarizes login sessions, including reboots:

last -n 10

If you see thousands of failed attempts, disable password authentication in SSH and use key-based logins only, or install Fail2ban to block offending IPs automatically.

Step 5 - Analyzing web server access logs

Application logs follow their own formats. Nginx and Apache use the combined log format by default, where each request is one line and the fields are separated by spaces. That makes awk a quick analysis tool. The examples use /var/log/nginx/access.log; for Apache, use /var/log/apache2/access.log.

The 10 IP addresses making the most requests:

sudo awk '{print $1}' /var/log/nginx/access.log | sort | uniq -c | sort -rn | head

Requests per HTTP status code. A jump in 5xx codes means the application is failing; many 404 codes often mean a scanner is probing for known paths:

sudo awk '{print $9}' /var/log/nginx/access.log | sort | uniq -c | sort -rn

  18542 200
   1204 304
    873 404
     61 502

The URLs that returned server errors:

sudo awk '$9 ~ /^5/ {print $7}' /var/log/nginx/access.log | sort | uniq -c | sort -rn | head

When you find 502 errors, look at the Nginx error log and at the backend service it proxies to at the same timestamps:

sudo tail -n 50 /var/log/nginx/error.log

Step 6 - Managing log rotation and disk usage

Logs grow continuously. Two mechanisms keep them in check on Ubuntu: logrotate for text files and journald's own size limits for the journal.

Check how much space logs use:

sudo du -sh /var/log
sudo journalctl --disk-usage

412M    /var/log
Archived and active journals take 296.0M in the file system.

logrotate runs daily from a systemd timer and reads one configuration file per package in /etc/logrotate.d/. Look at the one for rsyslog files:

cat /etc/logrotate.d/rsyslog

To add rotation for your own application's logs in /var/log/myapp/, create a file:

sudo nano /etc/logrotate.d/myapp

/var/log/myapp/*.log {
    daily
    rotate 14
    compress
    delaycompress
    missingok
    notifempty
    copytruncate
}

This keeps 14 daily copies, compresses all but the most recent and skips empty or missing files. copytruncate lets the application keep writing to the same file descriptor; if your application can reopen its log on a signal, use a postrotate script instead.

Test the configuration in debug mode, which shows what would happen without changing anything:

sudo logrotate -d /etc/logrotate.d/myapp

For the journal, set a maximum size so it cannot fill the disk. Create a drop-in file:

sudo mkdir -p /etc/systemd/journald.conf.d
sudo nano /etc/systemd/journald.conf.d/size.conf

[Journal]
SystemMaxUse=500M

Apply it and verify:

sudo systemctl restart systemd-journald
sudo journalctl --disk-usage

To free space immediately, remove archived journal files older than two weeks:

sudo journalctl --vacuum-time=2weeks

Warning

Never delete an open log file with rm to free space. The process keeps writing to the deleted file and the space is not released until it restarts. Use logrotate, or empty the file with sudo truncate -s 0 /path/to/file.log.

Troubleshooting

• journalctl -b -1 says Specified boot ID not found: the journal is not persistent or the previous boot has been rotated out. Check that /var/log/journal exists and that SystemMaxUse is not too small.
• /var/log/syslog or /var/log/auth.log is missing: rsyslog is not installed or not running. Check with systemctl status rsyslog, or use journalctl instead.
• Permission denied when reading a log: the file is owned by root or syslog with group adm. Use sudo or add your user to adm as shown in Step 1.
• A log file keeps growing after rotation: the application still writes to the old, renamed file. Add copytruncate or a postrotate command that tells the service to reopen its logs.

Conclusion

You now know where Ubuntu 24.04 keeps its logs, how to search text logs with tail, grep and zgrep, how to filter the systemd journal by service, time, boot and priority, and how to use awk to turn raw log lines into answers about SSH attacks and web errors. As next steps, harden SSH and install Fail2ban based on what auth.log shows, and ship logs to a central system such as Loki or Graylog so you can search across servers and keep history after a disk failure.