systemd-journald collects logs from the kernel, from every systemd service and from syslog into a single indexed journal, and journalctl is the tool to query it. Because each entry carries structured fields such as the unit, PID and priority, you can filter much more precisely than with plain text files. In this tutorial you will configure journald on Ubuntu 24.04 so logs survive reboots without filling the disk, then use journalctl to filter by service, time, boot, priority and field, and finally export and clean up the journal.
Prerequisites
To follow this tutorial you need:
- A server running Ubuntu 24.04 LTS, such as a CubePath VPS. The commands also work on Debian 12 and Rocky Linux 9; the differences are noted where they matter.
- A non-root user with
sudoprivileges.
Regular users only see their own logs. To read the whole system journal without sudo, add your user to the systemd-journal group (members of adm can also read it on Ubuntu) and log in again:
sudo usermod -aG systemd-journal your_user
The examples below still use sudo so they work either way.
Step 1 - Enabling persistent storage
journald keeps logs either in memory under /run/log/journal, which is lost on reboot, or on disk under /var/log/journal. The default setting, Storage=auto, stores logs on disk only if /var/log/journal exists. Ubuntu creates that directory, while Debian 12 and some minimal images do not.
Check whether you already have logs from previous boots:
sudo journalctl --list-boots
IDX BOOT ID FIRST ENTRY LAST ENTRY
-2 5f1c0e5a9c2d4b1d8f6a3e7b9c1d2e3f Mon 2026-09-15 08:11:02 UTC Tue 2026-09-23 17:45:10 UTC
-1 0a9b8c7d6e5f4a3b2c1d0e9f8a7b6c5d Tue 2026-09-23 17:46:01 UTC Thu 2026-09-25 09:58:40 UTC
0 3e2d1c0b9a8f7e6d5c4b3a2f1e0d9c8b Thu 2026-09-25 10:00:12 UTC Thu 2026-09-25 12:31:55 UTC
If only boot 0 is listed on a server that has been rebooted before, the journal is volatile. Rather than relying on the directory existing, make persistence explicit with a drop-in file. Drop-ins in /etc/systemd/journald.conf.d/ override the defaults without editing the packaged journald.conf:
sudo mkdir -p /etc/systemd/journald.conf.d
sudo nano /etc/systemd/journald.conf.d/10-persistent.conf
[Journal]
Storage=persistent
Restart journald so it creates /var/log/journal and starts writing there:
sudo systemctl restart systemd-journald
ls /var/log/journal/
d3f1a2b4c5d6e7f8091a2b3c4d5e6f70
The directory name is the machine ID. From now on, journalctl --list-boots will show one line per boot.
Step 2 - Limiting the journal size and retention
With persistent storage, journald by default uses up to 10% of the file system (capped at 4 GB) and always leaves 15% free. On a small VPS disk you may want a fixed limit and a maximum age. Create a second drop-in:
sudo nano /etc/systemd/journald.conf.d/20-retention.conf
[Journal]
SystemMaxUse=1G
SystemKeepFree=2G
SystemMaxFileSize=128M
MaxRetentionSec=1month
SystemMaxUse=1G: the journal on disk never grows beyond 1 GB.SystemKeepFree=2G: journald stops using space when less than 2 GB would remain free on the file system.SystemMaxFileSize=128M: individual journal files are rotated at this size, so old data can be removed in reasonable chunks.MaxRetentionSec=1month: entries older than one month are deleted even if there is space left.
Apply the change and check the effective configuration, which merges journald.conf with all drop-ins:
sudo systemctl restart systemd-journald
systemd-analyze cat-config systemd/journald.conf | grep -v '^#' | grep .
[Journal]
[Journal]
Storage=persistent
[Journal]
SystemMaxUse=1G
SystemKeepFree=2G
SystemMaxFileSize=128M
MaxRetentionSec=1month
Check how much space the journal uses right now:
sudo journalctl --disk-usage
Archived and active journals take 312.0M in the file system.
Step 3 - Viewing and following logs
Running journalctl without options shows the entire journal, oldest first, in a pager. In practice you almost always add a few options:
# Last 50 entries, no pager
sudo journalctl -n 50 --no-pager
# Follow new entries in real time
sudo journalctl -f
# Jump to the end of the journal inside the pager
sudo journalctl -e
# Newest entries first
sudo journalctl -r
Add -x to show explanatory help text from the message catalog next to some entries, which is useful when a unit fails.
Step 4 - Filtering by unit, time and boot
The most common filter is the systemd unit. On Ubuntu 24.04 the SSH server unit is called ssh; on Rocky Linux it is sshd:
sudo journalctl -u ssh
sudo journalctl -u nginx -u php8.3-fpm
When you pass several -u options, entries from any of them are shown, interleaved by time.
Filter by time with --since and --until. Both accept absolute timestamps and relative expressions:
sudo journalctl --since "2026-09-25 09:00" --until "2026-09-25 10:30"
sudo journalctl --since "1 hour ago"
sudo journalctl --since yesterday --until today
sudo journalctl -u nginx --since "15 min ago"
Filter by boot with -b. Without an argument it means the current boot; -1 is the previous one. This is how you find out why a server rebooted or crashed:
sudo journalctl -b -1 -e
The last lines of the previous boot show whether the shutdown was clean (services stopping one by one) or abrupt (the log simply ends).
Kernel messages, the same ones dmesg shows, are selected with -k:
sudo journalctl -k -b
Step 5 - Filtering by priority and message content
Every entry has a syslog priority from 0 (emerg) to 7 (debug). -p shows entries with the given priority or more severe:
| Priority | Name | Typical use |
|---|---|---|
| 0 | emerg | System is unusable |
| 1 | alert | Immediate action required |
| 2 | crit | Critical conditions |
| 3 | err | Errors |
| 4 | warning | Warnings |
| 5 | notice | Normal but significant |
| 6 | info | Informational |
| 7 | debug | Debug messages |
# Errors and worse in the current boot
sudo journalctl -p err -b
# Only warnings, not errors
sudo journalctl -p warning..warning --since today
To search inside messages, use -g (--grep). It takes a regular expression and is case-insensitive when the pattern is all lowercase:
sudo journalctl -u ssh -g 'failed password' --since today
sudo journalctl -k -g 'out of memory|oom-kill'
Sep 25 11:42:07 web01 kernel: Out of memory: Killed process 20431 (php-fpm8.3) total-vm:812344kB, anon-rss:402112kB
Step 6 - Filtering by journal fields
Each journal entry is a set of fields. Fields starting with _ are added by journald itself and cannot be faked by the application. Display every field of the latest entry of a unit with the verbose output:
sudo journalctl -u nginx -n 1 -o verbose
Use FIELD=value arguments to filter on any of them:
# Everything logged by one process
sudo journalctl _PID=1422
# Everything logged by a program, whatever unit started it
sudo journalctl _COMM=sshd
# Messages with a given syslog identifier (same as -t sshd)
sudo journalctl SYSLOG_IDENTIFIER=sshd
# Everything logged by processes running as UID 1000
sudo journalctl _UID=1000
Different fields are combined with AND, and repeating the same field combines the values with OR. To discover which values exist for a field, use -F:
sudo journalctl -F _SYSTEMD_UNIT | sort | head
cron.service
nginx.service
ssh.service
systemd-journald.service
systemd-logind.service
Step 7 - Changing the output format and exporting logs
-o changes the output format. The most useful ones are:
| Format | Use |
|---|---|
short-iso | Like the default, with ISO 8601 timestamps |
cat | Only the message text, good for piping to grep or awk |
json | One JSON object per line, for scripts and log shippers |
json-pretty | Indented JSON, for reading |
export | Binary-safe format to move journal data to another machine |
Export one service's logs from yesterday as JSON to attach to a ticket or feed to another tool:
sudo journalctl -u nginx --since yesterday --until today -o json --no-pager > nginx-yesterday.json
Because each line is a JSON object, jq can process the file. Install it with sudo apt install jq, then, for example, count messages per priority:
jq -r '.PRIORITY' nginx-yesterday.json | sort | uniq -c
To keep a full copy of the journal for later analysis, including all fields, copy the journal files themselves and open them with --directory on any machine with systemd:
sudo tar -czf journal-web01.tar.gz -C /var/log/journal .
mkdir journal-web01 && tar -xzf journal-web01.tar.gz -C journal-web01
journalctl --directory=journal-web01 -p err
Step 8 - Cleaning up the journal manually
The limits from Step 2 are enforced automatically, but sometimes you need to free space right away, for example when a disk is almost full. --vacuum-* deletes archived journal files until the condition is met:
# Keep only the last two weeks
sudo journalctl --vacuum-time=2weeks
# Shrink the journal to 500 MB
sudo journalctl --vacuum-size=500M
Deleted archived journal /var/log/journal/d3f1a2b4.../[email protected]~ (128.0M).
Vacuuming done, freed 384.0M of archived journals from /var/log/journal/d3f1a2b4....
Vacuuming only removes archived files, not the one being written. To include recent data, rotate first so the active file is archived:
sudo journalctl --rotate
sudo journalctl --vacuum-time=1d
Check the journal files for corruption with:
sudo journalctl --verify
Each file is listed as PASS. A file reported as FAIL, usually after a hard power loss, can be rotated away; journald keeps writing to a new file.
Troubleshooting
--list-bootsshows only the current boot. Storage is still volatile. Check thatStorage=persistentappears insystemd-analyze cat-config systemd/journald.confand that journald was restarted.- The journal uses more space than expected. Other limits are also in play: whichever of
SystemMaxUseandSystemKeepFreeis stricter wins. Confirm withsudo journalctl --disk-usageafter asudo journalctl --rotate. - Entries are missing and the journal shows
Suppressed N messages. journald rate-limits each service, by default 10,000 messages per 30 seconds. Reduce the verbosity of the application, or raise the limit for everything withRateLimitBurst=andRateLimitIntervalSec=in a journald drop-in, or for one service withLogRateLimitBurst=andLogRateLimitIntervalSec=in its unit override. - Without
sudoyou only see your own messages and journalctl prints a hint that you are not seeing messages from other users and the system. Your user is not in thesystemd-journaloradmgroup, or you have not logged in again after being added.
Conclusion
Your journal is now persistent, limited in size and age, and you can find any log line by unit, time, boot, priority, message text or journal field, export it as JSON and clean up space on demand. As next steps, use journalctl -o cat together with grep and awk for deeper text analysis, forward logs from several servers to a central rsyslog server, or combine journalctl -u with systemctl status to monitor and troubleshoot individual services.