systemd is the init system and service manager on Ubuntu, Debian, Rocky Linux and most other distributions. It describes everything it manages (services, sockets, timers, mount points, targets) as declarative unit files and starts them according to their dependencies. In this tutorial you will build a small hardened service on Ubuntu 24.04 (systemd 255) and use it to learn how unit files, dependencies, targets, drop-in overrides, socket activation, resource limits, timers and the journal work.

Prerequisites

To follow this tutorial you need:

  • A server running Ubuntu 24.04 LTS, for example a CubePath VPS. The commands also work on Debian 12 and Rocky Linux 9.
  • A non-root user with sudo privileges.
  • Python 3, which Ubuntu 24.04 includes by default, for the example service.

Unit types and where they live

Every unit has a type given by its file extension:

TypeExtensionManages
Service.serviceDaemons and one-shot commands
Socket.socketListening sockets that start a service on demand
Target.targetGroups of units, used as synchronization points
Timer.timerScheduled activation of another unit
Mount.mountFilesystem mount points
Path.pathActivation when a file or directory changes
Slice.slicecgroup groups for resource control

systemd reads unit files from several directories. When the same name exists in more than one, the first in this list wins:

DirectoryOwner
/etc/systemd/system/The administrator (your custom units and overrides)
/run/systemd/system/Runtime units, lost on reboot
/usr/lib/systemd/system/Units installed by packages; do not edit

Step 1 - Inspecting existing units

Before writing your own unit, look at how installed ones are defined. List enabled unit files:

systemctl list-unit-files --type=service --state=enabled

Show the file for a unit, including any drop-in overrides, with systemctl cat:

systemctl cat ssh.service
# /usr/lib/systemd/system/ssh.service
[Unit]
Description=OpenBSD Secure Shell server
Documentation=man:sshd(8) man:sshd_config(5)
After=network.target nss-user-lookup.target auditd.service
...

To see the effective value of any property, including defaults that are not in the file, use systemctl show:

systemctl show ssh.service -p Restart -p MainPID -p ActiveState
Restart=on-failure
MainPID=880
ActiveState=active

Step 2 - Writing a hardened service unit

Create a small web application to manage. It serves a directory over HTTP on port 8080 using Python's built-in server. First create a dedicated system user and the content directory:

sudo useradd --system --no-create-home --shell /usr/sbin/nologin demoapp
sudo mkdir -p /srv/demoapp
echo "hello from systemd" | sudo tee /srv/demoapp/index.html

Create the unit file:

sudo nano /etc/systemd/system/demoapp.service
[Unit]
Description=Demo HTTP application
Documentation=https://docs.python.org/3/library/http.server.html
Wants=network-online.target
After=network-online.target

[Service]
Type=exec
User=demoapp
Group=demoapp
WorkingDirectory=/srv/demoapp
Environment=PYTHONUNBUFFERED=1
ExecStart=/usr/bin/python3 -m http.server 8080 --bind 127.0.0.1 --directory /srv/demoapp
Restart=on-failure
RestartSec=5s

# Hardening
NoNewPrivileges=yes
ProtectSystem=strict
ProtectHome=yes
PrivateTmp=yes
PrivateDevices=yes
ProtectKernelTunables=yes
ProtectKernelModules=yes
ProtectControlGroups=yes
RestrictAddressFamilies=AF_INET AF_INET6 AF_UNIX

[Install]
WantedBy=multi-user.target

The sections have distinct roles:

  • [Unit] holds metadata and dependencies. Wants=network-online.target plus After= waits until the network is configured, not just started.
  • [Service] defines how to run the process. Type=exec tells systemd the service is up once the binary has been executed successfully, and Restart=on-failure restarts it after a crash but not after a clean stop.
  • The hardening options make the whole filesystem read-only for the service (ProtectSystem=strict), hide /home, and give it a private /tmp.
  • [Install] is only used by systemctl enable: it creates a link so multi-user.target pulls in the service at boot.

Reload systemd so it reads the new file, then enable and start the service:

sudo systemctl daemon-reload
sudo systemctl enable --now demoapp.service

Verify that it runs and answers:

systemctl status demoapp.service --no-pager
curl -s http://127.0.0.1:8080/
● demoapp.service - Demo HTTP application
     Loaded: loaded (/etc/systemd/system/demoapp.service; enabled; preset: enabled)
     Active: active (running) since Fri 2026-09-25 10:20:11 UTC; 3s ago
   Main PID: 2210 (python3)
hello from systemd

systemd can score how exposed a service is. Compare your unit with an unhardened one:

systemd-analyze security demoapp.service | tail -n 1
→ Overall exposure level for demoapp.service: 4.6 OK

Lower is better; a service with no hardening scores around 9.6 (UNSAFE). Your exact number depends on the options you set.

Step 3 - Understanding dependencies and ordering

systemd separates two questions: which units must be started together (requirement dependencies), and in which order (ordering dependencies). Setting one does not imply the other.

DirectiveEffect
Wants=Start the listed unit too; keep going if it fails. The recommended default
Requires=Start the listed unit too; if it fails to start, or is stopped explicitly, this unit is stopped
BindsTo=Like Requires=, and also stop this unit if the other one stops for any reason
PartOf=Stop or restart this unit when the listed unit is stopped or restarted
After= / Before=Ordering only: wait for the listed unit to finish starting
Conflicts=Starting this unit stops the listed one, and the reverse

A common mistake is Requires=mysql.service without After=mysql.service: both units start in parallel and the application may connect before the database is ready. Always pair a requirement with an ordering directive.

See what a unit pulls in:

systemctl list-dependencies demoapp.service

And which units depend on a given one, for example which services need the network online:

systemctl list-dependencies --reverse network-online.target

To analyze boot time, critical-chain shows the chain of units that delayed a target, and blame lists units by start time:

systemd-analyze critical-chain demoapp.service
systemd-analyze blame | head -n 5
demoapp.service +45ms
└─network-online.target @3.912s
  └─systemd-networkd-wait-online.service @1.203s +2.707s
    └─systemd-networkd.service @1.050s +140ms

The @ value is when the unit became active and + is how long it took to start.

Step 4 - Changing units safely with drop-in overrides

Never edit files in /usr/lib/systemd/system/: package upgrades overwrite them. Use a drop-in file instead, which systemd merges on top of the original. systemctl edit creates it for you:

sudo systemctl edit demoapp.service

The editor opens a file at /etc/systemd/system/demoapp.service.d/override.conf. Add only the settings you want to change between the comment markers:

[Service]
Environment=APP_ENV=production
RestartSec=10s

Save and exit; systemctl edit reloads the configuration automatically. Restart the service and check the merged result:

sudo systemctl restart demoapp.service
systemctl cat demoapp.service | tail -n 4
systemctl show demoapp.service -p RestartUSec -p Environment
RestartUSec=10s
Environment=PYTHONUNBUFFERED=1 APP_ENV=production

List-type settings such as ExecStart= are appended rather than replaced. To replace one, clear it first with an empty assignment:

[Service]
ExecStart=
ExecStart=/usr/bin/python3 -m http.server 9090 --bind 127.0.0.1 --directory /srv/demoapp

To remove all overrides, run sudo systemctl revert demoapp.service.

Step 5 - Working with targets

Targets group units and replace SysV runlevels. multi-user.target is a normal server without a graphical session, graphical.target adds a display manager, and rescue.target is single-user maintenance mode. Check the default:

systemctl get-default
multi-user.target

Servers should boot into multi-user.target. If a server was installed with a desktop and you want to stop starting it:

sudo systemctl set-default multi-user.target

systemctl isolate switches the running system to another target and stops everything that target does not include.

List active targets to see what is reached at the moment:

systemctl list-units --type=target

Step 6 - Starting services on demand with socket activation

With socket activation, systemd opens the listening socket itself and starts the service only when the first connection arrives. It also lets you restart the service without dropping connections waiting in the queue. The simplest example is an echo service where systemd starts one process per connection (Accept=yes) and connects the socket to its standard input and output.

Create the socket unit:

sudo nano /etc/systemd/system/echo.socket
[Unit]
Description=Echo service socket

[Socket]
ListenStream=127.0.0.1:7777
Accept=yes

[Install]
WantedBy=sockets.target

With Accept=yes the service must be a template (note the @), because systemd creates one instance per connection:

sudo nano /etc/systemd/system/[email protected]
[Unit]
Description=Echo service connection

[Service]
ExecStart=/usr/bin/cat
StandardInput=socket
DynamicUser=yes

Enable only the socket; the service starts by itself:

sudo systemctl daemon-reload
sudo systemctl enable --now echo.socket
systemctl list-sockets echo.socket

The output lists 127.0.0.1:7777 with echo.socket as its unit. No echo@ service is running yet.

Connect and type a line; it is echoed back. Press Ctrl+C to close:

nc 127.0.0.1 7777

For long-running daemons, use Accept=no (the default). systemd then passes the listening socket to a single service instance, which must support receiving it (for example through sd_listen_fds()).

Step 7 - Limiting resources

Every service runs in its own cgroup, so you can limit it directly in the unit. Add limits to the demo service with a drop-in:

sudo systemctl edit demoapp.service
[Service]
CPUQuota=50%
MemoryHigh=200M
MemoryMax=256M
TasksMax=64
LimitNOFILE=4096

CPUQuota=50% allows half of one CPU. MemoryHigh throttles and reclaims memory above 200 MB, and MemoryMax is the hard limit that triggers the OOM killer. TasksMax caps processes plus threads. Restart and verify:

sudo systemctl restart demoapp.service
systemctl show demoapp.service -p CPUQuotaPerSecUSec -p MemoryMax -p TasksMax
CPUQuotaPerSecUSec=500ms
MemoryMax=268435456
TasksMax=64

To change a limit on a running service without restarting it, use set-property. The --runtime flag makes the change temporary:

sudo systemctl set-property --runtime demoapp.service MemoryMax=300M

Watch live usage per cgroup with systemd-cgtop (press q to quit).

Step 8 - Replacing cron with a timer

Timers run a service on a schedule, log to the journal and can catch up on runs missed while the server was off. Create a service that removes files older than seven days from a temporary directory:

sudo mkdir -p /srv/demoapp-tmp
sudo nano /etc/systemd/system/cleanup-demoapp.service
[Unit]
Description=Remove old files from /srv/demoapp-tmp

[Service]
Type=oneshot
ExecStart=/usr/bin/find /srv/demoapp-tmp -type f -mtime +7 -delete

Create the timer with the same name:

sudo nano /etc/systemd/system/cleanup-demoapp.timer
[Unit]
Description=Daily cleanup of /srv/demoapp-tmp

[Timer]
OnCalendar=*-*-* 03:00:00
RandomizedDelaySec=10m
Persistent=true

[Install]
WantedBy=timers.target

Persistent=true runs the job at boot if the 03:00 run was missed. Check the calendar expression before enabling the timer:

systemd-analyze calendar "*-*-* 03:00:00"
  Original form: *-*-* 03:00:00
Normalized form: *-*-* 03:00:00
    Next elapse: Sat 2026-09-26 03:00:00 UTC
       From now: 16h left

Enable the timer (not the service) and list it:

sudo systemctl daemon-reload
sudo systemctl enable --now cleanup-demoapp.timer
systemctl list-timers cleanup-demoapp.timer

Run the job once by hand to test it: sudo systemctl start cleanup-demoapp.service.

Step 9 - Reading logs with journalctl

Everything a service writes to stdout and stderr ends up in the journal. Ubuntu 24.04 stores it persistently in /var/log/journal. The most useful filters:

journalctl -u demoapp.service -n 50 --no-pager

Follow new entries in real time:

journalctl -u demoapp.service -f

Only messages from the current boot, or from the previous one (useful after a crash):

journalctl -b -u demoapp.service
journalctl -b -1 -p err

-p err shows priority err and higher (crit, alert, emerg). Filter by time and export JSON for processing:

journalctl -u demoapp.service --since "1 hour ago" -o json-pretty | head -n 20

Check how much disk the journal uses and trim it if needed:

journalctl --disk-usage
sudo journalctl --vacuum-time=30d

Troubleshooting

status=203/EXEC. systemd could not execute ExecStart. The path must be absolute and the file executable; check with ls -l /path/to/binary. With ProtectSystem=strict or ProtectHome=yes, the binary may be in a hidden or read-only location.

status=217/USER. The user in User= does not exist. Create it or fix the name.

Changes to a unit file are ignored. Run sudo systemctl daemon-reload after editing any unit file by hand. systemctl status warns when the file on disk changed and needs a reload.

The service starts before the network is ready. Use both Wants=network-online.target and After=network-online.target. network.target only means the network management stack has started, not that interfaces have addresses.

A hardened service cannot write its data. ProtectSystem=strict makes everything read-only. Grant specific paths with ReadWritePaths=/var/lib/demoapp, or let systemd create one with StateDirectory=demoapp.

Conclusion

You wrote and hardened a service unit, learned how requirement and ordering dependencies differ, changed units safely with drop-ins, switched targets, used socket activation, applied cgroup resource limits, replaced a cron job with a timer and filtered the journal. Next, explore systemd-analyze security recommendations for your real services, read man systemd.exec and man systemd.resource-control for every available option, and move your remaining cron jobs to timers.