A Linux filesystem can fail with No space left on device for two different reasons: it ran out of data blocks, or it ran out of inodes (the metadata entries that every file and directory needs). In this guide you will check both on Ubuntu 24.04, track down the directories and files responsible, free space safely, and set up a small systemd timer that warns you before a filesystem fills up.

Prerequisites

  • A server running Ubuntu 24.04 LTS, such as a CubePath VPS. The commands also work on Debian 12; on Rocky Linux 9 replace apt with dnf.
  • A non-root user with sudo privileges.

Step 1 - Checking disk space with df

df reports how much space each mounted filesystem has. The -h flag prints human-readable sizes and -T adds the filesystem type. The -x flags hide pseudo-filesystems that only live in memory and would clutter the output:

df -hT -x tmpfs -x devtmpfs -x squashfs -x efivarfs

Filesystem     Type  Size  Used Avail Use% Mounted on
/dev/vda1      ext4   78G   31G   44G  42% /
/dev/vda15     vfat  105M  6.1M   99M   6% /boot/efi

The Use% column is the one to watch. Notice that Used plus Avail is smaller than Size: ext4 reserves 5% of the blocks for root by default, so regular users (and most services) hit "disk full" before df reaches 100%. You can see the reserve with tune2fs:

sudo tune2fs -l /dev/vda1 | grep -i 'reserved block count'

Reserved block count:     1036800

To check a single path, pass it to df. It shows the filesystem that contains that path:

df -h /var/lib

Step 2 - Checking inode usage with df -i

Each file, directory and symlink uses one inode. On ext4 the number of inodes is fixed when the filesystem is created, so millions of tiny files (session files, cache entries, mail queues) can exhaust inodes while plenty of space is still free. Check inodes with -i:

df -hi -x tmpfs -x devtmpfs -x squashfs -x efivarfs

Filesystem     Inodes IUsed IFree IUse% Mounted on
/dev/vda1        5.0M  312K  4.7M    7% /
/dev/vda15          0     0     0     - /boot/efi

A - in IUse% means the filesystem does not use a fixed inode table (vfat, Btrfs). XFS allocates inodes dynamically too, so inode exhaustion is mostly an ext4 problem.

If IUse% is high while Use% is low, skip to Step 4.

Step 3 - Finding large directories and files

Once you know which filesystem is full, find where the space went. du summarizes directory sizes. The -x flag stays on one filesystem (so / does not include /proc or other mounts) and -d 1 limits the output to one level:

sudo du -xh -d 1 / 2>/dev/null | sort -rh | head -n 10

31G     /
18G     /var
7.2G    /usr
3.9G    /home
1.1G    /opt

Repeat the command on the biggest entry to drill down, for example sudo du -xh -d 1 /var | sort -rh | head -n 10.

To list individual files larger than 500 MB on the root filesystem:

sudo find / -xdev -type f -size +500M -exec du -h {} + 2>/dev/null | sort -rh | head -n 20

9.8G    /var/lib/mysql/ibdata1
2.1G    /var/log/app/debug.log

Browsing interactively with ncdu

For exploring a full disk, ncdu is faster than repeating du by hand. It scans once and lets you navigate with the arrow keys:

sudo apt update
sudo apt install ncdu

sudo ncdu -x /

Use the arrow keys and Enter to move through directories, d to delete the selected item (after confirmation) and q to quit. Be careful with d: deleting files that belong to a package or a running service can break it.

Step 4 - Finding directories with too many files

When inodes are the problem, you need the directories with the most entries rather than the biggest ones. GNU du can count inodes instead of bytes:

sudo du -x --inodes -d 1 / 2>/dev/null | sort -rn | head -n 10

311904  /
196551  /var
84230   /usr
21480   /home

Drill down the same way until you find the culprit:

sudo du -x --inodes -d 1 /var 2>/dev/null | sort -rn | head -n 5

196551  /var
171220  /var/lib
18702   /var/cache

Typical offenders are PHP session directories, application cache directories, and mail queues filled by a misbehaving cron job. Deleting a directory with millions of files is faster with find than with rm -r on a glob, which can fail with Argument list too long:

sudo find /path/to/cache -type f -mtime +7 -delete

This removes files not modified in the last seven days. Replace the path with the directory you identified, and check that the application can rebuild those files before you run it.

Step 5 - Freeing space safely

Start with the places that grow on every server and that are safe to trim.

Remove packages that are no longer needed and clear the APT package cache:

sudo apt autoremove --purge
sudo apt clean

Check how much space the systemd journal uses:

journalctl --disk-usage

Archived and active journals take up 1.8G in the file system.

Shrink it to 500 MB immediately:

sudo journalctl --vacuum-size=500M

To keep it at that size permanently, create a drop-in configuration file:

sudo mkdir -p /etc/systemd/journald.conf.d
sudo nano /etc/systemd/journald.conf.d/size.conf

[Journal]
SystemMaxUse=500M

Restart journald to apply it:

sudo systemctl restart systemd-journald

If a single application log is huge, fix its log rotation rather than deleting the file. Files under /var/log are rotated by logrotate, configured in /etc/logrotate.d/. You can test a configuration without changing anything with sudo logrotate -d /etc/logrotate.conf.

Step 6 - Getting alerts before a disk fills up

Checking by hand only helps when you remember to do it. A short script run by a systemd timer can check every filesystem each hour and log a warning to the journal when block or inode usage crosses a threshold.

Create the script:

sudo nano /usr/local/bin/check-disk-usage

#!/usr/bin/env bash
# Warn when block or inode usage on any real filesystem reaches the threshold.
set -euo pipefail

threshold="${1:-85}"
status=0

while read -r pcent ipcent target; do
    used="${pcent%\%}"
    iused="${ipcent%\%}"
    [[ "$used" =~ ^[0-9]+$ ]] || used=0
    [[ "$iused" =~ ^[0-9]+$ ]] || iused=0

    if (( used >= threshold || iused >= threshold )); then
        logger -t check-disk-usage -p user.warning \
            "${target}: space ${used}%, inodes ${iused}% (threshold ${threshold}%)"
        echo "${target}: space ${used}%, inodes ${iused}%" >&2
        status=1
    fi
done < <(df --output=pcent,ipcent,target -x tmpfs -x devtmpfs -x squashfs -x efivarfs -x overlay | tail -n +2)

exit "$status"

The script reads df output directly, handles filesystems without inodes, and exits with status 1 when any filesystem is over the limit, so the systemd unit is marked as failed and shows up in systemctl --failed. Make it executable and test it with a low threshold so that it triggers:

sudo chmod 755 /usr/local/bin/check-disk-usage
sudo /usr/local/bin/check-disk-usage 10

/: space 42%, inodes 7%

Create a service unit that runs the check:

sudo nano /etc/systemd/system/check-disk-usage.service

[Unit]
Description=Check disk space and inode usage

[Service]
Type=oneshot
ExecStart=/usr/local/bin/check-disk-usage 85

Create a timer that runs it every hour:

sudo nano /etc/systemd/system/check-disk-usage.timer

[Unit]
Description=Hourly disk space and inode check

[Timer]
OnCalendar=hourly
Persistent=true

[Install]
WantedBy=timers.target

Enable the timer and run the check once to confirm it works:

sudo systemctl daemon-reload
sudo systemctl enable --now check-disk-usage.timer
sudo systemctl start check-disk-usage.service
systemctl list-timers check-disk-usage.timer

NEXT                        LEFT       LAST PASSED UNIT                   ACTIVATES
Thu 2026-09-24 15:00:00 UTC 41min      -    -      check-disk-usage.timer check-disk-usage.service

Warnings are written to the journal, where you can read them with:

journalctl -t check-disk-usage --since today

To get notified by email or chat, add an OnFailure= unit to the service, or let your existing monitoring (Netdata, Prometheus node_exporter, Zabbix) alert on the same thresholds.

Troubleshooting

df shows the disk is full but du finds much less

A process is probably still holding a deleted file open. The space is only released when the process closes it. List deleted-but-open files:

sudo lsof -nP +L1

COMMAND  PID  USER  FD  TYPE DEVICE   SIZE/OFF NLINK   NODE NAME
java    1432  app   5w   REG  252,1 8589934592     0 131091 /var/log/app/app.log (deleted)

Restart the service that owns the process (sudo systemctl restart your_service) to release the space. If lsof is not installed, install it with sudo apt install lsof.

Space is hidden under a mount point

Files written to a directory before a filesystem was mounted on it are hidden, but still use space. Bind-mount the root filesystem elsewhere to look underneath:

sudo mkdir -p /mnt/rootfs
sudo mount --bind / /mnt/rootfs
sudo du -xh -d 1 /mnt/rootfs/mnt /mnt/rootfs/var/lib 2>/dev/null
sudo umount /mnt/rootfs

No space left on device, but df -h shows free space

The filesystem is out of inodes. Confirm with df -i and follow Step 4.

Conclusion

You can now check block and inode usage with df, locate the directories and files responsible with du, find and ncdu, free space from the usual sources, and receive a journal warning from a systemd timer before a filesystem fills up. As next steps, configure log rotation for your own applications in /etc/logrotate.d/, put fast-growing data such as databases or backups on a separate volume, and feed disk metrics into a monitoring system to see growth trends over time.