When a filesystem reaches 100%, databases stop writing, services fail to start and even logging in can break. The error is almost always No space left on device, and the cause is usually one of a few things: logs, package caches, container images, backups left on the server, or files that were deleted but are still held open. In this tutorial you will find what is using the space on Ubuntu 24.04, free it safely and set limits so it does not happen again.
Prerequisites
- A server running Ubuntu 24.04 LTS, for example a CubePath VPS. The commands also work on Debian 12.
- A non-root user with
sudoprivileges. - An SSH or console session. If the root filesystem is completely full, some commands may fail to write temporary files; start with Step 4 to free a little space quickly.
Step 1 - Checking which filesystem is full
Show every mounted filesystem with its type, excluding virtual ones:
df -hT -x tmpfs -x devtmpfs -x squashfs -x overlay
Filesystem Type Size Used Avail Use% Mounted on
/dev/vda1 ext4 49G 47G 2.1G 96% /
/dev/vda15 vfat 105M 6.1M 99M 6% /boot/efi
Note the mount point that is full: every search below must stay on that filesystem.
A disk can also be "full" when it runs out of inodes, the entries that describe files, while plenty of space remains. This happens with millions of tiny files such as PHP sessions or mail queues. Check inode usage:
df -i /
Filesystem Inodes IUsed IFree IUse% Mounted on
/dev/vda1 3276800 3276800 0 100% /
If IUse% is at 100%, skip to Step 7.
Step 2 - Finding the largest directories
du measures directory sizes. Start at the root of the full filesystem and go one level deep. -x keeps it on this filesystem, so mounted disks and /proc are ignored:
sudo du -xh --max-depth=1 / 2>/dev/null | sort -h | tail -10
1.2G /usr
2.4G /home
3.1G /opt
38G /var
47G /
Repeat the command on the largest directory until you reach the culprit:
sudo du -xh --max-depth=1 /var 2>/dev/null | sort -h | tail -10
For interactive browsing, ncdu is much faster to work with. It scans once and lets you navigate with the arrow keys, press Enter to open a directory, and q to quit:
sudo apt install ncdu
sudo ncdu -x /
Warning
ncducan delete files with thedkey. Use it to find what is large, and delete deliberately with the commands in the next steps.
Step 3 - Finding the largest files
Sometimes the space is taken by a few huge files: a forgotten database dump, a VM image or a log that was never rotated. List the 20 largest files on the root filesystem:
sudo find / -xdev -type f -size +100M -printf '%s\t%p\n' 2>/dev/null | sort -rn | head -20 | numfmt --field=1 --to=iec
18G /var/log/app/debug.log
6.4G /var/backups/db-2026-06-01.sql
2.1G /home/deploy/release.tar.gz
To find large files that appeared recently, which usually points to what filled the disk suddenly, add a time filter (modified in the last 2 days):
sudo find / -xdev -type f -size +100M -mtime -2 -exec ls -lh {} + 2>/dev/null
Before deleting any file, make sure you know what it is. Move backups you still need off the server first.
Step 4 - Cleaning logs and the systemd journal
Logs are the most common cause. First check how much the systemd journal uses:
journalctl --disk-usage
Archived and active journals take up 3.9G in the file system.
Shrink it to 500 MB, removing the oldest archived entries:
sudo journalctl --vacuum-size=500M
To keep it at that size permanently, create a drop-in configuration:
sudo mkdir -p /etc/systemd/journald.conf.d
sudo nano /etc/systemd/journald.conf.d/size.conf
[Journal]
SystemMaxUse=500M
Apply it:
sudo systemctl restart systemd-journald
For text logs in /var/log, rotated and compressed files (*.gz, *.1) are safe to remove when you do not need the history:
sudo find /var/log -type f \( -name '*.gz' -o -name '*.[0-9]' \) -mtime +14 -delete
A single active log file that has grown huge, like debug.log above, is being written by a running process. Do not delete it: the process keeps writing to the deleted file and the space is not freed (see Step 6). Empty it in place instead:
sudo truncate -s 0 /var/log/app/debug.log
Then fix the cause: lower the application's log level, and make sure the file is covered by a logrotate rule in /etc/logrotate.d/.
Step 5 - Cleaning packages, snaps and Docker
The apt cache keeps every downloaded package. Remove it, along with packages and old kernels that are no longer needed:
sudo apt clean
sudo apt autoremove --purge
Snap keeps older revisions of every snap. List the disabled ones:
snap list --all | awk '/disabled/ {print $1, $3}'
Remove each listed revision with sudo snap remove --revision=<rev> <name>, and reduce how many revisions are kept in the future (2 is the minimum):
sudo snap set system refresh.retain=2
If Docker is installed, it is often the largest consumer. See how its space is split:
docker system df
TYPE TOTAL ACTIVE SIZE RECLAIMABLE
Images 38 6 14.2GB 11.8GB (83%)
Containers 9 6 120MB 2MB (1%)
Local Volumes 12 5 6.3GB 2.9GB (46%)
Build Cache 214 0 4.1GB 4.1GB
Remove stopped containers, unused networks, dangling images and the build cache:
docker system prune
Add -a to also remove every image not used by a container. Volumes are never removed unless you add --volumes, which deletes data of stopped containers, so only use it when you are sure.
Container logs can also grow without limit. Find the largest ones:
sudo du -h /var/lib/docker/containers/*/*-json.log | sort -h | tail -5
To cap them, set "log-opts": {"max-size": "10m", "max-file": "3"} in /etc/docker/daemon.json and restart Docker. The limit only applies to containers created after the change.
Step 6 - Recovering space from deleted open files
If df says the disk is full but du cannot find the space, a process is holding deleted files open. The space is only released when the process closes them. List them:
sudo lsof -nP +L1
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NLINK NODE NAME
java 2231 app 5w REG 252,1 19327352832 0 524301 /var/log/app/server.log (deleted)
The simplest fix is to restart the service that owns the process:
systemctl status 2231
sudo systemctl restart app.service
If it cannot be restarted right now, truncate the deleted file through the process's file descriptor. Use the PID and the number in the FD column (without the letter):
sudo truncate -s 0 /proc/2231/fd/5
Run df -h / again to confirm the space is back.
Step 7 - Fixing inode exhaustion
When inodes run out, find the directories with the most files. du --inodes counts files instead of bytes:
sudo du --inodes -x --max-depth=1 / 2>/dev/null | sort -n | tail -10
Drill down into the largest one the same way as in Step 2. Common causes are PHP session files, mail queues, cache directories of web applications and forgotten node_modules trees.
Remove old files from such a directory in one pass. Check the count first, then delete. This example removes PHP sessions older than one day:
sudo find /var/lib/php/sessions -type f -mtime +1 | wc -l
sudo find /var/lib/php/sessions -type f -mtime +1 -delete
find -delete handles millions of files, unlike rm *, which fails with Argument list too long.
Step 8 - Verifying and preventing the next time
Check the result:
df -h /
df -i /
Restart any service that failed while the disk was full, and check it came back:
systemctl list-units --state=failed
To prevent a repeat, keep the limits you set for the journal and Docker logs, make sure every application log has a logrotate rule, store backups on a separate volume or off the server, and set up monitoring that alerts at 80% usage, not 100%.
Conclusion
You located the full filesystem, found the largest directories and files with du, ncdu and find, cleaned logs, the journal, package caches, snaps and Docker, recovered space held by deleted open files and fixed inode exhaustion. Most full disks come from logs and leftovers, so the limits configured here prevent the majority of repeats. As next steps, set up automated off-server backups and a disk usage alert, and if your data keeps growing legitimately, plan for a larger disk or an additional volume.