Fix a disk that shows full but is not
We'll find the discrepancy between df and du, use lsof to locate processes holding deleted files, free the space by restarting or truncating—or tell you when to escalate.
What you'll need
- SSH or console access with sudo
Step-by-step diagnostic
Quick triage — pick your path
Get started
Choose the option that matches what you see. You can jump straight to that section.
Show full guide
Steps
Goal: Confirm df shows full but du does not, find deleted files held open, and free the space.
- Run
df -hto see which filesystem is full. Rundu -sh /path/to/mount(e.g.du -sh /var) and compare. - Good: df shows 100% but du sums to less—space held by deleted files held open. Proceed to Find deleted files held open.
- Bad: df and du match—see fix-linux-runs-out-of-disk for normal disk-full.
Check df vs du
Goal: Confirm the discrepancy.
- Run
df -hand note the Use% for the full filesystem. Rundu -shon that mount point. - When df shows full but du sums to less, the difference is held by deleted files that processes still have open.
- Good: Discrepancy confirmed. Proceed to Find deleted files held open.
- Bad: No discrepancy—see fix-linux-runs-out-of-disk.
Find deleted files held open
Goal: Identify which processes hold the deleted files.
- Run
lsof +L1to list open files with link count 0 (deleted). Orlsof | grep deleted. - The output shows PID, process name, and file path. Often logs (e.g. /var/log/syslog) or temp files.
- To see size:
ls -l /proc/PID/fd/Nfor the fd number from lsof. - Good: Processes and deleted files identified. Proceed to restart or truncate.
- Bad: No deleted files—run
df -i; if inodes full, see fix-linux-server-has-full-inode.
Restart or truncate
Goal: Free the space by closing the file handle.
- Restart the process:
systemctl restart serviceorkill -HUP PID(if the process supports log rotation). - Or truncate via /proc:
: | sudo tee /proc/PID/fd/N(N from lsof). Restarting is usually safer. - Run
df -hagain. Use% should drop. - Good: Space freed. Configure logrotate to prevent recurrence.
- Bad: Cannot restart safely—escalate with df -h, du, lsof output.
When to escalate
Escalate if:
- You cannot safely restart the process.
- The process is critical and restart would cause unacceptable downtime.
- You need to expand the filesystem.
Provide df -h, df -i, du output, and lsof +L1 output.
Verification
df -hshows increased free space on the affected filesystem.du -shon the mount now matches df more closely.- The process that held the deleted file is running (if restarted) and logging correctly.
- Logrotate or app config is updated to prevent recurrence.
Escalation ladder
Work from the device outward. Stop when the problem is fixed.
- Confirm df vs du Run df -h and du -sh to confirm the discrepancy.
- Find deleted files Use lsof +L1 or lsof | grep deleted to find processes holding deleted files.
- Restart or truncate Restart the process or truncate the file to free space.
- Check inodes Run df -i; if inodes full, see fix-linux-server-has-full-inode.
- Escalate Provide df -h, df -i, du, lsof +L1 output; consider expand or add storage.
What to capture if you need help
Before calling support or posting for help, have these ready. It speeds everything up.
- df -h output
- df -i output
- du -sh output for affected mount
- lsof +L1 or lsof | grep deleted output
- Steps already tried
Does df show full but du shows less?
Run df -h and du -sh on the full mount. A discrepancy means deleted files are held open.
You can change your answer later.
Find processes holding deleted files
lsof +L1 or lsof | grep deleted lists open deleted files with PID.
You can change your answer later.
Restart process or truncate file
Normal disk full
Escalate
Reviewed by Blackbox Atlas
Frequently asked questions
- Why does df show full but du does not?
- A process has a file open that was deleted. The file no longer has a directory entry (so du cannot see it) but the process holds the inode, so the space is not freed until the process closes the file.
- How do I find which process holds the deleted file?
- Run lsof +L1 (files with link count 0) or lsof | grep deleted. The output shows the PID and path. Restart the process or truncate the file to free space.
- When should I escalate?
- If you cannot safely restart the process, the process is critical and restart would cause downtime, or you need to expand the filesystem.
Rate this guide
Was this helpful?
Thanks for your feedback.