Linux Disk Full: Blocks, Inodes and Open Files

When a Linux application reports “No space left on device”, begin with the filesystem containing the failed write. The server may have several mounts, and free space elsewhere may be irrelevant. Deleting the largest visible log is a poor first response if the actual problem is a different volume or exhausted inodes.

Capture the error, affected path and time. Check whether the application is still writing successfully elsewhere and whether monitoring reports a sudden increase or a long-term capacity trend. Preserve useful incident evidence before choosing a cleanup action.

Check blocks and inodes separately

These read-only commands report filesystem usage:

df -hT
df -i

GNU df can report filesystem type and inode usage as well as storage space. Identify the mount containing the affected application path and interpret inode reporting according to that filesystem's design.

A filesystem can have room for data blocks but no capacity for additional file entries under its inode allocation. Large numbers of small cache files, temporary items or failed-job artefacts may therefore matter more than one large file. Check both forms of capacity rather than assuming the percentage shown in the first dashboard is sufficient.

If the figures do not explain the error, check the application's quota and the underlying storage system. A container, thin-provisioned volume or remote filesystem can introduce another capacity boundary. Record which layer reports the failure before attempting a repair at a different layer.

Locate growth within the affected mount

Use a bounded directory summary, starting with the relevant mount or application directory. For example:

sudo du -x -h --max-depth=1 /var

GNU du supports staying on one filesystem and limiting the displayed depth. Replace /var with the area you are investigating. The command reads directory metadata and can take time on a large tree; avoid launching several scans during an already stressed incident.

Follow the unusually large directory one level at a time. Identify its owning service and purpose. A database's transaction logs, a mail queue and an application cache need different recovery procedures even if all three occupy the same amount of space.

For suspected inode exhaustion, compare file counts within the relevant tree using appropriate tooling and permissions. Do not collect millions of filenames into a support ticket. The useful evidence is the responsible directory, rate of growth and process creating the entries.

Explain a mismatch between df and du

If filesystem usage is high but the visible directory totals do not account for it, check for deleted files still held open by a process:

sudo lsof +L1

The lsof manual documents this selection for open files whose link count is below one. A process can retain access to such a file after its directory entry has been removed; its storage may remain allocated until the open references are closed.

Identify the process, file size and affected filesystem before acting. A service-specific log reopen may be available, while another process may require a planned restart. Use the application's supported procedure and assess the interruption. Killing an important process solely because it appears in the output can create a second incident.

Also consider filesystem metadata, snapshots, permissions and mount boundaries when totals differ. A discrepancy is a reason to investigate, not proof that one particular cause must be responsible.

Choose a recovery action with an owner

Once the cause is known, agree the smallest supported action: repair a failing log rotation, expire a documented cache, process a stuck queue, add capacity or correct retention. For a database, follow its recovery documentation and preserve required logs. Never remove unknown database files to create breathing room.

Check whether the service needs temporary write capacity before it can complete its normal cleanup. Record how much space the proposed action should release and how quickly new data is arriving. A cleanup that buys only a few minutes is an interim measure and needs an immediate follow-up plan.

Keep production data and forensic evidence separate from disposable files. If expanding storage, verify the whole supported path from underlying volume to usable filesystem capacity and retain a recovery route.

Verify the service and prevent recurrence

After the controlled action, repeat the relevant capacity checks and perform a normal application operation. Confirm that queued work resumes, errors stop and space remains stable through the next scheduled task.

Add alerts for both capacity and unusual growth, including inode usage where relevant. Record the responsible process, the cause and the corrected retention or rotation setting. The incident is complete when the business service works and the same workload will not simply refill the filesystem tomorrow.

References

• GNU coreutils, df manual, Debian package documentation. Accessed 17 October 2026.
• GNU coreutils, du manual. Accessed 17 October 2026.
• lsof project, lsof manual. Accessed 17 October 2026.