System Disk is full

  • So I have OMV installed on a 128gb SSD. It is full and I can't figure out why. Tried clearing logs, but that was only 25mb. I tried 'apt autoremove' and 'apt clean' and 'du -sh /'. No luck figuring out what is filling the drive. I cleared an old kernel (only 3 of them were installed). Anyone here have any ideas on where I can look?


    The result from du -sh /* in case it helps.



    Any help would be greatly appreciated.

  • Normally this is caused by a not available filesystem (incl. shared folders) which is the target of a backup/rsync/whatever job. So you might concentrate on looking into the /srv directory where all filesystems are mounted. If a filesystem is not mounted and a rsync job or something else is writing to a shared folder the data is written into thr root filesystem instead.

  • Normally this is caused by a not available filesystem (incl. shared folders) which is the target of a backup/rsync/whatever job. So you might concentrate on looking into the /srv directory where all filesystems are mounted. If a filesystem is not mounted and a rsync job or something else is writing to a shared folder the data is written into thr root filesystem instead.

    I don't think I have RSync enabled. Is that something that is on by default?

  • And to make matters worse. The junk files may be hidden...


    If some job (or you), at one time, wrote a bunch of backup files, or whatever, to a mount point that wasn't mounted, it is possible that the mount point in question now is mounted. then the mounted device "mask" the local filesystem there. You can only see the mounted filesystem, not the local filesystem.


    So du can't report correct disk usage.


    One way to find out is to boot from other media (a thumb drive?) and examine the full root filesystem when it is not active. Then no mount points are "masked" and du can be expected to show correct local disk usage.


    Another common cause for full root fs is that /var/lib is filled with metadata and other stuff from plex/emby or docker. But that seems not to be the case here.

    Be smart - be lazy. Clone your rootfs.
    OMV 5: 9 x Odroid HC2 + 1 x Odroid HC1 + 1 x Raspberry Pi 4

  • Your du command should have to the -x flag to not traverse other filesystems. That way, you will still see usage in /srv even in filesystems are mounted on the subdirectories.


    sudo du -x -d1 -h / | sort -h

    omv 5.6.12 usul | 64 bit | 5.11 proxmox kernel | omvextrasorg 5.6.2 | kvm plugin 5.1.6
    omv-extras.org plugins source code and issue tracker - github


    Please read this before posting a question.
    Please don't PM for support... Too many PMs!

  • That way, you will still see usage in /srv even in filesystems are mounted on the subdirectories.

    I think you do not see usage of other filesystems mounted in /srv (or somewhere else).
    But you will see usage in /srv of folders that are not on other filesystems.

  • I think you do not see usage of other filesystems mounted in /srv (or somewhere else).
    But you will see usage in /srv of folders that are not on other filesystems.

    Correct but that was the idea. /srv/ shouldn't be large because it just has mountpoints for other filesystems or bind mounts in it. If it is large, you probably had something write to a sub-directory while the sub-directory wasn't mounted.

    omv 5.6.12 usul | 64 bit | 5.11 proxmox kernel | omvextrasorg 5.6.2 | kvm plugin 5.1.6
    omv-extras.org plugins source code and issue tracker - github


    Please read this before posting a question.
    Please don't PM for support... Too many PMs!

  • I also have thio problem today as my system disk is full (using urbackup but files are being saved on another hard disk).


    So after I enter: sudo du -sh /*


    I get 71G in the /srv. Reading this forum I think I understand that for some reason my sharedfolder was down so it began to write the backup into the /srv folder?


    if so, is it safe to delete everything within srv folder relating to non system stuff?

  • This is the output from df -h, if someone can kindly point me in the right direction to what to delete (and how)?


    Filesystem Size Used Avail Use% Mounted on
    udev 1.9G 0 1.9G 0% /dev
    tmpfs 388M 7.2M 381M 2% /run
    /dev/sda1 31G 31G 0 100% /
    tmpfs 1.9G 40K 1.9G 1% /dev/shm
    tmpfs 5.0M 0 5.0M 0% /run/lock
    tmpfs 1.9G 0 1.9G 0% /sys/fs/cgroup
    tmpfs 1.9G 0 1.9G 0% /tmp
    folder2ram 1.9G 63M 1.9G 4% /var/log
    /dev/sda3 39G 48M 37G 1% /srv/dev-disk-by-id-ata-ST980813AS_5NH1G0Y G-part3
    folder2ram 1.9G 0 1.9G 0% /var/tmp
    folder2ram 1.9G 852K 1.9G 1% /var/lib/openmediavault/rrd
    folder2ram 1.9G 20K 1.9G 1% /var/spool
    folder2ram 1.9G 13M 1.9G 1% /var/lib/rrdcached
    folder2ram 1.9G 4.0K 1.9G 1% /var/lib/monit
    folder2ram 1.9G 0 1.9G 0% /var/lib/php
    folder2ram 1.9G 1.3M 1.9G 1% /var/cache/samba
    overlay 31G 31G 0 100% /var/lib/docker/overlay2/c27b39a4b79b6318a ca7497baedd2bedb3b0880d87dfb982eab05dd412e2b9f3/merged
    overlay 31G 31G 0 100% /var/lib/docker/overlay2/4270ed88df78570c8 d71b8a14b7cb4e8b6dedc4954153c0784375b64d14ba47a/merged
    shm 64M 0 64M 0% /var/lib/docker/containers/d624c1fdcbbdb20 7c8db33c154f799ca65bac4d05c83890e86d0664e97ee81b3/mounts/shm

  • df is not interesting. You know there is 0% free. That is true for all of sda1.



    du is more interesting.


    Try:


    sudo du -x --max-depth=1 -h /


    Then you will get a list of all the folders directly in / and how much space they take up.


    Run the command again but replace / with the biggest folder found. Then you will find the problem.


    Feel free to post a list of the biggest 3-4 folders you find.


    However, see my earlier post. The problem may be hidden...


    Edit: It seems some docker image is very big. I assume that you will find it under /var/lib/docker.


    You should NOT use /var/lib/docker for docker images, if /var/lib/docker is on the rootfs.


    There are many threads on this forum about /var/lib/docker and docker base path and how to fix it. I would reinstall from scratch and set docker base path before I add any docker images. Use a shared folder on a storage drive. Again, there are many threads here about this.

    Be smart - be lazy. Clone your rootfs.
    OMV 5: 9 x Odroid HC2 + 1 x Odroid HC1 + 1 x Raspberry Pi 4

    Edited once, last by Adoby ().

  • thanks adoby. this is how far I get its all in /srv i think but I cant see it. i beginning to think i need to start all over again.


    root@omv:~# sudo du -x --max-depth=1 -h /srv
    8.0K /srv/ftp
    6.0G /srv/dev-disk-by-label-Panda
    32K /srv/pillar
    18G /srv/dev-disk-by-label-Elephant
    48K /srv/dev-disk-by-label-Vulture
    775M /srv/dev-disk-by-id-ada-ST980813AS_5NH1G0YG-part3
    1.6M /srv/salt
    24G /srv



    EDIT: Not sure I see anything in /var/lib/docker - it totals about 4.5G and there is plenty of room on the system disk but your point about having it off the system disk is noted!

  • it has 24GB when it shouldn't have anything large?

    Correct. Are you sure that the two drives/filesystems are not connected. Just to make sure we are not deleting something wrong. Otherwise you have to restore from your backup

  • Not sure I follow but in any event I can't log into OMV control panel because its full. Get this error message:


    Error #0:
    OMV\Exception: Failed to read file '/var/cache/openmediavault/cache.omv\controlpanel\login_js.json' (size=0). in /usr/share/php/openmediavault/json/file.inc:207
    Stack trace:
    #0 /usr/share/php/openmediavault/json/file.inc(223): OMV\Json\File->getContents()
    #1 /usr/share/php/openmediavault/controlpanel/controlpanelabstract.inc(173): OMV\Json\File->read()
    #2 /var/www/openmediavault/index.php(46): OMV\ControlPanel\ControlPanelAbstract->render()
    #3 {main}




    I don't need the backup on Elephant (which is completely separate from system disk) as I'm only testing urbackup with that drive. I just want to get back into OMV control panel, so does this mean a complete reinstall of OMV?

  • Is Elephant currently connected to your server? From the output above it looks like it is not connected. What I do not understand is the information of G-part3 ?(


    If you are sure you don't need the data on it, you can delete it with


    rm -r /srv/dev-disk-by-label-Elephant/*


    rm: remove
    -r: recursive
    the rest is the path to delete

  • tannaroo, as I mentioned in the other urbackup Newbie thread, I had this same login problem. I took the following steps, based on the suggestions above to solve this. there may be easier ways of doing this, but this worked for me.


    You need to boot from a different linux disk, (I booted from a linux install on a usb stick), and then mount the systemdisk, for me the original /dev/sda2 . I was then able to look around on my disk using du -h -x -d1 as suggested above. This allowed me to discover a directory in one of my sharedfolders mount points which had a lot of data, 127GB, that should not have been there. Because you do not have your other share disks mounted, the mount points in sharedfolders should have no files. When the disk was running normally those files were there taking up space, but hidden by the mounted share. I'm currently moving the files somewhere else and can see my original system disk getting less full as I move them.

  • No I disconnected Elephant from the server but followed that command and deleted it. The directory /srv is now showing 6.8GB rather than 24GB, so its worked - I think?


    Regarding G-Part 3 - its a 60gb drive of which one is a partition of 30gb (the other partition being OMV system for 30gb). I wanted to use a partition for things like app data etc.


    So i've cleared my cache but still cant get into OMV portal - still showing same error ?

  • tannaroo, as I mentioned in the other urbackup Newbie thread, I had this same login problem. I took the following steps, based on the suggestions above to solve this. there may be easier ways of doing this, but this worked for me.


    You need to boot from a different linux disk, (I booted from a linux install on a usb stick), and then mount the systemdisk, for me the original /dev/sda2 . I was then able to look around on my disk using du -h -x -d1 as suggested above. This allowed me to discover a directory in one of my sharedfolders mount points which had a lot of data, 127GB, that should not have been there. Because you do not have your other share disks mounted, the mount points in sharedfolders should have no files. When the disk was running normally those files were there taking up space, but hidden by the mounted share. I'm currently moving the files somewhere else and can see my original system disk getting less full as I move them.

    thanks ian6 per my other post. i just deleted the whole drive and now my system disk seems to be back under 10gb. but rebooted etc but still can't get into the OMV portal.

Participate now!

Don’t have an account yet? Register yourself now and be a part of our community!