System disk is full, but it really isn't. 1 TB hdd actually has approx 15 gb

    • System disk is full, but it really isn't. 1 TB hdd actually has approx 15 gb

      It's been three years on OMV, and this is the first time I've had to create my own thread---Google has failed me.

      Running OMV 3.0 on a 1 TB HDD. I'm unable to login with the WEB GUI, but SSH is fine. All services are working without any noticeable lag (Plex, Shell in a box, Radarr, SABNZBD). I'm certain there isn't a huge log, or any large file for that matter that's the culprit. All in all, I'm not using more than 10-20 GB on a 1 TB HDD. Root directory, "/", is at 100% capacity
      du -hx --max-depth=1 --exclude=proc

      du: cannot access ‘./proc/10373/fd/4’: No such file or directory
      du: cannot access ‘./proc/10373/fdinfo/4’: No such file or directory
      du: cannot access ‘./proc/18467’: No such file or directory
      13027695972 .
      12975465008 ./srv
      5604787056 ./srv/dev-disk-by-label-8tbXeonVault
      3167284320 ./srv/dev-disk-by-label-8tbXeonVault/MOVIES
      2236724512 ./srv/dev-disk-by-label-8tbXeonVault/TELEVISION
      1917482908 ./srv/dev-disk-by-label-PARITYdisk1
      1917482888 ./srv/dev-disk-by-label-PARITYdisk1/snapraid.parity
      1839466948 ./srv/dev-disk-by-label-BUdisk2
      1796074224 ./srv/dev-disk-by-label-BUdisk1
      1767773268 ./srv/dev-disk-by-label-BUdisk3
      1741663196 ./srv/dev-disk-by-label-BUdisk2/BACKUP_TELEVISION
      1704735924 ./srv/dev-disk-by-label-BUdisk3/UFS_TEST_BACKUP_MOVIES
      1365065088 ./srv/dev-disk-by-label-BUdisk1/UFS_TEST_BACKUP_MOVIES
      430690492 ./srv/dev-disk-by-label-BUdisk1/BACKUP_TELEVISION

      du -hx --max-depth=1
      8.0K ./media
      16M ./sbin
      9.7M ./bin
      557M ./lib
      257M ./opt
      8.6M ./etc
      713M ./var
      4.0K ./mnt
      44K ./root
      998M ./boot
      20K ./srv
      3.1G ./home
      1.5G ./usr
      16K ./lost+found
      4.0K ./lib64
      4.0K ./export

      7.0G .

      I've seen Ryan mention failed rsync jobs as a possible culprit, but i'm unable to go much further than that. Rsync seems likely to be involved---every time I reboot, I see my backup array spinning up, without executing a job.

      Any help would be greatly appreciated. Thanks :S

      The post was edited 1 time, last by Goobs ().

    • My install didn't have NCDU pre-installed so grabbed the tar. Still, nothing obvious is hogging 900+ GB. I've pasted the initial ncdu results. The largest being media in /srv which is part of a RAID 5 array--anything in /export is also a symlink to the array. /proc, /sys and /temp @ 0.0 KiB seems strangely low...that should report with something since there is data within these directories.

      ncdu is a great tool though...it'll come in handy in the future.

      --- / ------------------------------------------------------------------------------------------------------------------------------------------------------------------
      12.1TiB [##########] /srv
      23.6GiB [ ] /export
      20.3GiB [ ] /var
      3.0GiB [ ] /home
      1.5GiB [ ] /usr
      1.0GiB [ ] /boot
      556.3MiB [ ] /lib
      256.1MiB [ ] /opt
      18.0MiB [ ] /run
      15.6MiB [ ] /sbin
      9.6MiB [ ] /bin
      8.5MiB [ ] /etc
      44.0KiB [ ] /root
      e 16.0KiB [ ] /lost+found
      8.0KiB [ ] /dev
      8.0KiB [ ] /media
      4.0KiB [ ] /lib64
      e 4.0KiB [ ] /mnt
      . 0.0 B [ ] /proc
      0.0 B [ ] /sys
      0.0 B [ ] /tmp
      @ 0.0 B [ ] initrd.img.old
      @ 0.0 B [ ] initrd.img
      @ 0.0 B [ ] vmlinuz.old
      @ 0.0 B [ ] vmlinuz

      Total disk usage: 12.1TiB Apparent size: 140.1TiB Items: 1432822
    • My eyes are drawn to /boot.

      It has no business being 1GB. Is it on a separate partition then that may also be full. Should be cleaned up. Old kernels?

      Also, if you previously by mistake wrote to a mount point that for some reason was not mounted then that write is stored in that folder. If that mount point is mounted now, then the mounted drive may hide/mask what is stored locally there in that folder. A hidden room...

      I try to avoid pushing data out from servers, instead pulling if possible. Especially in scripts that run unattended. That eliminates this type of mishap. If what I suspect is correct...

      To figure out if this is the case, I recommend that you boot from other media, thumbdrive perhaps, and examine the suspect rootfs when it is not running and there are no mounted volumes on any of the mount points.

      For nothing is hidden that will not be revealed, and nothing concealed that will not be made known and brought to light.
      OMV 4, 7 x ODROID HC2, 1 x ODROID HC1, 3 x 12TB, 2 x 8TB, 1 x 4TB, 1 x 2TB SSHD, 1 x 500GB SSD, GbE, WiFi mesh

      The post was edited 1 time, last by Adoby ().

    • Wondering if you could use ncdu -x
      It should not descend into other filesystems, so if there is a folder in /srv which is very large it should be one that is not actually mounted.
      Odroid HC2 - armbian - Seagate ST4000DM004 - OMV4.x
      Asrock Q1900DC-ITX - 16GB - 2x Seagate ST3000VN000 - Intenso SSD 120GB - OMV4.x
      :!: Backup - Solutions to common problems - OMV setup videos - OMV4 Documentation - user guide :!: