OMV SSD full. Can't login in OMV

    • OMV SSD full. Can't login in OMV

      Hi,

      I'm having a very weird problem.
      I have a 28 GB SSD on witch is installed OMV, and since yesterday I can't login OMV anymore and it appears that it's because my SSD is Full.
      28 GB used for OMV seemed kinda strange, so I ran Ubuntu to try and understand and found this :



      0 bytes free on a 28 GB SSD, but it also states that the content is totalling 3.6 GB minus the unreadable content.
      Only 1 folder was not readable when I tried was in ETC/SSL if I'm remembering correctly.

      Was the hell is going on ? :)

      Also how can I make some space to login and backup everything I need without deleting something important ?
    • There are several threads on this forum about filled root fs. And how you can figure out what you did to cause it and how to fix it.

      Try searching on Google for something like this: "Help! My OMV system drive is full. How do I fix this?"

      Some possible culprits are:

      1. rsync push jobs that was intended for a remote drive, but the remote drive was not correctly mounted. That may fill the folder that is used as mount point. Probably somewhere under /srv.

      2. Metadata for plex or emby or some other software. Probably somewhere under /var/lib.

      3. Some error that generated insane amount of logs. Probably somewhere under /var/log.
      OMV 4, 7 x ODROID HC2, 1 x ODROID HC1, 5 x 12TB, 1 x 8TB, 1 x 2TB SSHD, 1 x 500GB SSD, GbE, WiFi mesh
    • No docker is not your problem. Your problem is that YOU used the rootfs for data. That is a bad thing, as you know see.

      If you delete anything then YOU may break the server, the docker app or docker itself. Or it may be fine.

      I would reinstall OMV and make sure this time to not use the root fs for data. Either use an extra partition on the SSD or on the HDD for all the docker stuff. Docker base path AND docker app data and config. Absolutely not use the rootfs for data. It might work fine with the docker base path on the rootfs and the docker app data on the HDD. But I would avoid using the rootfs for anything that easily could be elsewhere.
      OMV 4, 7 x ODROID HC2, 1 x ODROID HC1, 5 x 12TB, 1 x 8TB, 1 x 2TB SSHD, 1 x 500GB SSD, GbE, WiFi mesh
    • To be honest I did used one of my HDD for all dockers appdata (when creating the different dockers).
      but I think I might have had quite a few dockers (10+ I would say) but what I still don't get is why did
      the problem occurred out of nowhere when I didn't touch the server for weeks. So why now ?

      Anyway, still can't log in and I need to backup my server, so Reinstall is not yet an option.

      is there a folder / files I can safely delete that would still let me connect to OMV and backup what is needed ?
    • OK. Then it is not metadata from plex/emby that filled your SSD. You need to figure our what it is. Check the other threads for tools and how to use them. For instance df, du and find.

      If you reinstall you should only loose what is on the rootfs. Not anything that is stored on the HDD.

      Perhaps you can use find and timestamps for files to figure out what large files have arrived on the SSD since the last update?

      Do you run some automatic backup scripts that has failed or file downloader?

      Perhaps you can delete old docker images that has been replaced with newer?Old log files.
      Old kernels from previous updates.
      Most likely everything in /tmp.
      Anything you are willing to loose.
      OMV 4, 7 x ODROID HC2, 1 x ODROID HC1, 5 x 12TB, 1 x 8TB, 1 x 2TB SSHD, 1 x 500GB SSD, GbE, WiFi mesh
    • I found what is using space, it's my dockers images. I think my SSD is too small for the 8 dockers I have downloaded and used. Some of them are a few GB.
      I managed to find 2 I can delete so I'll try that but not sure how big these 2 are and if it'll be enough to log back in OMV.

      After that if this still doesn't work I'll have to reinstall, but I wanted to backup the settings from some of my dockers as it will take a Long time to put back :)

      Anyway thanks for the help. I hope it's gonna work !
    • just one quick question, after using the sudo du -d1 -h -x / | sort -hcommand, here is the result

      Source Code

      1. 4.0K /export
      2. 4.0K /home
      3. 4.0K /lib64
      4. 4.0K /mnt
      5. 4.0K /sharedfolders
      6. 8.0K /media
      7. 12K /srv
      8. 16K /lost+found
      9. 52K /.idea
      10. 172K /opt
      11. 680K /root
      12. 3.0M /.git
      13. 9.2M /etc
      14. 13M /bin
      15. 15M /sbin
      16. 92M /boot
      17. 726M /lib
      18. 1.5G /usr
      19. 6.9G /var
      20. 9.3G /
      Display All
      what does the 9.3G / means ? Is it the total used size ?

      I do not remember exactly how much it was before deleting 2 dockers, but I sure was more than 15 GB.
      So I don't know why I stil can't log in OMV. ;(
    • Yes. 9.3GB is the whole filesystem.

      A full rootfs is an exceptional and potentially catastrophic event. It should never, never happen.

      It might be possible that it has destroyed the OMV install.

      But as said before, the data you have on the HDD is most likely safe. And you can access the rootfs now so you can backup whatever files you want.

      From your description you haven't lost any data. Just the install and config.
      OMV 4, 7 x ODROID HC2, 1 x ODROID HC1, 5 x 12TB, 1 x 8TB, 1 x 2TB SSHD, 1 x 500GB SSD, GbE, WiFi mesh
    • DeadServer wrote:

      Done !

      Thanks to the docker help page, I managed to list my dockers, stop, delete running dockers and mostly, delete images of non running dockers.
      Now I can log in !!
      Missing space on boot drive...

      This thread details the problem and how to fix it without deleting your containers. The problem is, you need to assign a folder to store your containers on one of your data drives.
      Air Conditioners are a lot like PC's... They work great until you open Windows.