var/log is running out of space

  • Hello,


    tried to get an answer already on an existing thread. Unfortunately the thread was already tagged as solved. So it seems that no one was reading it.Hope a new thread won't be seen as spam from my side.


    On my OMV 4.1.26 installation var/log is running out of space after a while. Available space is only 50M. When the space is out I receive error messages when I want to do a USB back up.


    So here is what I found out so far after some debugging trials via shell:


    Code
    root@Netzwerkspeicher:~# armbianmonitor -u
    System diagnosis information will now be uploaded to http://ix.io/1ZVr
    Please post the URL in the forum where you've been asked for.

    Disabling the armbian-ram-logging service in /etc/default/armbian-ramlog is changing it to folder2ram. Will there be any issues in the future disabling the log service?
    Is there any other workaround for this topic how to switch from ram2log to folder2ram vor not running out of space for var/log?


    Solved the topic temporarily with increasing the space from 50M to 100M. But won't solve the cause of it.


    Would be great If someone can help me on that! :thumbsup:

  • How about taking a look inside the logs to see what is filling them? And do something about that.


    If you can figure out what is writing to the log files, and fix that, I suspect that you will find that Bob is your uncle. Or at least a very close friend.

    Be smart - be lazy. Clone your rootfs.
    OMV 5: 9 x Odroid HC2 + 1 x Odroid HC1 + 1 x Raspberry Pi 4

  • How about taking a look inside the logs to see what is filling them? And do something about that.


    If you can figure out what is writing to the log files, and fix that, I suspect that you will find that Bob is your uncle. Or at least a very close friend.

    Done!


    Debug has 24MB and has a lot of entries with the following:

    Code
    Oct 24 22:52:00 Netzwerkspeicher kernel: [100557.897144] database(12390): WRITE block 3306944 on mmcblk1p2 (32 sectors)


    syslog.1 has 22MB and showing also the same message:



    Unfortunately also uncle Bob can't help me on that.


    Why are there that many entries in the log files?

  • You can see if you have the same problems with another card.


    Do you use some custom install? Not using the flash memory plugin? Then the card might die very quickly?

    Be smart - be lazy. Clone your rootfs.
    OMV 5: 9 x Odroid HC2 + 1 x Odroid HC1 + 1 x Raspberry Pi 4

  • I installed the image from sourceforge.net
    And the flash memory plugin was already installed and my understanding was that I don t have to change anything on that?!
    Isn't that correct? DO I have to change anything via fstab?


    fstab has the following input:

    Code
    UUID=d0da7bbe-e3af-4588-8715-aa5c4478eb88 / btrfs defaults,noatime,nodiratime,commit=600,compress=lzo 0 1
    UUID=5bab0a55-56f1-4443-8cac-297e1181425c /boot ext4 defaults,commit=600,errors=remount-ro 0 2
    tmpfs /tmp tmpfs defaults,nosuid 0 0
    # >>> [openmediavault]
    /dev/disk/by-label/Backupintern /srv/dev-disk-by-label-Backupintern xfs defaults,nofail,noexec,usrquota,grpquota 0 2
    /srv/dev-disk-by-label-Backupintern/Backupintern/Eigene\040Dateien/Christoph/30\040Back\040Ups/20\040ioBroker /export/BackUpioBroker none bind,nofail 0 0
    # <<< [openmediavault]
  • Find out what is flooding your logs and fix that. Also, if you aren't running logrotate you should be.

    --
    Google is your friend and Bob's your uncle!


    OMV AMD64 5.x on ASRock Rack C2550D4I C0 Stepping - 16GB ECC - Silverstone DS380 + Silverstone DS380 DAS Box.

  • A normal install of the image you used, will not cause any problems like the ones you mention. I have several HC2 running fine without filling any log files. I don't use any extra config to rotate the logs. It isn't needed.


    Something is wrong. You need to figure out what.


    One of the most obvious and simplest ways is a fresh install, from scratch, on a new card. And check after you configure or add anything if the logs start growing again. Then you know what is causing the problem with the growing log files.


    I used to have a lot of strange problems with my HC2s. I figured out that if I used or activated ANY of the physical disk properties, then everything went haywire. So I tried installing and not touching the physical disk properties. And now my HC2s are rock steady. It took me at least 5 full reinstall from scratch before I figured it out. But then I am pretty slow.

    Be smart - be lazy. Clone your rootfs.
    OMV 5: 9 x Odroid HC2 + 1 x Odroid HC1 + 1 x Raspberry Pi 4

  • It is not uncommon here that this is because PLEX DB is filling your boot drive. So, yes, find out what is bogarting your space.

    That is usually an issue with the Plex DB filling /var/lib/docker and is not related to Plex logs (that I have ever seen)


    It appears there is an issue with his sdcard and the constant logging of these issues is filling the drive.


    I would buy a new sdcard.

  • I am not using Plex. So seems that this can't be the reason for filling up the space in my syslog and debug log file.


    Still don't get it! After I opened both files yesterday no additional space was filled. So it seems that log rotate is working correctly now?!


    Can someone explain me how to check if there is an issue with logrotate?


    Found something in an other thread that a workaround is to use the following common in cron job:

    Code
    find /var/log/ -type f -mtime 90 -exec rm -rf {} \';


    If i put this command in shell I get the following and it seems not working right:


    Code
    root@Netzwerkspeicher:~# find /var/log/ -type f -mtime 90 -exec rm -rf {} \';
    find: missing argument to `-exec'
    Try 'find --help' for more information.

    Is it only possible to execute via cron?


    Hope that is not required to use a new SD card and start from scratch. But I agree with Adoby that I deactivated every physical disk properties and everything is working fine so far. ;)

  • It doesn't matter if log rotate is working or you have Plex or not (I don't think anyone suggested you did)...


    Those log entries suggest there's an issue with your sdcard...

  • There is a lot of fake storage media on the way. New SD cards, USB sticks etc. should always be checked with H2testw, Etcher or similar tools before using.

    OMV 3.0.99 (Gray style)
    ASRock Rack C2550D4I C0-stepping - 16GB ECC - 6x WD RED 3TB (ZFS 2x3 Striped RaidZ1)- Fractal Design Node 304

    Edited 2 times, last by cabrio_leo: typo ().

  • It doesn't matter if log rotate is working or you have Plex or not (I don't think anyone suggested you did)...


    Those log entries suggest there's an issue with your sdcard...


    There is a lot of fake storage media on the way. New SD cards, USB sticks etc. should always be checked with Hwtest2, Etcher or similar tools before using.

    Thanks, guys!
    So since yesterday there are no more logs like there was the ones before. Really don't know what is different now. But hopefully that topic is solved and won't come back in future. :thumbup:
    I am using a brand new SanDisk Extreme 32GB ordered from Amazon. Is there really a chance to get a faked one? 8|

Participate now!

Don’t have an account yet? Register yourself now and be a part of our community!