[Tutorial][Experimental][Third-party Plugin available]Reducing OMV's disk writes, also to install it on USB flash

  • So it maybe wrote every sector once on your flash disk. OpenMediaVault hammers certain sectors with hundreds of writes. Depending on what type of flash storage, a cell can only sustain a couple of thousand writes. If those cells wear out you start experiencing problems.


    Greetings
    David

    "Well... lately this forum has become support for everything except omv" [...] "And is like someone is banning Google from their browsers"


    Only two things are infinite, the universe and human stupidity, and I'm not sure about the former.

    Upload Logfile via WebGUI/CLI
    #openmediavault on freenode IRC | German & English | GMT+1
    Absolutely no Support via PM!

  • My first post here...


    What a good topic, I was looking for some way that I can reduce disk I/O (because I want to make use of CF card as OS drive)
    To me, reducing disk IO, means more chance to let drive to spin down, less heat/noise? And if possible for a USB memory installation a SATA port can be saved as well!


    But I do have a question, for the fs2ram, I don't see there is any setting to flush memory content to disk, did I missed something?

    • Offizieller Beitrag

    Look at /etc/fs2ram.conf after you have enabled the plugin. It will answer your question. Or you can look at line 38 through 45 of this file.

    omv 7.0.4-2 sandworm | 64 bit | 6.5 proxmox kernel

    plugins :: omvextrasorg 7.0 | kvm 7.0.10 | compose 7.1.2 | k8s 7.0-6 | cputemp 7.0 | mergerfs 7.0.3


    omv-extras.org plugins source code and issue tracker - github


    Please try ctrl-shift-R and read this before posting a question.

    Please put your OMV system details in your signature.
    Please don't PM for support... Too many PMs!

    • Offizieller Beitrag

    It doesn't sync on a scheduled basis. That would defeat the purpose of reducing writes. It syncs whenever you reboot. I wouldn't use fs2ram without a UPS battery backup. None of the directories are critical (in my opinion). So, if they were lost, it wouldn't be a big deal. The plugin didn't code this. This is how the fs2ram package works. The plugin just tells it which directories to mount to ram to reduce writes.

    omv 7.0.4-2 sandworm | 64 bit | 6.5 proxmox kernel

    plugins :: omvextrasorg 7.0 | kvm 7.0.10 | compose 7.1.2 | k8s 7.0-6 | cputemp 7.0 | mergerfs 7.0.3


    omv-extras.org plugins source code and issue tracker - github


    Please try ctrl-shift-R and read this before posting a question.

    Please put your OMV system details in your signature.
    Please don't PM for support... Too many PMs!

  • I was wondering if it makes sense to add a CRON Job for RSync every X minutes, as I'm not restarting my Server very often. Don't know much about fs2ram though


    EDIT: Nevermind. Didn't see that Installed files and configurations aren't affected of fs2ram :D Very good plugin though!

    • Offizieller Beitrag

    It is the amount of kilobytes written to the OS drive since the last reboot. Not real important but it shows you how little is being written to the drive after the plugin is installed. This should translate to a much longer life.

    omv 7.0.4-2 sandworm | 64 bit | 6.5 proxmox kernel

    plugins :: omvextrasorg 7.0 | kvm 7.0.10 | compose 7.1.2 | k8s 7.0-6 | cputemp 7.0 | mergerfs 7.0.3


    omv-extras.org plugins source code and issue tracker - github


    Please try ctrl-shift-R and read this before posting a question.

    Please put your OMV system details in your signature.
    Please don't PM for support... Too many PMs!

    • Offizieller Beitrag

    I've been averaging 20 - 60 MB per day on my systems. 137 MB is still a lot less than my systems not using the plugin.

    omv 7.0.4-2 sandworm | 64 bit | 6.5 proxmox kernel

    plugins :: omvextrasorg 7.0 | kvm 7.0.10 | compose 7.1.2 | k8s 7.0-6 | cputemp 7.0 | mergerfs 7.0.3


    omv-extras.org plugins source code and issue tracker - github


    Please try ctrl-shift-R and read this before posting a question.

    Please put your OMV system details in your signature.
    Please don't PM for support... Too many PMs!

  • the plugin is running on my Raspberry pi B model.
    Before installing OMV, I had raspbian running for 2 years continuously on a different SD card. The system was running 24/7 to perform Wake on Lan, Torrentsync duties and occasionally some torrent downloads (on a separate USB key). The card is still good but I just decided to get a new one and to install OMV. Hopefully this will last as long as my previous one!

    • Offizieller Beitrag

    That is amazing luck :) I definitely think the plugin will make cards last that long. Since I started making the RPi and Odroid-C1 images, I have probably had 15 sd/microsd cards fail.

    omv 7.0.4-2 sandworm | 64 bit | 6.5 proxmox kernel

    plugins :: omvextrasorg 7.0 | kvm 7.0.10 | compose 7.1.2 | k8s 7.0-6 | cputemp 7.0 | mergerfs 7.0.3


    omv-extras.org plugins source code and issue tracker - github


    Please try ctrl-shift-R and read this before posting a question.

    Please put your OMV system details in your signature.
    Please don't PM for support... Too many PMs!

  • Hah, not dead yet lol.
    Good to see that the raspberry/banana/whateverARMboard/hackedNAS crowd is now able to enjoy OMV without nuking the SD card/USB drive within months.
    Thanks ryecoaaron for the plugin! :)


    Might be useful adding the script from Solo0815 so you can save logs or whatever at a regular schedule in a user-defined folder (in the data drives or in the system drive). [Tutorial][Experimental]Reducing OMV's disk writes, also to install it on USB flash


    For low-ram devices, the advice to comment out the swap partition might not be the best choice, as they might actually find themselves in situations where they need it (to the contrary of the average commandeered PC someone installs OMV on).
    An old friend of mine, swapspace might be useful here.
    It's a package in the debian repos since ages ago, it contains a smart script that auto-generates and auto-mounts swap files when needed. It just needs to know the location where it can make the swapfiles (defined inside its config file, among other settings that can be left alone without consequences).
    A good place would be inside the data drives (the raid, or whatever).
    Although nowadays you can easily afford to make a permanent 4 GB swap file and link it yourself, given the disk sizes we are working with. There is another package designed to do the leg work for this too, dphys-swapfile (yet another script, yay). It just looks at RAM and generates or enlarges or shrinks the swapfile at boot.


    Afaik, the only thing that a swapfile cannot do is storing hybernation data (to hybernate the PC), performance is on par with a swap partition.


    Given that OMV is a server/NAS distro (hybernation isn't exactly a killer feature on a NAS, correct?), it might be a good idea to not have it waste OS drive space (and wearing the flash cells) with swap partitions when installing and maybe leaving the swap needs to a script like swapspace or dphys-swapfile running automatically in the data partition/drives.
    Again I'm talking of the low-end devices with SD cards and USB flash drives, not the high-end setups run by tekkb.


    Ah, one last thing, Btrfs hates swap files, so make sure no script tries to make a swap file in a btrfs partition.


    So it is possible to limit the size of tmpfs 5 or 10MB? Because with 512MB of ram it can be a bit limited.

    tmpfs occupies only as much space as the data in it, and if you have swap (file or partition) it can be swapped when ram is needed.
    It's max size is half your total ram (automatically determined), and it can be limited to whatever size by editing the /etc/fstab
    changing the tmpfs entry into (or creating it fi there was none)

    Code
    tmpfs   /tmp         tmpfs   nodev,nosuid,size=20m          0  0


    size=20m means the tmpfs is limited to 20 Mb.
    if you write size=1G that is 1 GB.
    Don't give it a lot of RAM unless you have plenty to spare. If you set too high values and the tmpfs fills up the system will be starved for RAM and that's not going to end well.
    Also don't limit the tmpfs too much as it might be too small and that's an issue as other system processes use the tmpfs for their stuff.
    Play with the values at your own risk.
    If you do play with them, please post the results. :)

    • Offizieller Beitrag

    For low-ram devices, the advice to comment out the swap partition might not be the best choice, as they might actually find themselves in situations where they need it (to the contrary of the average commandeered PC someone installs OMV on).


    The standard Raspbian image doesn't use swap and neither does my odroid-c1 image. The kernel still knows what to do when it runs out of ram and/or swap space. I have been testing an RPi (and three other systems) and haven't had issues. The swap space comment was more x86 users switching to a usb stick.

    omv 7.0.4-2 sandworm | 64 bit | 6.5 proxmox kernel

    plugins :: omvextrasorg 7.0 | kvm 7.0.10 | compose 7.1.2 | k8s 7.0-6 | cputemp 7.0 | mergerfs 7.0.3


    omv-extras.org plugins source code and issue tracker - github


    Please try ctrl-shift-R and read this before posting a question.

    Please put your OMV system details in your signature.
    Please don't PM for support... Too many PMs!

  • Wonderful plugin ryecoaaron!


    I had been just tweaking my OMV installation on the HP MicroServer Gen8.


    Before plugin, with no data drive set up and nothing being pretty much done it did write ~ 450M in ~ 15 minutes.
    After the plugin has been installed I can see 0.5M in the same time frame.
    I'm actually assuming "awk '/sd/ {print $3"\t"$10 / 2 / 1024}' /proc/diskstats" gives me the output in MegaBytes.


    I'm wondering, given the Swap partition has been commented out, is it suggested to actually remove the partition from the drive and completely remove the entry from the fstab?
    This would free some space in the SD card :)


    Additionally, is there anything that can be done to flush the data written to tempfs every now and then, without needing to reboot?
    Just to avoid to have much RAM allocated for logs and files which are not needed.


    Thanks,
    Tommy

    HP MicroServer Gen 8
    HP DL360e Gen 8 + HP StorageWorks D2700

    HP DL380e Gen 8 + HP StorageWorks D2600

    • Offizieller Beitrag

    I removed the swap partition and line from fstab but you don't have to. I don't recommend using the OS drive for data and a larger OS partition usually isn't needed.


    I haven't look into flushing tmpfs since I haven't run into memory issues (even on RPi).

    omv 7.0.4-2 sandworm | 64 bit | 6.5 proxmox kernel

    plugins :: omvextrasorg 7.0 | kvm 7.0.10 | compose 7.1.2 | k8s 7.0-6 | cputemp 7.0 | mergerfs 7.0.3


    omv-extras.org plugins source code and issue tracker - github


    Please try ctrl-shift-R and read this before posting a question.

    Please put your OMV system details in your signature.
    Please don't PM for support... Too many PMs!

  • Thanks @ryecoaaron,


    I have removed the swap partition and the entry in the fstab.
    You're right even with fs2ram enabled the RAM utilization is pretty low (1% of 16GB).


    Not that the server is busy as it's only creating a Raid volume.


    On a side note, very silly question.


    I have 2 3TB drives, sda and sdb, these have no partitions as I'm building the raid 1 (currently 6% throughout the process).
    In the fstab files I see some entries referring to sda (1,2 and 5) which in my opinion are incorrect.

    Code
    # <file system>                                 <mount point>   <type>          <options>                               <dump>  <pass>
    UUID=4804f249-fd46-4f71-85c9-62b7b0fa9737       /               ext4            noatime,nodiratime,errors=remount-ro    0       1
    /dev/sr0                                        /media/cdrom0   udf,iso9660     user,noauto                             0       0
    /dev/sda1                                       /media/usb0     auto            rw,user,noauto                          0       0
    /dev/sda2                                       /media/usb1     auto            rw,user,noauto                          0       0
    /dev/sda5                                       /media/usb2     auto            rw,user,noauto                          0       0
    tmpfs                                           /tmp            tmpfs           defaults,noatime,nodiratime             0       0


    Shall I remove these, as there are no USB drives connected to the NAS?
    I suspect these were added during the setup (which was done via USB key) as there were few USB thumb drives connected (the target SD and its clone).


    On a side note, something I'd like to share and perhaps worth sharing with who aims to reduce the writes to drive (not only the SD card or the USB flash)


    I added the "noatime,nordiartime" options on every drive in the fstab (including data drives, ssd, flash drives such as SD and USB thumb drive).
    Additionally on the SSD entry I added the "discard" option so to trigger the Trim.


    The goal is to reduce the data written as such, so that potentially useless information are not committed to disk (such as the last time a file/directory was accessed).
    Without the noatime ext4 would log any read performed on a given file; Same applies to directories when the nodiratime option isn't specified.


    Tommy

    HP MicroServer Gen 8
    HP DL360e Gen 8 + HP StorageWorks D2700

    HP DL380e Gen 8 + HP StorageWorks D2600

    • Offizieller Beitrag

    I remove the usb lines from fstab.

    omv 7.0.4-2 sandworm | 64 bit | 6.5 proxmox kernel

    plugins :: omvextrasorg 7.0 | kvm 7.0.10 | compose 7.1.2 | k8s 7.0-6 | cputemp 7.0 | mergerfs 7.0.3


    omv-extras.org plugins source code and issue tracker - github


    Please try ctrl-shift-R and read this before posting a question.

    Please put your OMV system details in your signature.
    Please don't PM for support... Too many PMs!

Jetzt mitmachen!

Sie haben noch kein Benutzerkonto auf unserer Seite? Registrieren Sie sich kostenlos und nehmen Sie an unserer Community teil!