[Tutorial][Experimental][Third-party Plugin available]Reducing OMV's disk writes, also to install it on USB flash

  • I agree it is not reliable. I think noatime and nodiratime should be set for all devices. And according to a few pages, Ubuntu (14.04 LTS) stopped using discard in favor of daily (anacron?) fstrim -v. If OMV did both of these ideas, it wouldn't need to determine rotation or not. If it is bad to run fstrim on a hard drive, maybe latency could be used to determine if it is a rotational (hybrid hard drives might cause issue)?

    omv 6.0.8-1 Shaitan | 64 bit | 5.15 proxmox kernel | omvextrasorg 6.0.5 | kvm plugin 6.0.3
    omv-extras.org plugins source code and issue tracker - github


    Please read this before posting a question.
    Please don't PM for support... Too many PMs!

  • Just tested. All my flash USB drives and all my SD cards (around 10-13 different devices of 5 different brands + 3 without a brand, yes I'm cheap) are rotational according to the kernel.
    Yo, I did not do something stupid, it's an actual kernel bug.


    So yeah, that script needs to look at something else or it's never going to work.
    Opened a bug report here, feel free to join the party http://bugtracker.openmediavault.org/view.php?id=1318


    Agree on latency checks, anything based on flash memory is random-access so should perform VERY differently if benched on latency alone (SSD or USB or SD card).


    ioping seems to be the best tool for this job. Available in wheezy repos, but I pulled down the one from unstable repos. https://github.com/koct9i/ioping


    Now we see how even a crappy SD card of mine blasts away the sata drives in the latency test

    Code
    ioping -R /dev/sdb
    --- /dev/sdb (block device 1397 GiB) ioping statistics ---
    168 requests completed in 3.02 s, 55 iops, 223.1 KiB/s
    min/avg/max/mdev = 6.28 ms / 17.9 ms / 30.3 ms / 4.89 ms

    This is a data drive, 1.5 TB.


    Quote

    ioping -R /dev/sdd


    --- /dev/sdd (block device 29.7 GiB) ioping statistics ---
    1.15 k requests completed in 3.00 s, 388 iops, 1.52 MiB/s
    min/avg/max/mdev = 1.59 ms / 2.57 ms / 23.3 ms / 806 us

    SD card in a USB reader. Yeah, pretty obvious that its seek time is MUCH better on average.


    Can't test on SSD, but I assume that it won't be worse than a SD card.

  • If its a kernel bug, reporting it to the debian/kernel.org stuff might get clearance about that.


    Also crosschecking between stable and backports kernel should be done, and jessie kernel too.


    Greetings
    David

    "Well... lately this forum has become support for everything except omv" [...] "And is like someone is banning Google from their browsers"


    Only two things are infinite, the universe and human stupidity, and I'm not sure about the former.

    Upload Logfile via WebGUI/CLI
    #openmediavault on freenode IRC | German & English | GMT+1
    Absolutely no Support via PM!

  • Ubuntu (14.04 LTS) stopped using discard in favor of daily (anacron?) fstrim -v.

    I think it is the other way around.
    older implementations of TRIM in the kernel (what happens when you place the discard mount option) sucked big way when deleting files (and the TRIM kicked in)
    https://patrick-nagel.net/blog/archives/337
    Nowadays the impact is minimal (unless you love to delete hundreds of GB of small files repeatedly from the SSD for some reason, which is an unrealistic use case) and should get better.
    https://kparal.wordpress.com/2…-option-on-intel-525-ssd/


    Quote

    (hybrid hard drives might cause issue)

    Possible, but I really doubt that someone will place one of these in a NAS. Will need some investigation.


    Quote

    If its a kernel bug, reporting it to the debian/kernel.org stuff might get clearance about that.


    Also crosschecking between stable and backports kernel should be done, and jessie kernel too.


    Greetings
    David

    I'm waiting for Antergos to download (Arch linux, easy to install), I want to see if the same issue persists in the latest kernel too before posting bugs in the kernel bugtracker.

  • The mod in the original post has been doing fine for me for the past ~8 months on a Sandisk Cruzer 4GB and 4GB RAM. I did have an issue in the past such that I am missing status logs for a large period of time and web-ui update seemed to not work but I seem to have solved that today by setting my apt source to wheezy and doing a dist-upgrade. I'll probably upgrade to Arch on my fancy SLC usb drive once I get bored enough or this setup dies.

  • Why would you "upgrade" to Arch?

    omv 6.0.8-1 Shaitan | 64 bit | 5.15 proxmox kernel | omvextrasorg 6.0.5 | kvm plugin 6.0.3
    omv-extras.org plugins source code and issue tracker - github


    Please read this before posting a question.
    Please don't PM for support... Too many PMs!

  • So, having read this entire thread. What's next? Is this something that will be part of the next version of OMV? Or does it stay an experimental plugin? Or is it trusted and tried at this time? It's not a strange idea to want to run OMV from a large USB 3.0 stick I think. Provided that it is stable of course...


    Ralph

    ASRock H97 Pro4 | 8Gb 1600 | i3 4130T | 3x WD Red 4TB with SnapRAID | Backup to Crashplan & External HDD

  • It will never be more than "experimental" or "in testing" as long as Volker itself does not implement it into the CORE of OMV.


    However, it already is stable and works really good.

  • It is always more risky to have data in ram vs. being written to disk if there is power failure. With a server that has important data people should be using a battery backup and I think AVR is good too.

  • Is there a downside of using the plugin? If no, why not always enable it, even with a normal boot HDD?

    The only downside is that in case of sudden power otuage you lose some logs, and the databases that OMV uses to make the pretty CPU/Network/ram usage graphs.


    Other stuff is just disabled by the plugin to avoid unnecessary writes (like say downloading packages whenever there are new ones in repos, to make a faster install when updating, which is not really needed).


    If you want to save that stuff regularly to disk, you need to set a chron job manually calling a "folder2ram -syncall", which syncs the contents of the ram folders to the disk folders.


    Quote

    It will never be more than "experimental" or "in testing" as long as Volker itself does not implement it into the CORE of OMV.

    That's kinda unfair, it's not based off fs2ram anymore now, the only failures we had are not possible with folder2ram because of safety checks.


  • Other stuff is just disabled by the plugin to avoid unnecessary writes (like say downloading packages whenever there are new ones in repos, to make a faster install when updating, which is not really needed).


    on my raspberry pi new packages are always downloaded...? is there a way to check that the plugin is working correctly?

  • this is what I get


  • Running from USB for a week or so. How do I know it works?
    cat /sys/block/sdf/queue/rotational = 1 on my SanDisk Ultra USB 3.0 16Gb
    I get around 65 Mb/hr of writes.

    ASRock H97 Pro4 | 8Gb 1600 | i3 4130T | 3x WD Red 4TB with SnapRAID | Backup to Crashplan & External HDD

  • ... If you want to save that stuff regularly to disk, you need to set a chron job manually calling a "folder2ram -syncall", which syncs the contents of the ram folders to the disk folders...


    I didn't even knew that this is possible - thank you for implementing it!
    Maybe I'll give it a try syncing data back to disk once every week (mostly because it gives a more consistent view for the graphics in case of power loss).

    OMV 1.19 (OMV-Extras.org 1.34) | BanaNAS | 16GB microSD (using FlashMemory plugin) | 1TB SATA WD Blue Mobile

  • I started following the first page guide but I have


    I saw a plugin dedicated to use OMV on flash memory (I'm running omv on a rasoberry pi B with usb card)
    Is this thread/guide outdated or there are any motives to still do these operations manually?


    Thanks!

  • All the computer masters,are you hear the new kind of ransomware? who was named VVV, my computer was be hitted, and i don't know what should i do

  • I started following the first page guide but I have


    I saw a plugin dedicated to use OMV on flash memory (I'm running omv on a rasoberry pi B with usb card)
    Is this thread/guide outdated or there are any motives to still do these operations manually?


    Thanks!


    Just install the flashmemory plugin. That is all you need to do.

    omv 6.0.8-1 Shaitan | 64 bit | 5.15 proxmox kernel | omvextrasorg 6.0.5 | kvm plugin 6.0.3
    omv-extras.org plugins source code and issue tracker - github


    Please read this before posting a question.
    Please don't PM for support... Too many PMs!

Participate now!

Don’t have an account yet? Register yourself now and be a part of our community!