Flash Memory Plugin

  • SSH is slow, when loggin in and when typing commands. Also, the OMV web interface is almost dead now, it is so slow. I've rebooted the server and there seems to be no change after the reboot

    Maybe your flash drive sucks (which is pretty normal, the majority of these things does when used as an OS drive).


    Can you please provide the output of these two commands (stopping the second one after 60 seconds with ctrl-c:

    Code
    df -h
    iostat 10

    For iostat to work on x86 it most probably needs an apt-get install sysstat before. And while iostat is running, please click around in OMV's web UI.

  • Thank you. I will do so now. I've installed 'apt-get install sysstat', it worked fine.
    However, when I do 'df -h iostat 10' i get an error message:


    root@quincy:~# df -h iostat 10
    df: iostat: No such file or directory
    df: 10: No such file or directory
    root@quincy:~#

  • Sorry, I really am a noob. I will try again. Ithought it was one command. Before anybody responded on the forum, I managed to remove the flash memory utility, but it seems to have made no difference.


    root@quincy:~# df -h
    Filesystem Size Used Avail Use% Mounted on
    udev 1.9G 0 1.9G 0% /dev
    tmpfs 384M 6.4M 378M 2% /run
    /dev/sde1 25G 16G 7.5G 68% /
    tmpfs 1.9G 0 1.9G 0% /dev/shm
    tmpfs 5.0M 0 5.0M 0% /run/lock
    tmpfs 1.9G 0 1.9G 0% /sys/fs/cgroup
    tmpfs 1.9G 0 1.9G 0% /tmp
    /dev/sdb1 7.3T 5.2T 2.1T 72% /srv/dev-disk-by-label-Data2
    /dev/sdd1 7.3T 618G 6.7T 9% /srv/dev-disk-by-label-Data4
    /dev/sda1 7.3T 5.7T 1.6T 79% /srv/dev-disk-by-label-Data1
    /dev/sdc1 7.3T 93M 7.3T 1% /srv/dev-disk-by-label-Data3
    cy:~# df -h iostat 10
    Linux 4.19.0-0.bpo.4-amd64 (quincy) 22/05/2019 _x86_64_ (2 CPU)
    avg-cpu: %user %nice %system %iowait %steal %idle
    2.14 0.00 6.37 22.38 0.00 69.10
    Device: tps kB_read/s kB_wrtn/s kB_read kB_wrtn
    avg-cpu: %user %nice %system %iowait %steal %idle
    1.73 0.00 6.66 2.95 0.00 88.67
    Device: tps kB_read/s kB_wrtn/s kB_read kB_wrtn
    avg-cpu: %user %nice %system %iowait %steal %idle
    1.97 0.00 5.20 21.05 0.00 71.78
    Device: tps kB_read/s kB_wrtn/s kB_read kB_wrtn
    avg-cpu: %user %nice %system %iowait %steal %idle
    1.89 0.00 5.66 4.59 0.00 87.87
    Device: tps kB_read/s kB_wrtn/s kB_read kB_wrtn
    avg-cpu: %user %nice %system %iowait %steal %idle
    1.78 0.00 7.57 12.20 0.00 78.44
    Device: tps kB_read/s kB_wrtn/s kB_read kB_wrtn

  • Sorry, I really am a noob. I will try again. Ithought it was one command. Before anybody responded on the forum, I managed to remove the flash memory utility, but it seems to have made no difference.



    root@quincy:~# df -h
    Filesystem Size Used Avail Use% Mounted on
    udev 1.9G 0 1.9G 0% /dev
    tmpfs 384M 6.5M 378M 2% /run
    /dev/sde1 25G 16G 7.5G 68% /
    tmpfs 1.9G 0 1.9G 0% /dev/shm
    tmpfs 5.0M 0 5.0M 0% /run/lock
    tmpfs 1.9G 0 1.9G 0% /sys/fs/cgroup
    tmpfs 1.9G 0 1.9G 0% /tmp
    /dev/sdb1 7.3T 5.2T 2.1T 72% /srv/dev-disk-by-label-Data2
    /dev/sdd1 7.3T 618G 6.7T 9% /srv/dev-disk-by-label-Data4
    /dev/sda1 7.3T 5.7T 1.6T 79% /srv/dev-disk-by-label-Data1
    /dev/sdc1 7.3T 93M 7.3T 1% /srv/dev-disk-by-label-Data3
    root@quincy:~# iostat 10
    Linux 4.19.0-0.bpo.4-amd64 (quincy) 22/05/2019 _x86_64_ (2 CPU)
    avg-cpu: %user %nice %system %iowait %steal %idle
    2.06 0.00 6.57 19.42 0.00 71.94
    Device: tps kB_read/s kB_wrtn/s kB_read kB_wrtn
    sdd 0.34 17.08 0.01 18594 9
    sdc 0.36 18.65 0.01 20294 9
    sdb 1.24 20.30 0.01 22094 9
    sda 33.38 61.27 21553.29 66686 23457089
    sde 15.70 360.23 77.83 392047 84704
    avg-cpu: %user %nice %system %iowait %steal %idle
    6.07 0.00 7.95 7.24 0.00 78.74
    Device: tps kB_read/s kB_wrtn/s kB_read kB_wrtn
    sdd 1.00 54.75 0.00 547 0
    sdc 0.60 37.95 0.00 379 0
    sdb 0.20 0.80 0.00 8 0
    sda 1.20 3.15 2.40 31 24
    sde 25.70 520.80 279.20 5208 2792
    avg-cpu: %user %nice %system %iowait %steal %idle
    4.38 0.00 6.88 20.63 0.00 68.11
    Device: tps kB_read/s kB_wrtn/s kB_read kB_wrtn
    sdd 0.00 0.00 0.00 0 0
    sdc 0.00 0.00 0.00 0 0
    sdb 0.00 0.00 0.00 0 0
    sda 52.80 1.20 48894.00 12 488940
    sde 24.90 247.60 21.20 2476 212
    avg-cpu: %user %nice %system %iowait %steal %idle
    10.03 0.00 9.67 13.28 0.00 67.02
    Device: tps kB_read/s kB_wrtn/s kB_read kB_wrtn
    sdd 0.00 0.00 0.00 0 0
    sdc 0.00 0.00 0.00 0 0
    sdb 0.00 0.00 0.00 0 0
    sda 4.00 0.00 3715.60 0 37156
    sde 122.60 7287.20 52.40 72872 524
    avg-cpu: %user %nice %system %iowait %steal %idle
    6.79 0.00 9.55 17.11 0.00 66.55
    Device: tps kB_read/s kB_wrtn/s kB_read kB_wrtn
    sdd 0.00 0.00 0.00 0 0
    sdc 0.00 0.00 0.00 0 0
    sdb 0.00 0.00 0.00 0 0
    sda 49.20 0.00 45701.60 0 457016
    sde 35.90 686.00 41.60 6860 416

  • df -h iostat 10

    My mistake, sorry. It should've been two lines instead.


    Anyway, your output shows that the flashmemory plugin is NOT active and that your flash drive sucks as expected. That's the output:


    Code
    avg-cpu: %user %nice %system %iowait %steal %idle
    2.14 0.00 6.37 22.38 0.00 69.10
    1.73 0.00 6.66 2.95 0.00 88.67
    1.97 0.00 5.20 21.05 0.00 71.78
    1.89 0.00 5.66 4.59 0.00 87.87
    1.78 0.00 7.57 12.20 0.00 78.44

    High %iowait values are an indication that your flash drive is horribly slow if it's about random IO.


    I would try a reboot now and also disable Monitoring in the UI since most probably that's the culprit. Then please check again with df -h since there should be a bunch of folder2ram occurrences otherwise the plugin still isn't active.

  • I've rebooted the system now. Logging in via the shellinabox is much faster now. Before you answered I did uninstall that flash-drive utility, so it is no longer on the system.


    root@quincy:~# df -h
    Filesystem Size Used Avail Use% Mounted on
    udev 1.9G 0 1.9G 0% /dev
    tmpfs 384M 5.9M 379M 2% /run
    /dev/sde1 25G 16G 7.5G 68% /
    tmpfs 1.9G 0 1.9G 0% /dev/shm
    tmpfs 5.0M 0 5.0M 0% /run/lock
    tmpfs 1.9G 0 1.9G 0% /sys/fs/cgroup
    tmpfs 1.9G 0 1.9G 0% /tmp
    /dev/sda1 7.3T 5.7T 1.6T 79% /srv/dev-disk-by-label-Data1
    /dev/sdb1 7.3T 5.2T 2.1T 72% /srv/dev-disk-by-label-Data2
    /dev/sdc1 7.3T 93M 7.3T 1% /srv/dev-disk-by-label-Data3
    /dev/sdd1 7.3T 618G 6.7T 9% /srv/dev-disk-by-label-Data4
    root@quincy:~#



    root@quincy:~# iostat 10
    Linux 4.19.0-0.bpo.4-amd64 (quincy) 22/05/2019 _x86_64_ (2 CPU)
    avg-cpu: %user %nice %system %iowait %steal %idle
    1.53 0.11 1.95 9.12 0.00 87.29
    Device: tps kB_read/s kB_wrtn/s kB_read kB_wrtn
    sda 40.65 212.30 2031.08 66202 633373
    sdb 4.39 70.84 0.03 22090 9
    sdd 1.17 57.92 0.03 18062 9
    sdc 1.13 57.78 0.03 18018 9
    sde 58.24 1127.05 20.38 351459 6356
    avg-cpu: %user %nice %system %iowait %steal %idle
    5.22 0.00 7.70 9.27 0.00 77.81
    Device: tps kB_read/s kB_wrtn/s kB_read kB_wrtn
    sda 14.10 0.00 13901.60 0 139016
    sdb 0.00 0.00 0.00 0 0
    sdd 0.00 0.00 0.00 0 0
    sdc 0.00 0.00 0.00 0 0
    sde 5.10 3.60 27.20 36 272
    avg-cpu: %user %nice %system %iowait %steal %idle
    16.47 0.00 11.59 28.11 0.00 43.82
    Device: tps kB_read/s kB_wrtn/s kB_read kB_wrtn
    sda 47.80 0.40 45646.40 4 456464
    sdb 0.00 0.00 0.00 0 0
    sdd 0.00 0.00 0.00 0 0
    sdc 0.00 0.00 0.00 0 0
    sde 83.10 4796.00 26.00 47960 260
    ^C
    root@quincy:~#


  • To post these command outputs please use the </> symbol in the toolbar since otherwise spacing is incorrect.


    That's what iostat still says:


    Code
    avg-cpu: %user %nice %system %iowait %steal %idle
    16.47 0.00 11.59 28.11 0.00 43.82
    Device tps kB_read/s kB_wrtn/s kB_read kB_wrtn
    sda 47.80 0.40 45646.40 4 456464
    sde 83.10 4796.00 26.00 47960 260

    sde is your flash drive and there's an awful lot of read activity (+80 transactions per second). And %iowait is still way too high. I would try to reinstall the flashmemory plugin and check again.

  • I don't trust the flash drive installation


    There's nothing wrong with flash other than


    • Counterfeit flash storage (that's the root cause for so many people having bad experiences with flash drives. If you buy genuine flash products and test them for fake capacity first then everything's ok)
    • Due to high Write Amplification flash can wear out a lot faster than necessary. That's what the flashmemory plugin is for and that's why on installations running on flash memory OMV's internal monitoring should be disabled (unfortunately the so called 'New User Guide' recommends to enable monitoring -- see the remark for page 38). I already suggested to disable monitoring. Did you follow the advice?
  • Just make sure to use a good quality USB thumbdrive and the flash memory plugin. And don't use the rootfs for data storage. Two thumbdrives simplifies backups and rollback a lot. Flash improves performance. And frees a valuable hdd for use for data. Possibly use high end USB drives, like SanDisk Extreme Pro. But still use the flash memory plugin.

    Be smart - be lazy. Clone your rootfs.
    OMV 5: 9 x Odroid HC2 + 1 x Odroid HC1 + 1 x Raspberry Pi 4

    Edited once, last by Adoby ().

  • HI tkaiser,


    Yes, I followed your advice, it is disabled, and the flash utility has been re-installed. The system seems stable today. I will see once all files have been synchronized. Then data transfers will be much reduced.
    I'll check but I believe it is a high quality thumb drive, bought it from Makro, a reputable dealer (I would like to believe they are). Makro are owned by MassMart these days. (I live in South Africa). It is a SanDisk UltraFit USB 3.1 Flash drive. The rated speeds are up to 130MB/s on the packaging.

  • It is a SanDisk UltraFit USB 3.1 Flash drive

    That's at least written on the outside. SanDisk estimated some time ago that 30% of flash products showing their logo are fakes. That's why testing for fake capacity prior to usage is so important :)


    Even reputable sellers can't prevent fake products being inserted into the supply chain early. Though reputable sellers with a 'no questions asked' return/refund policy at least allow for compensation. But this still requires testing the device in question with either F3 or H2testw. Testing with f3 is even possible from within a running Linux/OMV install (on the OMV images for ARM I prepared this long ago, there it's just executing armbianmonitor -c $HOME but on x86 you are on your own -- in case of interest feel free to ask for details)

  • (on the OMV images for ARM ... it's just executing armbianmonitor -c $HOME

    Since I had to check the internal eMMC of a mini server (NanoPi M1 Plus) anyway I did it right now and here the results as an example (the most important stuff at the bottom: Health status and performance overview):

Participate now!

Don’t have an account yet? Register yourself now and be a part of our community!