New approach for Raspberry Pi OMV images

  • So this means that we can trash our OMV instances and restart from scratch with official images?

    Hey,
    no I think you can trash the RPi itself ! I'm not very good with english, but I think I've understood since a moment ago Tkaiser think the RPis are "crap" if used as a NAS (by many reasons written on this forum, 1st is USB/LAN bad performance, powerfailure, sdcard slow/broken... etc.), and now he's fed up with working on this board, if I'm not wrong.

    maintaining OMV on crap



    There is a topic Which energy efficient ARM platform to choose? where @richprim ask about low power ARM platforms.
    It can be useful to find informations for finding a better hardware for Christmas gift for replacement, and being able to have good perf with OMV. (and keeping OMV compatibility hardware/software support...)



    @tkaiser I don't want to say mistakes in this post about you and your work, please fix me if I'm wrong ;)

  • I think you can trash the RPi itself

    No, it's totally fine if you use an RPi as OMV NAS just as it's totally fine if you loose your data. If you like the combination of slow and unreliable then a Raspberry Pi as NAS is always a great choice.


    It's just that if I spend time on something I want to be part of a solution and not just a problem. And the real problem is that people seem not to understand/accept the limitations of such toys. There was a bizarre thread a few days ago with someone playing USB RAID5 on a Raspberry Pi (which is just insane), complaining that the regular scrubs take way too much time (that's since you're using the most crappy RAID platform possible) and reported switching to scrubbing the RAID only every 6 months would be the solution (and this is not just insane but plain crazy).


    I asked whether we can prevent using RAID with those SBC toys or at least implement very annoying nag dialogs asking people whether they know what they do but other OMV devs disagree. So it's an easy decision for me to stop encouraging OMV users doing stupid/dangerous things.


    That said: there are 24 dedicated and highly optimized OMV images available for ~30 ARM boards that are all WAY BETTER for the NAS use case than any Raspberry Pi (here and there) and in the future situation will even improve


    The current official OMV image for RPi 2 and 3 OMV_3_0_88_RaspberryPi_2_3_4.9.41.img.xz still is there, works and is confirmed to work with OMV4 too (personally tested to upgrade from OMV 3 to 4 with this image several times, all minor glitches found fixed in the meantime). It receives upgrades for the next couple of years (kernel/bootloader from RPi repos, OS from upstream Debian repos, OMV from its own repo) but contains one little bug I (and most probably no one else) won't fix any more (the missing 'exit 0' in one file).

  • And a quick tutorial how to use more than one disk connected to a Raspberry Pi (applies more or less to most other ARM boards too). What you should never do if you want to attach more than one disk to your RPi is to make use of RAID. RAID is no data protection, no backup, it's really just about availability and in this special case with 'toy grade hardware' and all disks behind one single internal USB hub it's just a laughable try to increase availability.


    So instead of setting up an USB RAID5 (which will fail anyway sooner or later and will get corrupted even with minor USB connectivity issues) the way better alternative is to use btrfs (since able to provide data integrity) and its linear mode if it's about adding the capacity of two or more disks. Unfortunately this requires using the command line so create a new user if not already done who is member of the 'sudo' and 'ssh' groups, login via SSH remotely and become root using 'sudo -s'.


    Given we have 2 disks (sda and sdb) the following will create a sane btrfs linear setup:

    Code
    parted -s /dev/sda mklabel gpt
    parted -s /dev/sdb mklabel gpt
    partprobe
    mkfs.btrfs -f -d single -m raid1 -L BTRFS_LINEAR /dev/sd[ab]

    The first two commands wipe out partitions on both disks, then the kernel reads in the (now non-existent) partition table and finally we create a btrfs 'multi device' setup storing data in a linear fashion (starting with sda and if full then using sdb) taking care that metadata (filesystem structures as well as checksums for all stored data!) are spread over both disks so even if one disk is affected by data corruption we are able to spot this since we have two copies of every metadata chunk.


    Now the tricky part: we created one multi device btrfs setup consisting of both sda and sdb. To be able to use it we can either mount sda or sdb, it really doesn't matter since the btrfs code searches for remaining disks on its own. So with our setup we now simply use OMV UI to mount sda (ignoring sdb!) and then it looks like this:

    Afterwards it looks like this:


    Code
    root@raspberrypi:~# lsblk -o NAME,FSTYPE,SIZE,MOUNTPOINT,LABEL /dev/sda
    NAME FSTYPE SIZE MOUNTPOINT LABEL
    sda btrfs 465.8G /srv/dev-disk-by-label-BTRFS_LINEAR BTRFS_LINEAR
    root@raspberrypi:~# df -h | grep '/dev/sd'
    /dev/sda 578G 428M 574G 1% /srv/dev-disk-by-label-BTRFS_LINEAR

    While lsblk doesn't get the btrfs internals (multi disk setup), df and also OMV happily reports the real size. So now it's simply creating a new share using this device and we're already done:



    When we now write to or read from this btrfs device accessing data always means touching just a single disk but metadata (checksums) get written/read from all disks. It then looks like this for example when we copy something to the NAS ('iostat 5' output):



    Code
    avg-cpu: %user %nice %system %iowait %steal %idle
    0.28 0.00 8.04 8.20 0.00 83.48
    Device: tps kB_read/s kB_wrtn/s kB_read kB_wrtn
    mmcblk0 0.00 0.00 0.00 0 0
    sda 94.20 0.00 10721.60 0 53608
    sdb 0.40 0.00 38.40 0 192

    (since sda is almost empty all writes go to this disk while only some metadata chunks arrive in RAID-1 mode at sdb. Once sda is full data will be sent to sdb and so on -- we could've done this with as much disks as physically possible).



    But now the interesting part and why this is worth a try: As already explained above, btrfs creates checksums (sort of) for all data that will be written. When reading data later these checksums will be calculated again, the original metadata will be read from disk too and in case a mismatch happens (data corruption detected!) this will be logged (dmesg and syslog).


    But instead of detecting data corruption only by accident we can also use regular scrubs to check for this. I relied on Marc Merlin's scrub script (but without the lockfile stuff, see here for cleaned up script code) and let this run once a month from crontab redirecting the output to a logfile). Script output looks then like this (with an almost empty device and no errors):


    So in case errors pop up you know that data corruption happened (only alternative to this: ZFS. But that's not an option on weak Raspberry Pis!).



    Evil me now disconnected sdb (where currently only metadata is stored) and let the scrub run again. Produces of course an awful lot of errors but all the data remaining at sda is still accessible without any problems. And in such a situation when a disk is missing that (yet) does not contain data (only metadata in raid1 topology) then btrfs will even be able to recover from this when the missing disk is added back and a new scrub runs: https://pastebin.com/8jVTYchH

  • Hello. Is there any way to get a raspimon utility without OMV? It's a really handy tool, but OMV for Pi in its current state is not exactly my cup of tea, as its networking features are lacking for my goal. Also loading not so fast Pi with the GUI I can perfectly live without is not a best idea imho. But simple utility is a really really neat addition to any Pi build.

  • Hallo,


    i have installed the OMV_3_0_88_RaspberryPi_2_3_4.9.4 image on a Raspberry Pi 2. Im trying to install mine WiFi dongle with chip 7601.
    Maybe i missed the part where the solution for mine problem has been posted, please direct me to it.
    nmtui-connect got the same error message as flmaxey.
    From mine experience with raspian jessie i had to connect the dongle and it appears in the interface list
    ifconfig -a
    Here is that not the case
    Im new in Linux and Raspberry.
    I would like post the raport from armbianmonitor, but got an response that sounds "500 internal Server error"
    Any help appreciate. Thanks in advance.

  • First of all sorry for my bad english


    I have read all posts about gigabit adapter for rpi3 and when I found this Searching for compatible OMV Raspberry Pi 3 network adapater I came here where you said members can find support for ASIX AX88179 chipset
    I read all the thread but I didn't found ho to install drivers
    The usb/gigabit adapter is not listed in ifconfig
    I have installed the latest offcial image for rpi3 OMV_3_0_88_RaspberryPi_2_3_4.9.41.img
    But I don't see any restart after firtst installation on boot
    I have waited more than 20 minutes
    What I'm loosing
    Thanks for help

  • On the first try, I obeyed the on-screen instructions and tried to log in with user admin, 17 times. After that I followed the instructions in the Readme on sourceforge, logged in with root, waited 8 minutes for the auto-reboot and it worked perfectly fine, by remote, from the web interface of a different computer (probably what it is supposed to do).


    and some notes
    # Next step, installing the ASIX AX88179 or Realtek USB gigabit is really important for NAS throughput, albeit limited to 40 megaBytes per second on the pi's USB2 (range 17 to 26MB/s if also accessing hard drive).
    # Little tweaks such as SD card to 80, disable usb autosuspend (to keep the drive connected), add io_is_busy to the governor (switch to high speed during io) and disable+powerdown the radios (cool off the pi3), could help incrementally.
    #It may be necessary to manually alter polling settings for individual USB hard drives (if the drive has greatest priority, then write-flush pauses are eliminated).
    # For NTFS3, the tweaks that Synology made, such as enable large block writes, noatime, and disable indexing of ntfs volumes, could make a 4x difference in speed (mainly due to cpu loading difference).

  • Working off my list, post 98, here's #1: gigabit

    The usb/gigabit adapter is not listed in ifconfig. . .

    First, use the onboard port during a new install, login, at the console, as root, openmediavault, wait for the busy indicator led to stop flashing and the auto restart to occur.


    I have a random realtek gigabit, and it didn't work either. Because, eth1 was not in /etc/network/interfaces.
    Note: for "stretch" or newer distros, issue the command "sudo systemctl enable networking" to bypass dhcpcd before editing /etc/network/interfaces.


    ifconfig write down your router/gateway number
    sudo nano /etc/network/interfaces
    #
    auto lo
    iface lo inet loopback


    allow-hotplug eth0
    iface eth0 inet dhcp


    allow-hotplug eth1
    iface eth1 inet static
    address 192.168.1.250
    network 192.168.1.0
    netmask 255.255.255.0
    broadcast 192.168.1.255
    gateway 192.168.1.1
    dns-nameservers 192.168.1.1


    edit the 3rd digit (x.x.?.x) to match your router
    ctrl-0, ctrl-x, then plug in the gigabit adapter, reboot
    find it at 192.168.?.250 in your web browser, and there the login is admin, openmediavault


    P.S.
    Speed of gigabit to samba shared ramdisk aboard the pi = 40 megabytes per second read and write. <however> The Pi has only 1 internal usb2 hub; and, so, sum total bandwidth (probably 50) is divided by 2 in the case of both ethernet and a hard drive plugged in. When running 2 usb devices simultaneously (hd+gigabit) 25 megabytes per second is theoretically possible; however, what you can expect is 20 megabytes per second nas throughput (as tested with large files).

  • Working off my list, post 98, here's #2: SD speed.
    There are a few slow defaults, which are from upstream (so not the fault of OMV at all).


    1
    Defaults for the cpu governor aren't scaled to fit the Pi, resulting in the slowest cpu speed. Fortunately, there's a page on the topic. https://obihoernchen.net/1235/…-xu4-with-openmediavault/ Scroll down to CPU governor. The "CPU4" lines aren't needed on the Pi. Values for the Pi are:
    devices/system/cpu/cpu0/cpufreq/ondemand/io_is_busy = 1
    devices/system/cpu/cpu0/cpufreq/ondemand/sampling_down_factor = 120
    devices/system/cpu/cpu0/cpufreq/ondemand/sampling_rate = 100000
    devices/system/cpu/cpu0/cpufreq/ondemand/up_threshold = 30


    2
    The default hard drive cache tuning didn't suit me so I added the following at the end of /etc/sysctl.conf
    vm.dirty_expire_centisecs = 0


    3
    The default SD card reader speed is set on slowest, historically to support cards with now-discontinued specs. https://www.raspberrypi.org/fo…ewtopic.php?f=63&t=140461 Even my worn sandisk can do 80. On the OpenMediaVault image, the usual dtparam=sd_overclock=80 has no effect. Fortunately, the older command does work:
    In /boot/config.txt
    dtoverlay=sdhost,overclock_50=80
    # I actually set mine to 90, but that did require a better SD card.


    4
    Fstab options. It takes a spot of care because a typo could prevent startup.
    for the ext4 defaults,noatime,nodiratime,commit=600
    for the vfat defaults,noatime



    P.S.
    And then last on my list, NTFS: We probably shouldn't use NTFS on linux, but my drive was already full of content. That article https://obihoernchen.net/1235/…-xu4-with-openmediavault/ also has the ntfs mount information. In /etc/fstab, I used this: UUID=driveidhere /mnt/usbhd ntfs-3g rw,auto,user,noatime,async,big_writes,dmask=0000,fmask=0000,uid=1000 0 0
    And also added ntfs ntfs-3g to the prunefs list in /etc/updatedb.conf to prevent indexing of the USB external ntfs hard drive.

  • what you can expect is 20 megabytes per second nas throughput (as tested with large files)

    So instead of wasting your time with a Raspberry Pi better look for any of the dirt cheap and far better alternatives that allow for almost 40 MB/s (or buy a slightly more expensive ARM board and then enjoy 'PC NAS performance' with a consumption not exceeding that of RPi + external network adapter that much. See this thread for details)


    Working off my list, post 98, here's #2: SD speed. And its a 4-parter.

    I can't recommend following any of your points since

    • Useless tweaks, the OMV image takes care of cpufreq governor settings in an appropriate way (/etc/defaults/cpufrequtils and /etc/init.d/armhwinfo do the job)
    • 'hard drive cache tuning doesn't support SD cards'. So what? Data is on attached USB disks anyway?
    • 'The default SD card reader speed is set on slowest' for a reason: Data corruption is nothing anyone wants and the main problem with SD cards is neither their sequential speed nor that access times are too high but that majority of people isn't aware that random IO is far more important for SD cards in computers and that there's a huge counterfeit problem with faked SD cards.
    • 'Fstab options' -- you obvisouly missed that there's the flashmemory plugin active (fiddling around with /var/log in fstab is then a really bad thing). You're right wrt 'noatime,nodiratime,commit=600' but with flashmemory plugin it didn't make much of a difference (I checked it with 'armbianmonitor -d mmcblk0p2') so I chose to stay with Raspbian defaults since I didn't know whether any of the packages we get from archive.raspberrypi.org messes with fstab
    • NTFS is not recommended to be used with OMV (no POSIX filesystem so you run in a lot of hassles anyway, poor performance being one of the minor ones)


    For anyone being concerned about 'OMV performance' on his Raspberry Pi there is an excellent solution for this problem: Simply throw your Raspberry Pi away or do something different with it since it's the most lousy device for the job (due to its lack of decent networking and everything behind one single slow USB2 connection there). Better alternatives are countless and a lot of them cheaper anyway. Just choose the appropriate hardware, don't fiddle around with defaults and you're done.

  • So instead of wasting your time with a Raspberry Pi better look for any of the dirt cheap and far better alternatives that allow for almost 40 MB/s (or buy a slightly more expensive ARM board and then enjoy 'PC NAS performance' with a consumption not exceeding that of RPi + external network adapter that much. See this thread for details).

    Good info there! The gigabit equipped NanoPi Neo2 costs even less than a usb gigabit adapter, as well as doing NAS duties at least twice as fast as RPi3 (even 4x faster than a stock-condition RPi3).


    So, that means it is financially advisable to return the RPI3 to its mainstream use as an educational device. Although an exception could be some 150n, 300n, 100mbit networks that run near to the same pace as the RPi3. It also suits my use as a small web server.


    I especially liked your recommend of the Helios4, because that ECC memory could prevent accumulating holes in my photos and mp3 files. Also my i5 file server wastes most of its throughput on waking up; so, instead of a high wattage device that is asleep, I'd much rather have a low wattage device that is awake.


    As far as power-savers go, I've had a wishlist item for a long time: I'd love to have the hard drive scheduled to stay awake during work hours, but allow spindown at night. Is there such a thing as a Spindown Scheduler?

  • Wrt power savings... If your disk responds to hdparm parametrization (beware of crappy USB disk enclosures that prevent this) then I usually do this via crontab. One entry every evening allowing for short idle time (spindown after 10 min idle) and another one executed in the morning on workdays with eg. 240 minutes configured.

  • Quick compatibility announcement wrt next RPi model.


    We'll see rather soon a new RPi model (most probably not called RPi 4 but a somewhat extended 3 and Zero -- I expect the new SoC to both obsolete BCM2385 and BCM2387) with Gigabit Ethernet. Of course not real/fast Ethernet but USB2 attached (I would assume they take Microchip LAN7850) if Broadcom and the 'Foundation' choose to still rely on backwards compatibility and the horribly outdated VideoCore IV platform (most probably then with a die shrink to 28nm, replacing the Cortex-A53 with A35 to better cope with RPi 3 thermal and consumption constraints).


    Preparations for the new Ethernet chip happened already a while ago: https://github.com/raspberrypi…drivers/net/usb/lan78xx.c


    So maybe my latest OMV image for RPi will boot flawlessly on this new RPi model and can then be updated to latest versions of kernel and 'firmware'. But maybe an adjusted 'firmware' is necessary to boot this new SoC (of course it's not a firmware but a closed source RTOS called ThreadX that runs on the VC4 main CPU) and then someone would need to fiddle around with latest OMV image again though I would prefer to drop support for the RPi platform then since if it's still VideoCore IV then the 'single USB2 connection between SoC and outside' limitation still applies and NAS throughput will be still very limited (18 or maybe 19 MB/s since every bit has to pass the single USB2 connection twice!).

  • @tkaiser - thanks for the info and I hope all is well with you. Thanks to your advice/posts, I will not be buying a RPi 4!


    I'd like a little help for a new project for my RPi 3. I'm going to turn it into a VPN server as I have moved from London to Sydney and I need proper BBC iplayer, Netflix, Amazon Video etc.


    What I need is a nice clean base image/build. I was thinking about using the OMV image (from this post) but I think I just want the latest vanilla Armbian / Jessie. What would you suggest?

Participate now!

Don’t have an account yet? Register yourself now and be a part of our community!