New approach for Raspberry Pi OMV images

    • tkaiser wrote:

      Since RPi folks in the meantime released a newer kernel maybe I bake a new image soon
      Latest questions in the forum convinced me to delete OMV_3_0_88_RaspberryPi_2_3_4.9.41.img.xz and I will delete github.com/ThomasKaiser/OMV_for_Raspberries within 24 hours (so anyone interested in maintaining OMV on crap might want to fork the repo now)
    • tkaiser wrote:

      so anyone interested in maintaining OMV on crap might want to fork the repo now
      I really don't want to maintain it but I forked it anyway.
      omv 4.1.11 arrakis | 64 bit | 4.15 proxmox kernel | omvextrasorg 4.1.11
      omv-extras.org plugins source code and issue tracker - github

      Please read this before posting a question and this and this for docker questions.
      Please don't PM for support... Too many PMs!
    • mcgyver83 wrote:

      So this means that we can trash our OMV instances and restart from scratch with official images?
      What are official images?
      omv 4.1.11 arrakis | 64 bit | 4.15 proxmox kernel | omvextrasorg 4.1.11
      omv-extras.org plugins source code and issue tracker - github

      Please read this before posting a question and this and this for docker questions.
      Please don't PM for support... Too many PMs!
    • mcgyver83 wrote:

      So this means that we can trash our OMV instances and restart from scratch with official images?
      Hey,
      no I think you can trash the RPi itself ! I'm not very good with english, but I think I've understood since a moment ago Tkaiser think the RPis are "crap" if used as a NAS (by many reasons written on this forum, 1st is USB/LAN bad performance, powerfailure, sdcard slow/broken... etc.), and now he's fed up with working on this board, if I'm not wrong.

      tkaiser wrote:

      maintaining OMV on crap
      [IMG:http://www.executive-dating.co.uk/images/Distressed_man_on_phone_365_202.jpg]


      There is a topic Which energy efficient ARM platform to choose? where @richprim ask about low power ARM platforms.
      It can be useful to find informations for finding a better hardware for Christmas gift for replacement, and being able to have good perf with OMV. (and keeping OMV compatibility hardware/software support...)


      @tkaiser I don't want to say mistakes in this post about you and your work, please fix me if I'm wrong ;)
    • petrus wrote:

      I think you can trash the RPi itself
      No, it's totally fine if you use an RPi as OMV NAS just as it's totally fine if you loose your data. If you like the combination of slow and unreliable then a Raspberry Pi as NAS is always a great choice.

      It's just that if I spend time on something I want to be part of a solution and not just a problem. And the real problem is that people seem not to understand/accept the limitations of such toys. There was a bizarre thread a few days ago with someone playing USB RAID5 on a Raspberry Pi (which is just insane), complaining that the regular scrubs take way too much time (that's since you're using the most crappy RAID platform possible) and reported switching to scrubbing the RAID only every 6 months would be the solution (and this is not just insane but plain crazy).

      I asked whether we can prevent using RAID with those SBC toys or at least implement very annoying nag dialogs asking people whether they know what they do but other OMV devs disagree. So it's an easy decision for me to stop encouraging OMV users doing stupid/dangerous things.

      That said: there are 24 dedicated and highly optimized OMV images available for ~30 ARM boards that are all WAY BETTER for the NAS use case than any Raspberry Pi (here and there) and in the future situation will even improve

      The current official OMV image for RPi 2 and 3 OMV_3_0_88_RaspberryPi_2_3_4.9.41.img.xz still is there, works and is confirmed to work with OMV4 too (personally tested to upgrade from OMV 3 to 4 with this image several times, all minor glitches found fixed in the meantime). It receives upgrades for the next couple of years (kernel/bootloader from RPi repos, OS from upstream Debian repos, OMV from its own repo) but contains one little bug I (and most probably no one else) won't fix any more (the missing 'exit 0' in one file).
    • And a quick tutorial how to use more than one disk connected to a Raspberry Pi (applies more or less to most other ARM boards too). What you should never do if you want to attach more than one disk to your RPi is to make use of RAID. RAID is no data protection, no backup, it's really just about availability and in this special case with 'toy grade hardware' and all disks behind one single internal USB hub it's just a laughable try to increase availability.

      So instead of setting up an USB RAID5 (which will fail anyway sooner or later and will get corrupted even with minor USB connectivity issues) the way better alternative is to use btrfs (since able to provide data integrity) and its linear mode if it's about adding the capacity of two or more disks. Unfortunately this requires using the command line so create a new user if not already done who is member of the 'sudo' and 'ssh' groups, login via SSH remotely and become root using 'sudo -s'.

      Given we have 2 disks (sda and sdb) the following will create a sane btrfs linear setup:

      Source Code

      1. parted -s /dev/sda mklabel gpt
      2. parted -s /dev/sdb mklabel gpt
      3. partprobe
      4. mkfs.btrfs -f -d single -m raid1 -L BTRFS_LINEAR /dev/sd[ab]
      The first two commands wipe out partitions on both disks, then the kernel reads in the (now non-existent) partition table and finally we create a btrfs 'multi device' setup storing data in a linear fashion (starting with sda and if full then using sdb) taking care that metadata (filesystem structures as well as checksums for all stored data!) are spread over both disks so even if one disk is affected by data corruption we are able to spot this since we have two copies of every metadata chunk.

      Now the tricky part: we created one multi device btrfs setup consisting of both sda and sdb. To be able to use it we can either mount sda or sdb, it really doesn't matter since the btrfs code searches for remaining disks on its own. So with our setup we now simply use OMV UI to mount sda (ignoring sdb!) and then it looks like this:
      [IMG:http://kaiser-edv.de/tmp/2kkDm0/Bildschirmfoto%202017-11-24%20um%2018.16.05.png]
      Afterwards it looks like this:

      Source Code

      1. root@raspberrypi:~# lsblk -o NAME,FSTYPE,SIZE,MOUNTPOINT,LABEL /dev/sda
      2. NAME FSTYPE SIZE MOUNTPOINT LABEL
      3. sda btrfs 465.8G /srv/dev-disk-by-label-BTRFS_LINEAR BTRFS_LINEAR
      4. root@raspberrypi:~# df -h | grep '/dev/sd'
      5. /dev/sda 578G 428M 574G 1% /srv/dev-disk-by-label-BTRFS_LINEAR
      While lsblk doesn't get the btrfs internals (multi disk setup), df and also OMV happily reports the real size. So now it's simply creating a new share using this device and we're already done:

      [IMG:http://kaiser-edv.de/tmp/2kkDm0/Bildschirmfoto%202017-11-24%20um%2018.18.13.png]
      When we now write to or read from this btrfs device accessing data always means touching just a single disk but metadata (checksums) get written/read from all disks. It then looks like this for example when we copy something to the NAS ('iostat 5' output):


      Source Code

      1. avg-cpu: %user %nice %system %iowait %steal %idle
      2. 0.28 0.00 8.04 8.20 0.00 83.48
      3. Device: tps kB_read/s kB_wrtn/s kB_read kB_wrtn
      4. mmcblk0 0.00 0.00 0.00 0 0
      5. sda 94.20 0.00 10721.60 0 53608
      6. sdb 0.40 0.00 38.40 0 192
      (since sda is almost empty all writes go to this disk while only some metadata chunks arrive in RAID-1 mode at sdb. Once sda is full data will be sent to sdb and so on -- we could've done this with as much disks as physically possible).


      But now the interesting part and why this is worth a try: As already explained above, btrfs creates checksums (sort of) for all data that will be written. When reading data later these checksums will be calculated again, the original metadata will be read from disk too and in case a mismatch happens (data corruption detected!) this will be logged (dmesg and syslog).

      But instead of detecting data corruption only by accident we can also use regular scrubs to check for this. I relied on Marc Merlin's scrub script (but without the lockfile stuff, see here for cleaned up script code) and let this run once a month from crontab redirecting the output to a logfile). Script output looks then like this (with an almost empty device and no errors):

      Source Code

      1. root@raspberrypi:~# /usr/local/bin/btrfs-scrub.sh
      2. root: Quick Metadata and Data Balance of /srv/dev-disk-by-label-BTRFS_LINEAR (/dev/sda)
      3. Done, had to relocate 1 out of 7 chunks
      4. Done, had to relocate 3 out of 6 chunks
      5. Done, had to relocate 1 out of 5 chunks
      6. Done, had to relocate 1 out of 5 chunks
      7. root: Starting scrub of /srv/dev-disk-by-label-BTRFS_LINEAR
      8. btrfs scrub start -Bd /srv/dev-disk-by-label-BTRFS_LINEAR
      9. scrub device /dev/sda (id 1) done
      10. scrub started at Fri Nov 24 18:33:15 2017 and finished after 159 seconds
      11. total bytes scrubbed: 3.09GiB with 0 errors
      12. scrub device /dev/sdb (id 2) done
      13. scrub started at Fri Nov 24 18:33:15 2017 and finished after 1 seconds
      14. total bytes scrubbed: 2.56MiB with 0 errors
      Display All
      So in case errors pop up you know that data corruption happened (only alternative to this: ZFS. But that's not an option on weak Raspberry Pis!).


      Evil me now disconnected sdb (where currently only metadata is stored) and let the scrub run again. Produces of course an awful lot of errors but all the data remaining at sda is still accessible without any problems. And in such a situation when a disk is missing that (yet) does not contain data (only metadata in raid1 topology) then btrfs will even be able to recover from this when the missing disk is added back and a new scrub runs: pastebin.com/8jVTYchH

      The post was edited 2 times, last by tkaiser ().

    • Hello. Is there any way to get a raspimon utility without OMV? It's a really handy tool, but OMV for Pi in its current state is not exactly my cup of tea, as its networking features are lacking for my goal. Also loading not so fast Pi with the GUI I can perfectly live without is not a best idea imho. But simple utility is a really really neat addition to any Pi build.
    • Hallo,

      i have installed the OMV_3_0_88_RaspberryPi_2_3_4.9.4 image on a Raspberry Pi 2. Im trying to install mine WiFi dongle with chip 7601.
      Maybe i missed the part where the solution for mine problem has been posted, please direct me to it.
      nmtui-connect got the same error message as flmaxey.
      From mine experience with raspian jessie i had to connect the dongle and it appears in the interface list
      ifconfig -a
      Here is that not the case
      Im new in Linux and Raspberry.
      I would like post the raport from armbianmonitor, but got an response that sounds "500 internal Server error"
      Any help appreciate. Thanks in advance.
    • First of all sorry for my bad english

      I have read all posts about gigabit adapter for rpi3 and when I found this Searching for compatible OMV Raspberry Pi 3 network adapater I came here where you said members can find support for ASIX AX88179 chipset
      I read all the thread but I didn't found ho to install drivers
      The usb/gigabit adapter is not listed in ifconfig
      I have installed the latest offcial image for rpi3 OMV_3_0_88_RaspberryPi_2_3_4.9.41.img
      But I don't see any restart after firtst installation on boot
      I have waited more than 20 minutes
      What I'm loosing
      Thanks for help
    • On the first try, I obeyed the on-screen instructions and tried to log in with user admin, 17 times. After that I followed the instructions in the Readme on sourceforge, logged in with root, waited 8 minutes for the auto-reboot and it worked perfectly fine, by remote, from the web interface of a different computer (probably what it is supposed to do).

      and some notes
      # Next step, installing the ASIX AX88179 or Realtek USB gigabit is really important for NAS throughput, albeit limited to 40 megaBytes per second on the pi's USB2 (range 17 to 26MB/s if also accessing hard drive).
      # Little tweaks such as SD card to 80, disable usb autosuspend (to keep the drive connected), add io_is_busy to the governor (switch to high speed during io) and disable+powerdown the radios (cool off the pi3), could help incrementally.
      #It may be necessary to manually alter polling settings for individual USB hard drives (if the drive has greatest priority, then write-flush pauses are eliminated).
      # For NTFS3, the tweaks that Synology made, such as enable large block writes, noatime, and disable indexing of ntfs volumes, could make a 4x difference in speed (mainly due to cpu loading difference).

      The post was edited 5 times, last by deluxecat ().

    • Working off my list, post 98, here's #1: gigabit

      Akkor wrote:

      The usb/gigabit adapter is not listed in ifconfig. . .
      First, use the onboard port during a new install, login, at the console, as root, openmediavault, wait for the busy indicator led to stop flashing and the auto restart to occur.

      I have a random realtek gigabit, and it didn't work either. Because, eth1 was not in /etc/network/interfaces.
      Note: for "stretch" or newer distros, issue the command "sudo systemctl enable networking" to bypass dhcpcd before editing /etc/network/interfaces.

      ifconfig write down your router/gateway number
      sudo nano /etc/network/interfaces
      #
      auto lo
      iface lo inet loopback

      allow-hotplug eth0
      iface eth0 inet dhcp

      allow-hotplug eth1
      iface eth1 inet static
      address 192.168.1.250
      network 192.168.1.0
      netmask 255.255.255.0
      broadcast 192.168.1.255
      gateway 192.168.1.1
      dns-nameservers 192.168.1.1

      edit the 3rd digit (x.x.?.x) to match your router
      ctrl-0, ctrl-x, then plug in the gigabit adapter, reboot
      find it at 192.168.?.250 in your web browser, and there the login is admin, openmediavault

      P.S.
      Speed of gigabit to samba shared ramdisk aboard the pi = 40 megabytes per second read and write. <however> The Pi has only 1 internal usb2 hub; and, so, sum total bandwidth (probably 50) is divided by 2 in the case of both ethernet and a hard drive plugged in. When running 2 usb devices simultaneously (hd+gigabit) 25 megabytes per second is theoretically possible; however, what you can expect is 20 megabytes per second nas throughput (as tested with large files).
    • Working off my list, post 98, here's #2: SD speed.
      There are a few slow defaults, which are from upstream (so not the fault of OMV at all).

      1
      Defaults for the cpu governor aren't scaled to fit the Pi, resulting in the slowest cpu speed. Fortunately, there's a page on the topic. obihoernchen.net/1235/odroid-xu4-with-openmediavault/ Scroll down to CPU governor. The "CPU4" lines aren't needed on the Pi. Values for the Pi are:
      devices/system/cpu/cpu0/cpufreq/ondemand/io_is_busy = 1
      devices/system/cpu/cpu0/cpufreq/ondemand/sampling_down_factor = 120
      devices/system/cpu/cpu0/cpufreq/ondemand/sampling_rate = 100000
      devices/system/cpu/cpu0/cpufreq/ondemand/up_threshold = 30

      2
      The default hard drive cache tuning didn't suit me so I added the following at the end of /etc/sysctl.conf
      vm.dirty_expire_centisecs = 0

      3
      The default SD card reader speed is set on slowest, historically to support cards with now-discontinued specs. raspberrypi.org/forums/viewtopic.php?f=63&t=140461 Even my worn sandisk can do 80. On the OpenMediaVault image, the usual dtparam=sd_overclock=80 has no effect. Fortunately, the older command does work:
      In /boot/config.txt
      dtoverlay=sdhost,overclock_50=80
      # I actually set mine to 90, but that did require a better SD card.

      4
      Fstab options. It takes a spot of care because a typo could prevent startup.
      for the ext4 defaults,noatime,nodiratime,commit=600
      for the vfat defaults,noatime


      P.S.
      And then last on my list, NTFS: We probably shouldn't use NTFS on linux, but my drive was already full of content. That article obihoernchen.net/1235/odroid-xu4-with-openmediavault/ also has the ntfs mount information. In /etc/fstab, I used this: UUID=driveidhere /mnt/usbhd ntfs-3g rw,auto,user,noatime,async,big_writes,dmask=0000,fmask=0000,uid=1000 0 0
      And also added ntfs ntfs-3g to the prunefs list in /etc/updatedb.conf to prevent indexing of the USB external ntfs hard drive.

      The post was edited 3 times, last by deluxecat: edited for reducing word-count ().

    • Users Online 3

      3 Guests