Building OMV automatically for a bunch of different ARM dev boards

    This site uses cookies. By continuing to browse this site, you are agreeing to our Cookie Policy.

    • tkaiser wrote:


      Another thing I think should be disabled by default on all arm boards is Monitoring. It uses cpu/ram, writes a lot, and most arm users probably don't need the graphing. I disable it on all of the images I have created.
      I was talking about the stuff Hardkernel is now using testing with their Cloudshell2:
      Yep, those parameters would probably work well on all arm boards. I do have a cloudshell2 to test on now and I have a native Windows 10 box to test with.

      tkaiser wrote:

      Is it possible with OMV to tweak smb.conf's global section without risking to loose the changes with an update?
      Yep. Put the options in the extra options box on the smb/cifs settings tab. This will survive updates.
      omv 4.0.6 arrakis | 64 bit | 4.12 backports kernel | omvextrasorg 4.1.0
      omv-extras.org plugins source code and issue tracker - github.com/OpenMediaVault-Plugin-Developers

      Please don't PM for support... Too many PMs!
    • Just a quick cloudshell2 test using my xu4 image (older 3.10 kernel) on OMV 3.0.70 to one 1TB seagate writing 3 GB ISOs.

      No optimization = ~70 MB/s
      Hardkernel optimizations = ~ 80 MB/s
      arm-blog optimizations = ~76 MB/s


      Drive benchmark

      Source Code

      1. # dd if=/dev/zero of=/srv/dev-disk-by-label-test/zero.dd bs=1M count=10000 conv=fdatasync && sync
      2. 10000+0 records in
      3. 10000+0 records out
      4. 10485760000 bytes (10 GB) copied, 82.2036 s, 128 MB/s
      Images
      • FullSizeRender.jpg

        71.85 kB, 640×461, viewed 116 times
      • IMG_7888.JPG

        109.78 kB, 640×480, viewed 171 times
      omv 4.0.6 arrakis | 64 bit | 4.12 backports kernel | omvextrasorg 4.1.0
      omv-extras.org plugins source code and issue tracker - github.com/OpenMediaVault-Plugin-Developers

      Please don't PM for support... Too many PMs!

      The post was edited 1 time, last by ryecoaaron ().

    • Thanks for info about smb.conf tweaking and also performance numbers -- I agree that values can be shared between all ARM boards.

      Would be interesting if you can give the mainline/UAS capable OMV image a try and test again since my Netatalk numbers varied a lot between both variants: forum.armbian.com/index.php?/t…findComment&comment=29304
      'OMV problems' with XU4 and Cloudshell 2? Nope, read this first. 'OMV problems' with Cloudshell 1? Nope, just Ohm's law or queue size.
    • tkaiser wrote:

      Would be interesting if you can give the mainline/UAS capable OMV image a try
      Yep, that is in the plans. Just need to find the time :)
      omv 4.0.6 arrakis | 64 bit | 4.12 backports kernel | omvextrasorg 4.1.0
      omv-extras.org plugins source code and issue tracker - github.com/OpenMediaVault-Plugin-Developers

      Please don't PM for support... Too many PMs!
    • tkaiser wrote:


      My real Samba use case is preparing the transition from AFP (Netatalk) to SMB (Samba) but that requires a Samba version recent enough and some special settings in smb.conf's global section. I'll postpone discussing this until Stretch is released (Samba 4.5.8 looks sufficient)

      A comment about this. This new patch was mentioned by someone else at this forum if Omv could integrate this. Is kind of difficult to maintain, if Debian does it should be enough. But also shouldn't be difficult just for someone to build from Debian source applying the patch.
      I did build samba from Debian source in stretch applying the patch and the results where no good. macOS Sierra seees the tm volume but doesn't connect. Wire shark displayed the required properties however.
      Someone else at the github samba where this patch was submitted built a docker container using samba build from git. This works as expected, so I don't know why ATM seems to be a problem using the samba version from stretch and this patch. Latest samba shouldn't be a problem.
      chat support at #openmediavault@freenode IRC | Spanish & English | GMT+10
      telegram.me/openmediavault broadcast channel
      openmediavault discord server
    • subzero79 wrote:

      macOS Sierra seees the tm volume but doesn't connect
      Well, TM will be the last thing I look into (since it works still with AFP/Netatalk). My main concerns with Samba for macOS clients were performance (almost resolved) and are reliability (I let the graphics department at a customer do a lot of nasty things and shares began to fail -- this is something we'll look into the next weeks/months).

      For the moment I'll try to focus here on improving OMV image creation for ARM boards. Status: for me it looks already good after following a lot of @ryecoaaron's advices and tweaking settings (especially on ODROID-XU4 with mainline kernel). Also I wouldn't fear running OMV from flash after adoption of the folder2ram approach but since the OMV images are all based on Armbian it's easy for users to move their installation to a connected disk (burn the image to a slow 4GB SD card, start it once, call 'nand-sata-install' and select the target disk. From then on the SD card will be read-only and is only required since it stores bootloader+initrd but everything else lives on the disk)

      But I think @ryecoaaron is right and disabling a few more services by default doesn't hurt if users can easily switch stopped daemons/functionality later on again.

      So what are the best places to provide OMV images with stopped 'monitoring'? Just /etc/openmediavault/config.xml or somewhere else? And how to pre-fill the 'extra options box on the smb/cifs settings tab'?
      'OMV problems' with XU4 and Cloudshell 2? Nope, read this first. 'OMV problems' with Cloudshell 1? Nope, just Ohm's law or queue size.
    • tkaiser wrote:

      So what are the best places to provide OMV images with stopped 'monitoring'? Just /etc/openmediavault/config.xml or somewhere else
      images? Or are you asking how to disable monitoring?

      tkaiser wrote:

      And how to pre-fill the 'extra options box on the smb/cifs settings tab'?
      I would use xmlstarlet.

      Shell-Script

      1. #!/bin/bash
      2. . /etc/default/openmediavault
      3. . /usr/share/openmediavault/scripts/helper-functions
      4. xmlstarlet ed -L -u "/config/services/smb/extraoptions" -v "${value}" ${OMV_CONFIG_FILE}
      omv 4.0.6 arrakis | 64 bit | 4.12 backports kernel | omvextrasorg 4.1.0
      omv-extras.org plugins source code and issue tracker - github.com/OpenMediaVault-Plugin-Developers

      Please don't PM for support... Too many PMs!
    • tkaiser wrote:

      Yep, sorry for being not precise
      Disabling monitoring might be better performed with an rpc call. I will look into this.
      omv 4.0.6 arrakis | 64 bit | 4.12 backports kernel | omvextrasorg 4.1.0
      omv-extras.org plugins source code and issue tracker - github.com/OpenMediaVault-Plugin-Developers

      Please don't PM for support... Too many PMs!
    • @tkaiser

      Here is a way to disable monitoring:

      omv-rpc -u admin "perfstats" "set" '{"enable":false}'
      omv-rpc -u admin "config" "applyChanges" '{ "modules": ["monit","rrdcached","collectd"],"force": true }'
      omv 4.0.6 arrakis | 64 bit | 4.12 backports kernel | omvextrasorg 4.1.0
      omv-extras.org plugins source code and issue tracker - github.com/OpenMediaVault-Plugin-Developers

      Please don't PM for support... Too many PMs!
    • tkaiser wrote:

      hope you get back to me in the meantime with some XU4 results with mainline kernel and UAS
      I started working on it and ran into a couple of issues. One issue was that the cloudshell2 module uses enough cpu to make the load 1.0 all the time. Once I rmmod'd it, the load went to normal. I want to re-write the image to make sure that I didn't alter anything else. Maybe tonight.

      I do know that just changing to the armbian image made the dd speed test increase by 20MB/s. We will see if samba increases proportionally.
      omv 4.0.6 arrakis | 64 bit | 4.12 backports kernel | omvextrasorg 4.1.0
      omv-extras.org plugins source code and issue tracker - github.com/OpenMediaVault-Plugin-Developers

      Please don't PM for support... Too many PMs!
    • @tkaiser
      @odroidq

      Here are some samba write numbers using the cloud shell2 and the armbian image with the 4.9 kernel. Lets just say the armbian image and the 4.9 kernel with the hardkernel samba tuning is fast! The images will speak for themselves but basically the xu4 is saturating gigabit over samba :)
      Images
      • armbian_4.9_samba_hardkernel_settings.jpg

        32.02 kB, 442×236, viewed 83 times
      • armbian_4.9_samba_armblog_settings.jpg

        34.3 kB, 443×234, viewed 82 times
      omv 4.0.6 arrakis | 64 bit | 4.12 backports kernel | omvextrasorg 4.1.0
      omv-extras.org plugins source code and issue tracker - github.com/OpenMediaVault-Plugin-Developers

      Please don't PM for support... Too many PMs!
    • ryecoaaron wrote:

      Source Code

      1. echo performance >/sys/devices/system/cpu/cpu0/cpufreq/scaling_governor


      The images will speak for themselves but basically the xu4 is saturating gigabit over samba
      Thank you for the numbers though I'm not satisfied ;)

      I did some more tests in the meantime (quite surprising stuff comparing performance of little and big cluster) and am wondering whether you get better results (using Hardkernel's Samba settings) when executing this before to keep the little cores at 1.3 GHz all the time:

      Source Code

      1. echo performance >/sys/devices/system/cpu/cpu0/cpufreq/scaling_governor
      (112 MB/sec is the goal to reach ;) )

      To be honest: I would also love to see some LanTest numbers since LanTest also tests other stuff like latency and in a slightly different way that's a lot more interesting than Explorer performance.

      Explanation: helios.de/web/EN/support/TI/157.html
      Tool: webshare.helios.de (user/pwd tools/tools) and then
      'OMV problems' with XU4 and Cloudshell 2? Nope, read this first. 'OMV problems' with Cloudshell 1? Nope, just Ohm's law or queue size.
    • odroidq wrote:

      BTW, can you tell me which driver module utilizes the CPU load?
      Please show me "lsmod" when the driver module uses CPU too much.
      I think a useless touchscreen driver might be a root cause.
      It was definitely the touchscreen driver. I can't remember the name now (maybe had 4765 in it??) and the system is off. I will update later.

      tkaiser wrote:

      I'm not satisfied
      Somehow I'm not surprised :)

      tkaiser wrote:

      when executing this before to keep the little cores at 1.3 GHz all the time:
      I will try this but I think the pursuit of performance is getting away from why people use arm boards in the first place - energy use. I was disappointed to see the xu4, cloudshell2, and two Seagate 1TB barracudas using 24 watts at idle (drives spinning and no I didn't spin them down). The cloudshell2 and drives spinning without the xu4 were using 14 watts.

      tkaiser wrote:

      (112 MB/sec is the goal to reach )
      My test setup probably would have reached/exceeded that goal if I wasn't connected to the headless Windows box via VNC.

      tkaiser wrote:

      To be honest: I would also love to see some LanTest numbers since LanTest also tests other stuff like latency and in a slightly different way that's a lot more interesting than Explorer performance.
      I think those numbers are pointless if it doesn't "feel" fast. When I look at the transfer speed and it is staying at 110 MB/s for most of the time, that is good enough for me. Someone else will have to run lantest since I used all of my free time doing these tests.
      omv 4.0.6 arrakis | 64 bit | 4.12 backports kernel | omvextrasorg 4.1.0
      omv-extras.org plugins source code and issue tracker - github.com/OpenMediaVault-Plugin-Developers

      Please don't PM for support... Too many PMs!
    • ryecoaaron wrote:

      I will try this but I think the pursuit of performance is getting away from why people use arm boards in the first place - energy use
      Sure but most if not all people don't understand what processors do anyway :)

      The 4 little cores in the Exynos SoC are Cortex-A7 (so rather energy efficient) and clock between 600 MHz and 1300 MHz. When there's nothing to do then it doesn't matter that much at which clockspeed they run. Modern ARM cores implement advanced clock gating for power saving purposes and if the processor is sitting on a WFI instruction, then the power consumption is already significantly reduced regardless of the CPU clock speed.

      I just tried it out: idle consumption increase is below 70mW (negligible) and peak consumption doesn't change for obvious reasons (since with both ondemand and performance they will finally clock up to 1300 MHz). But switching the little cluster from ondemand to performance made handling of small files more snappy so it's a win (please also compare with the 'race to idle' concept -- there are certain situations where CPU cores allowed to clock higher will end up saving more energy since entering low power states more early/often!).

      Please note that this is only a change on the A7 cores, the more power hungry A15 cores in the big cluster run still with ondemand.

      I just implemented all the changes we talked about in the way you proposed: github.com/igorpecovnik/lib/co…8f6aaf333e3f31ad0b67ecf00

      Am currently building a new OS image for XU4 (compiling kernel now with GCC 6.3.1 instead of 5.4 as before) after a little bugfix: '. /usr/share/openmediavault/scripts/helper-functions' was missing.
      'OMV problems' with XU4 and Cloudshell 2? Nope, read this first. 'OMV problems' with Cloudshell 1? Nope, just Ohm's law or queue size.
    • Let me know when the new image is ready. I will try it. What compiler optimizations are you using for gcc when compiling the kernel?
      omv 4.0.6 arrakis | 64 bit | 4.12 backports kernel | omvextrasorg 4.1.0
      omv-extras.org plugins source code and issue tracker - github.com/OpenMediaVault-Plugin-Developers

      Please don't PM for support... Too many PMs!
    • Edit: 's/performance/consumption/' in next sentence:

      BTW: I'm a bit shocked about your consumption numbers. 10W difference only due to XU4 running or not are insane.

      My idle numbers for XU4 are 3.3W (with a not so effective PSU, IIRC that's the one from my UP board) and I slightly exceed 10W when running eg. cpuminer/minerd on the big cluster. Unfortunately from all my SBC the UP board and those ODROIDs are the most power hungry in idle but if it's really 10W then there's something wrong (see also github.com/igorpecovnik/lib.do…master/docs/board_details).

      Just as a comparison: I set up yesterday an Orange Pi Zero as Armbian torrent seed node that idles at 540mW with a 128GB Samsung EVO+ (peak consumption around 1500mW due to optimized settings). Anyway: 3W idle for a board are still ok-ish but disks should be able/allowed to spin down in a controlled way (and that's where it gets interesting with this Cloudshell2 since I doubt that's possible with RAID modes)
      'OMV problems' with XU4 and Cloudshell 2? Nope, read this first. 'OMV problems' with Cloudshell 1? Nope, just Ohm's law or queue size.

      The post was edited 1 time, last by tkaiser ().

    • What are you measuring power consumption with? I've never seen any of my XU4s idle at 3.3 watts and I have four of them. The odroid-c2 might be around that. I'm using a kill-a-watt meter (have 3) but I can break out my Fluke multimeter if I need to. As a comparison, my RPi3 idles between 1.6 and 2.0 watts.
      omv 4.0.6 arrakis | 64 bit | 4.12 backports kernel | omvextrasorg 4.1.0
      omv-extras.org plugins source code and issue tracker - github.com/OpenMediaVault-Plugin-Developers

      Please don't PM for support... Too many PMs!
    • Users Online 1

      1 Guest