Building OMV automatically for a bunch of different ARM dev boards

    • Offizieller Beitrag

    New image: OMV_3_0_75_Odroidxu4_4.9.27.7z

    I'll try it out tonight. Took last weekend off from the forum/internet to be outside/drink beer :)

    omv 7.0.4-2 sandworm | 64 bit | 6.5 proxmox kernel

    plugins :: omvextrasorg 7.0 | kvm 7.0.10 | compose 7.1.2 | k8s 7.0-6 | cputemp 7.0 | mergerfs 7.0.3


    omv-extras.org plugins source code and issue tracker - github


    Please try ctrl-shift-R and read this before posting a question.

    Please put your OMV system details in your signature.
    Please don't PM for support... Too many PMs!

    • Offizieller Beitrag

    Seems to be working for me as well. Uploading to sourceforge now.

    omv 7.0.4-2 sandworm | 64 bit | 6.5 proxmox kernel

    plugins :: omvextrasorg 7.0 | kvm 7.0.10 | compose 7.1.2 | k8s 7.0-6 | cputemp 7.0 | mergerfs 7.0.3


    omv-extras.org plugins source code and issue tracker - github


    Please try ctrl-shift-R and read this before posting a question.

    Please put your OMV system details in your signature.
    Please don't PM for support... Too many PMs!

  • Uploading to sourceforge now.

    If it works for you both once new version is uploaded there's no reason to keep OMV_3_0_71_Odroidxu4_4.9.23.7z.


    I would suggest adding a warning that eMMC installation is currently broken and that with some UAS capable chipsets (especially in Seagate disk enclosures) problems might occur that will most probably not be resolved but can be eliminated by blacklisting -- details somewhere in https://forum.odroid.com/viewtopic.php?f=146&t=26016

  • I freshly installed the OMV_3_0_75_Odroidxu4_4.9.27 image again to explain the mounting issue.


    When I ran the OMV first time, I waited 3~4 minutes to complete the rootfs resizing process.
    After auto-rebooting, the LCD showed a long boot log scrolls and login prompt appeared within 20~30 seconds.


    I connected to Admin Web UI to mount HDDs.
    Under Storage --> File Systems menu, I could see my HDDs.


    And I could mount two HDDs



    I could see two entries on /srv directory.

    Code
    root@odroidxu4:/srv# ls -alp
    total 20
    drwxr-xr-x  5 root root    4096 May 16 09:13 ./
    drwxr-xr-x 23 root root    4096 May 15 15:17 ../
    drwxr-xr-x  4 root root    4096 May 15 03:43 dev-disk-by-id-usb-JMicron_0_RANDOM__0143833EEF5D-0-0-part1/
    drwxr-xr-x  6 root root    4096 May 15 06:28 dev-disk-by-id-usb-JMicron_1_RANDOM__0143833EEF5D-0-1-part1/

    After shutdown and rebooting, the booting time was over 3~4 minutes due to this process. (from serial console)



    I have to mount the disks manually again since the HDDs were not mounted automatically.



    After mounting two disks, I can see two additional entries in /srv directory.


    Code
    root@odroidxu4:/srv# ls -alp
    total 28
    drwxr-xr-x  7 root root    4096 May 16 09:30 ./
    drwxr-xr-x 23 root root    4096 May 15 15:17 ../
    drwx------  2 root root    4096 May 16 09:13 dev-disk-by-id-usb-JMicron_0_RANDOM__0143833EEF5D-0-0-part1/
    drwxr-xr-x  4 root root    4096 May 15 03:43 dev-disk-by-id-usb-JMicron_0_RANDOM__1ACDEBB6A084-0-0-part1/
    drwx------  2 root root    4096 May 16 09:13 dev-disk-by-id-usb-JMicron_1_RANDOM__0143833EEF5D-0-1-part1/
    drwxr-xr-x  6 root root    4096 May 15 06:28 dev-disk-by-id-usb-JMicron_1_RANDOM__1ACDEBB6A084-0-1-part1/


    Every time I rebooted, the number of disk entries are increasing and the booting time also increasing significantly.
    I must mount the disks and setup the SMB configurations every time.


    @ryecoaaron
    Can you reproduce this issue with your CloudShell2?

  • After mounting two disks, I can see two additional entries in /srv directory.

    Nice, so for whatever reasons the Cloudshell 2 messes up with dev-disk-by-id conventions. As already suggested output from 'sudo armbianmonitor -u' might help as well as output from 'sudo blkid' (the latter two times with a reboot in between)

    • Offizieller Beitrag

    Every time I rebooted, the number of disk entries are increasing and the booting time also increasing significantly.
    I must mount the disks and setup the SMB configurations every time.



    Nice, so for whatever reasons the Cloudshell 2 messes up with dev-disk-by-id conventions.


    No. This is two problems.


    First, most (all?) arm boards fail to mount usb disks from time to time. I've always thought it was because the board boots faster than the usb controller initializes and the disk isn't ready when systemd mounts drives from fstab. rootwait and bootwait parameters help but don't fix the problem. Need to find a solution for this because it definitely is not a new issue.


    Second, EVERY time you mount a drive from the filesystems tab, it generates a new mntent entry and mount point. This has nothing to do with cloudshell. I think the mount function should check the mntent entries to see if the drive already exists before creating new entries. I will discuss this (and possibly submit a pull request to do this) with Volker.

    omv 7.0.4-2 sandworm | 64 bit | 6.5 proxmox kernel

    plugins :: omvextrasorg 7.0 | kvm 7.0.10 | compose 7.1.2 | k8s 7.0-6 | cputemp 7.0 | mergerfs 7.0.3


    omv-extras.org plugins source code and issue tracker - github


    Please try ctrl-shift-R and read this before posting a question.

    Please put your OMV system details in your signature.
    Please don't PM for support... Too many PMs!

  • Thank you for the kind help.
    Here is an armbian-monitor dump.
    http://sprunge.us/ZcYN


    Code
    root@odroidxu4:~# sudo blkid
    /dev/mmcblk1p1: UUID="ec72f535-bb76-49e0-afbc-aa0c2f7ef2e8" TYPE="ext4" PARTUUID="b49c7a50-01"
    /dev/sda1: UUID="da440d9b-b04b-4b6d-8d6b-1052ad574d95" TYPE="ext4" PARTUUID="98bca8e0-7fe8-4fb4-a4fc-bee393d7eb85"
    /dev/sdb1: UUID="40bf4f59-e7e7-4c8b-864e-c259773930aa" TYPE="ext4" PARTUUID="f049d01f-9e48-40d9-bca3-8cd153fc0d2c"
    /dev/mmcblk1: PTUUID="b49c7a50" PTTYPE="dos"
    /dev/mmcblk1p2: PARTUUID="b49c7a50-02"


    I think CloudShell2 should use two JMS567 chips instead of stupid JMS561 even JMS561 has slightly faster hardware RAID. :(

  • First, most (all?) arm boards fail to mount usb disks from time to time. I've always thought it was because the board boots faster than the usb controller initializes and the disk isn't ready when systemd mounts drives from fstab. rootwait and bootwait parameters help but don't fix the problem. Need to find a solution for this because it definitely is not a new issue.

    Probably not on ODROID-XU4 at least.
    I've been using the original Cloudshell over 12 months but I've never met the mounting issue.
    Anyway, I will try rootwait and bootwait options on CloudShell2 tomorrow.

  • Second, EVERY time you mount a drive from the filesystems tab, it generates a new mntent entry and mount point.

    IMO you should start to use PARTUUID for this in OMV since it's there for exactly that reason (being unique!), please see also over there: https://forum.odroid.com/viewt…t=26016&start=150#p190399


    Then I NEVER observed any USB (or SATA) mounting issues within the last 2 years working with/on Armbian (nor did we get such reports). If such things would occur this would be terrible since we encourage users to move rootfs to USB or SATA disks (on those hosts that feature native SATA). If done correctly there's no problem at all.


    Just checked https://forum.armbian.com/inde…ges-for-sbc-with-armbian/ again -- within the last 6 weeks i tested on at least 10 different SBC all the time with USB or SATA mounts and did not incur a single problem.


    The Cloudshell problem above seems to be related to hiding the drive's serials (maybe dependant on RAID mode set with JMS561 -- don't know since I would never ever buy such unreliable hardware RAID implementations)

  • You are right. The serial numbers are changing on every boot randomly even I use the JBOD/PM mode.


    Serial number: RANDOM__9A8087A021D5


    Is there any workaround on OMV side?


    I will also report the issue to CloudShell2 developer to check if there is any firmware update for JMS561 or not.

  • Is there any workaround on OMV side?

    Sure, a workaround would try to detect the string 'RANDOM' to try to deal with an IMO broken firmware on a hardware RAID controller (just to fail again in a few months with another such thing), a real fix would be to start using PARTUUID instead of serial (use blkid output, get '98bca8e0-7fe8-4fb4-a4fc-bee393d7eb85' and construct /srv/dev-disk-by-id-usb-JMicron_0_98bca8e0_7fe8_4fb4_a4fc_bee393d7eb85 for the mountpoint since drive's serial known to be unreliable when using fancy hardware raid controllers)

    • Offizieller Beitrag

    I'd swear you guys think I make this shit up...


    Probably not on ODROID-XU4 at least.

    Search the forum. The xu4 is not immune to this problem.


    Then I NEVER observed any USB (or SATA) mounting issues within the last 2 years working with/on Armbian (nor did we get such reports). If such things would occur this would be terrible since we encourage users to move rootfs to USB or SATA disks (on those hosts that feature native SATA). If done correctly there's no problem at all.

    That's because armbian is perfect... Do some searching. This is not OMV specific. Pretty sure I have read about it on arch's forum as well. The less rebooting you do, the better. Some users may see the issue more when they power the arm board and usb drive at the same time. It might be more common on some usb controllers. I don't have many usb to sata controllers and I rarely reboot the ones I have.


    IMO you should start to use PARTUUID for this in OMV since it's there for exactly that reason (being unique!),

    If you look at the commits in the last couple of months, you will see why Volker moved away from using UUIDs that OMV has used for years. They aren't unique when you start dealing with LVM snapshots. Also, zfs doesn't have UUIDs.

    omv 7.0.4-2 sandworm | 64 bit | 6.5 proxmox kernel

    plugins :: omvextrasorg 7.0 | kvm 7.0.10 | compose 7.1.2 | k8s 7.0-6 | cputemp 7.0 | mergerfs 7.0.3


    omv-extras.org plugins source code and issue tracker - github


    Please try ctrl-shift-R and read this before posting a question.

    Please put your OMV system details in your signature.
    Please don't PM for support... Too many PMs!

    • Offizieller Beitrag

    It seems Hardkernel should contact JMicron people soon to make a way to obtain a fixed USB serial number. Right?

    In my opinion, yes.

    omv 7.0.4-2 sandworm | 64 bit | 6.5 proxmox kernel

    plugins :: omvextrasorg 7.0 | kvm 7.0.10 | compose 7.1.2 | k8s 7.0-6 | cputemp 7.0 | mergerfs 7.0.3


    omv-extras.org plugins source code and issue tracker - github


    Please try ctrl-shift-R and read this before posting a question.

    Please put your OMV system details in your signature.
    Please don't PM for support... Too many PMs!

  • That's because armbian is perfect

    Nope, but Armbian is not a static OS image but an actively maintained distro + build system. From time to time such issues occur but they get fixed as soon as the root cause is identified (eg. we had that with USB disks and kernel 4.3 on sun7i devices -- Allwinner A20 based like Banana Pi for example -- and fix was part of 4.4.1 kernel package provided via apt.armbian.com later or we had something similar with legacy kernel 3.4.112 at the same time where CONFIG_CHR_DEV_SG was not set).


    That's why 'armbianmonitor -u' exists -- to nail problems down and to avoid what's usually happening when people use unmaintained/static OS images: reports of random behaviour no one seems to be able to explain.


    Wrt PARTUUID: I didn't know that OMV moved away from this just recently for reasons so we can consider Cloudshell 2 without a firmware fix 'broken hardware' for now. Is adding Cloudshell partitions (sda1/sdb1) manually to fstab a sufficient workaround for now or does this cause other troubles?

    • Offizieller Beitrag

    Nope, but Armbian is not a static OS image but an actively maintained distro + build system. From time to time such issues occur but they get fixed as soon as the root cause is identified

    I am well aware of that. What magic does armbian to do make sure fstab entries are mounted on boot that straight debian does not?



    That's why 'armbianmonitor -u' exists -- to nail problems down and to avoid what's usually happening when people use unmaintained/static OS images: reports of random behaviour no one seems to be able to explain.

    Have you seen the report tab (Diagnostics -> System Information -> Report) in the OMV web interface? It allows you download a file of important info to post on this forum. Allows a user to remove any info they don't want to post.


    Is adding Cloudshell partitions (sda1/sdb1) manually to fstab a sufficient workaround for now or does this cause other troubles?

    No this doesn't work because you need mntent entries to use in the OMV web interface.

    omv 7.0.4-2 sandworm | 64 bit | 6.5 proxmox kernel

    plugins :: omvextrasorg 7.0 | kvm 7.0.10 | compose 7.1.2 | k8s 7.0-6 | cputemp 7.0 | mergerfs 7.0.3


    omv-extras.org plugins source code and issue tracker - github


    Please try ctrl-shift-R and read this before posting a question.

    Please put your OMV system details in your signature.
    Please don't PM for support... Too many PMs!

  • What magic does armbian to do make sure fstab entries are mounted on boot that straight debian does not?

    Well, we switched to PARTUUID for choosing the boot/root device (single partition scheme for different reasons) and then sg as module or not is always the first thing to check (dmesg output and then adjusting kernel config regarding CONFIG_CHR_DEV_SG). Maybe reading from here on gives the idea: https://forum.armbian.com/inde…=findComment&comment=4111 ?


    With inappropriate CONFIG_CHR_DEV_SG definition mount behaviour will be random/erratic while with correct setting it just works.

    • Offizieller Beitrag

    With inappropriate CONFIG_CHR_DEV_SG definition mount behaviour will be random/erratic while with correct setting it just works.

    Thanks for the info. Why would it need to be compiled in for a data drive though? I can see it needing to be compiled for the OS drive but I would think a module would work just fine for a data drive.

    omv 7.0.4-2 sandworm | 64 bit | 6.5 proxmox kernel

    plugins :: omvextrasorg 7.0 | kvm 7.0.10 | compose 7.1.2 | k8s 7.0-6 | cputemp 7.0 | mergerfs 7.0.3


    omv-extras.org plugins source code and issue tracker - github


    Please try ctrl-shift-R and read this before posting a question.

    Please put your OMV system details in your signature.
    Please don't PM for support... Too many PMs!

Jetzt mitmachen!

Sie haben noch kein Benutzerkonto auf unserer Seite? Registrieren Sie sich kostenlos und nehmen Sie an unserer Community teil!