Installing-Migrating OMV to USB stick

    • OMV 3.x
    • Installing-Migrating OMV to USB stick

      I'm running latest version on my HP N40L Microserver from internal sata, but i'm thnking on moving the installation to a usb stick in order to save that sata port for an additional storage drive. My questions

      -What will the results be when booting off that usb stick? Like how much more loading time for OMV when booting, and other performance marks
      -Can i migrate my installation with some partition tool like AOMEI backupper or other in order to be more seamless and save me some time from performaing a clean OS installation and reseting the server
      -Other tips on improving performance and life expency of the driver
    • therock003 wrote:

      What will the results be when booting off that usb stick?
      That depends solely on the USB stick in question. The cheap and counterfeit stuff is mostly made from flash memory ripped out of recycled smartphones while you can also get quality thumb drives from reputable vendors (then only the counterfeit problem remains, that's why each and every flash media has to be tested immediately after purchase by either F3 or H2testw (this is MANDATORY, if you miss this step and got a counterfeit drive with less real capacity than faking you run into troubles for sure).

      When running off of flash media enabling OMV's flashmemory plugin is mandatory and disabling monitoring is a good idea: Add drives to an existing array, then extend the existing partition/file system?
    • Wow first of all that was pretty usefull information to have concerning the fake flash. I'll run all of my sticks and cards through that just in case. After the stick has passed the test, wether fake or not, what read/write speeds would you consider sufficient for this task? And how much space does the installation require? Now for my sata i see drives usage be at 1.31GB of used space

      Also care to comment on the migration part? IT would be best for me to clone the existing installation on the stick and get this done quickly and with no downtime

      I'll also make sure to tweak those values you mentioned
    • therock003 wrote:

      what read/write speeds would you consider sufficient for this task?
      None since irrelevant. And the usual numbers you find (sequential write/read in MB/s) are irrelevant anyway since if something will happen on an OMV rootfs then it's random IO which is something entirely different from a performance point of view. I have thumb drives that write with 120MB/s sequential stuff but as soon as a random IO workload is tested they perform horribly low (not even 100 4K IOPS which would be translated to KB/s less than 400)

      Just check it yourself on your current installation (might need an 'apt install sysstat'):

      Source Code

      1. root@lime2:~# iostat 5
      2. Linux 4.11.6-sunxi (lime2) 12/17/2017 _armv7l_ (2 CPU)
      3. avg-cpu: %user %nice %system %iowait %steal %idle
      4. 1.45 0.63 1.53 0.22 0.00 96.17
      5. Device: tps kB_read/s kB_wrtn/s kB_read kB_wrtn
      6. mmcblk0 3.63 76.10 118.73 763613405 1191432594
      7. sda 0.01 0.13 2.35 1324064 23541860
      8. avg-cpu: %user %nice %system %iowait %steal %idle
      9. 0.60 0.80 1.71 0.00 0.00 96.88
      10. Device: tps kB_read/s kB_wrtn/s kB_read kB_wrtn
      11. mmcblk0 5.20 1.60 24.00 8 120
      12. sda 0.00 0.00 0.00 0 0
      13. ^C
      14. root@lime2:~# uptime
      15. 17:22:15 up 116 days, 3:23, 1 user, load average: 0.16, 0.19, 0.12
      Display All
      This is a web server running off a quality SD card. The first values are average values since last reboot so this is what's most interesting. In the above case that's KB/s values and most probably it won't look any different on your SATA drive today. Just check it and maybe let an 'iostat 3600' run in a shell and check after 24 hours what happened (iostat 3600 will print new values every 3600 seconds).

      In your case you're fine with an installation media that's just 4GB in size but you don't get anything great with that low capacity and on the other hand the larger the drive the later it will wear out (it's really that easy). I personally choose only USB3 stuff these days (since performance can be expected to be somewhat ok-ish at least) and use them on USB2 ports to avoid USB3-A receptacle hassles that happen from time to time.

      Sorry, can't help with migration strategies since I use OMV still on tiny ARM devices (on x64 only in VMs right now) but I would assume you find a lot of tutorials/posts by using the search function above.
    • BTW: Just collected the pendrives flying around and already found the 1st one that has to go to the trash (really old 4GB thingie):

      Source Code

      1. root@rock64:/mnt/sda1# f3write /mnt/sda1/
      2. Free space: 3.74 GB
      3. Creating file 1.h2w ... OK!
      4. Creating file 2.h2w ... OK!
      5. Creating file 3.h2w ... OK!
      6. Creating file 4.h2w ... OK!
      7. Free space: 16.00 MB
      8. Average writing speed: 1.94 MB/s
      9. root@rock64:/mnt/sda1# f3read /mnt/sda1/
      10. SECTORS ok/corrupted/changed/overwritten
      11. Validating file 1.h2w ... 2097136/ 16/ 0/ 0
      12. Validating file 2.h2w ... 2097136/ 16/ 0/ 0
      13. Validating file 3.h2w ... 2097152/ 0/ 0/ 0
      14. Validating file 4.h2w ... 1527432/ 0/ 0/ 0
      15. Data OK: 3.73 GB (7818856 sectors)
      16. Data LOST: 16.00 KB (32 sectors)
      17. Corrupted: 16.00 KB (32 sectors)
      18. Slightly changed: 0.00 Byte (0 sectors)
      19. Overwritten: 0.00 Byte (0 sectors)
      20. Average reading speed: 18.13 MB/s
      Display All
    • And another one for the trash (obviously fake 16GB stick that died after writing 4GB to it):

      Source Code

      1. root@rock64:~# f3write /mnt/sdb1/
      2. Free space: 13.94 GB
      3. Creating file 1.h2w ... OK!
      4. Creating file 2.h2w ... OK!
      5. Creating file 3.h2w ... OK!
      6. Creating file 4.h2w ... 25.56% -- 2.84 MB/s -- 58:58f3write: f3write.c:241: measure: Assertion `!fdatasync(fd)' failed.
      7. Aborted
      8. root@rock64:~# dmesg | tail -n 33
      9. [ 438.421929] EXT4-fs (sdb1): mounted filesystem with ordered data mode. Opts: (null)
      10. [ 2206.340762] usb 1-1: reset high-speed USB device number 2 using dwc2
      11. [ 2206.794875] usb 1-1: device firmware changed
      12. [ 2206.795427] usb 1-1: USB disconnect, device number 2
      13. [ 2206.800864] sd 0:0:0:1: [sdb] tag#0 UNKNOWN(0x2003) Result: hostbyte=0x01 driverbyte=0x00
      14. [ 2206.801783] sd 0:0:0:1: [sdb] tag#0 CDB: opcode=0x2a 2a 00 00 7b f0 00 00 00 e8 00
      15. [ 2206.802580] blk_update_request: I/O error, dev sdb, sector 8122368
      16. [ 2206.803270] EXT4-fs warning (device sdb1): ext4_end_bio:330: I/O error -5 writing to inode 15 (offset 603979776 size 2215936 starting block 1015325)
      17. [ 2206.804708] Buffer I/O error on device sdb1, logical block 1014272
      18. [ 2206.805394] Buffer I/O error on device sdb1, logical block 1014273
      19. [ 2206.806082] Buffer I/O error on device sdb1, logical block 1014274
      20. [ 2206.806735] Buffer I/O error on device sdb1, logical block 1014275
      21. [ 2206.807403] Buffer I/O error on device sdb1, logical block 1014276
      22. [ 2206.808080] Buffer I/O error on device sdb1, logical block 1014277
      23. [ 2206.808735] Buffer I/O error on device sdb1, logical block 1014278
      24. [ 2206.809403] Buffer I/O error on device sdb1, logical block 1014279
      25. [ 2206.810079] Buffer I/O error on device sdb1, logical block 1014280
      26. [ 2206.810734] Buffer I/O error on device sdb1, logical block 1014281
      27. [ 2206.811648] sd 0:0:0:1: [sdb] tag#0 UNKNOWN(0x2003) Result: hostbyte=0x01 driverbyte=0x00
      28. [ 2206.812576] sd 0:0:0:1: [sdb] tag#0 CDB: opcode=0x2a 2a 00 00 c5 12 d8 00 00 28 00
      29. [ 2206.813421] blk_update_request: I/O error, dev sdb, sector 12915416
      30. [ 2206.814147] Aborting journal on device sdb1-8.
      31. [ 2206.814890] JBD2: Error -5 detected when updating journal superblock for sdb1-8.
      32. [ 2207.117772] usb 1-1: new high-speed USB device number 3 using dwc2
      33. [ 2207.572583] usb 1-1: New USB device found, idVendor=13fe, idProduct=5500
      34. [ 2207.573313] usb 1-1: New USB device strings: Mfr=1, Product=2, SerialNumber=3
      35. [ 2207.574087] usb 1-1: Product: USB DISK 53X
      36. [ 2207.574640] usb 1-1: Manufacturer: Phison
      37. [ 2207.578596] usb 1-1: SerialNumber: 000000000003
      38. [ 2207.583131] usb-storage 1-1:1.0: USB Mass Storage device detected
      39. [ 2207.587351] scsi host1: usb-storage 1-1:1.0
      40. [ 2208.596509] scsi 1:0:0:0: Direct-Access 13FE USB DISK 50X PMAP PQ: 0 ANSI: 4
      41. [ 2208.613649] sd 1:0:0:0: [sda] Attached SCSI removable disk
      Display All

      The stick when now reinserted appears as USB device but no partitions and no data anymore accessible :)

      Source Code

      1. root@rock64:~# lsusb | grep -v Linux
      2. Bus 004 Device 002: ID 13fe:5500 Kingston Technology Company Inc.
      3. root@rock64:~# grep -v mmc /proc/partitions
      4. major minor #blocks name
      5. 1 0 4096 ram0
      When installing OMV on such a stick everything would've been perfectly fine until the day when total amount of data written to the stick would've exceeded 4GB. Then no data would've been written to the thumb drive any more and the next reboot would've failed.

      That's problem number one with USB thumb drives and SD cards as boot media: counterfeit stuff that fakes wrong capacity and users not being aware of this problem but focusing on 'limited number of writes' which is a problem too but not the main reason why people have that bad experiences with installations on flash storage.
    • Wow are we talking about that much of a difference. I thought you meant like a 10% shave, not 4 gigs posing as 16. I used to have 64GB microSD for my android device that act weird when nearing the end of the available space and i may suspect something fishy, but i never thought of the extent of which you're talking about.

      Anyway i have a 2GB old usb2 sandisk stick laying around tested it, and it accounted for every last bit of its nominal space. So i guess 2 is really 2. But you re saying and suggesting at least 4. Since after setting OMV there are no more writings happening, why would you need so much space. I mean its not like you store or create ane new files on the OS, you got the storage disks for storing files. What othere information is there that OMV would keep storing files thus require more than 2 and then 4GB of storage space

      Also as for speeds, you mentioned that random IOs would make anything look insuffucient but i wasnt asking for a maximum. I mean good will probably never be good enough, but when does bad gets worse than it should be. I have some micro thumb drives, that really do like 3MB/s read/writes which i feel should not be acceptable, and i was wondering if i can make use of those, or if i should buy a new usb stick specifically for this task
    • therock003 wrote:

      Wow are we talking about that much of a difference. I thought you meant like a 10% shave, not 4 gigs posing as 16
      Actually it's quite easy to get counterfeit flash memory with even less than 1% of the faked capacity. Want 2TB for 15 bucks? Here you go: ebay.com/itm/128GB-256GB-512GB…rives-U-Disk/272857717072 (and yes, consumers buy this crap, insert it into their PC, read 2TB capacity and are happy)

      On an OS drive logfiles are created and if something serioulsy goes wrong they might take a lot of space. That's why at least 4GB are advisable.

      And wrt random IO I've not been very clear it seems. Random IO is about IOPS (IO operations per second) and not throughput (MB/s). This is important since an OS drive shows exactly this write pattern (storing little pieces of this and that here and there) while the normal use case for almost all flash media is something entirely different: storing large chunks of data in a sequential fashion (eg storing pictures or video streams).

      You can get a thumb drive with just 1 MB/s sequential write performance but +500 4K IOPS and you can get another thumb drive that shows 10 MB/s but just 20 4K IOPS. The former drive will show magnitudes higher performance when used as OS drive.

      BUT... this doesn't matter that much any more as soon as you use the flashmemory plugin and disable monitoring since from now on writes to your OS drive happen only very sporadically.

      Since all flash media will wear out eventually and since drives with larger capacity will wear out later when same amount of data is written too I would try to get a new quality/genuine pendrive of at least 16GB size even if the OS fits within 2 GB. Just for some safety headroom.

      Reputable sources for flash memory in my opinion are only those vendors who
      • produce their own NAND flash
      • produce their own controllers or choose only quality stuff here
      • have internal knowledge
      • combine their own NAND dies with controllers to retail products
      For these reasons I would never buy from noname, Kingston, PNY, Intenso and other dubious vendors (who all suffer from a high probability of counterfeit stuff being inserted into their supply chains very early) and we ended up buying lowend flash products only from SanDisk, Samsung, Toshiba or Transcend any more...
    • Ok so the only real performance downside i see then, would be the initial boot time. My internal sata takes 1 and a half minute to load all the moduls and make my shares available. I would imagine the stick with With 3MB/s read speed would take considerably more each time i power on the machine

      As for the matter of the writes happening...I dont need logs. Can i disable them completely from being written? Also how do i manually erase the existing ones?
    • therock003 wrote:

      My internal sata takes 1 and a half minute to load all the moduls and make my shares available. I would imagine the stick with With 3MB/s read speed would take considerably more each time i power on the machine
      No, why?

      Just add the following prior to the 'exit 0' to /etc/rc.local:

      Source Code

      1. (iostat ; awk '{print $1}' /proc/uptime) >/tmp/boot-data.txt

      This will print what happened wrt storage access to /tmp/boot-data.txt and also show how many seconds passed until boot. On something the average OMV user would call a toy (my great EspressoBin that outperforms the majority of x86/x64 OMV hosts on this planet) it looks like this when running off a rather shitty SD card:

      Source Code

      1. root@espressobin:~# cat /tmp/boot-data.txt
      2. Linux 4.14.2-mvebu64 (espressobin) 12/20/2017 _aarch64_ (2 CPU)
      3. avg-cpu: %user %nice %system %iowait %steal %idle
      4. 16.99 0.00 25.54 9.24 0.00 48.23
      5. Device: tps kB_read/s kB_wrtn/s kB_read kB_wrtn
      6. mtdblock0 2.05 8.18 0.00 108 0
      7. sda 38.26 1844.24 0.00 24344 0
      8. mmcblk0 108.79 3687.73 472.12 48678 6232
      9. 13.25
      Display All
      13.25 seconds until boot and average read rate of below 4 MB/s. BTW: SATA HDDs are slow as hell when it's about random IO. If you buy a good USB thumb drive it will be a lot faster in this area.