Mirror OS drive

    • OMV 4.x

    This site uses cookies. By continuing to browse this site, you are agreeing to our Cookie Policy.

    • LeonThePC wrote:

      No, I am talking about mirroring the OS Flash drive so if one flash drive dies the system will be able to boot from the other. I thought I read somewhere that it is possible. Am I wrong?
      If you are realtime mirroring two usb sticks (please don't - if you need redundancy, you something better than a usb stick), once one usb stick starts failing, it is going to mirror the corruption to the other usb stick.
      omv 4.1.6 arrakis | 64 bit | 4.16 backports kernel | omvextrasorg 4.1.7
      omv-extras.org plugins source code and issue tracker - github.com/OpenMediaVault-Plugin-Developers

      Please read this before posting a question.
      Please don't PM for support... Too many PMs!
    • Are we talking about file corruption or Media corruption. If a flash drive starts to fail will it cause the files to be corrupted on the other flash drive? Then would you say that would be the same with a mirroring on a hard drive? It one drive starts to fail the the other drive will also get corrupted?

      If flash drives would die after a few months even with the Flash Plugin enabled why is install the OS recommended to be done to a flash drive.
    • LeonThePC wrote:

      Are we talking about file corruption or Media corruption.
      file corruption.

      LeonThePC wrote:

      If a flash drive starts to fail will it cause the files to be corrupted on the other flash drive?
      Yes.

      LeonThePC wrote:

      Then would you say that would be the same with a mirroring on a hard drive? It one drive starts to fail the the other drive will also get corrupted?
      Yes but they typically fail after a much longer period of time.

      LeonThePC wrote:

      If flash drives would die after a few months even with the Flash Plugin enabled why is install the OS recommended to be done to a flash drive.
      I didn't say they would only last a few months with the flashmemory plugin. And it is definitely not *recommended* to use a flash drive. We stopped recommending against flash drives because of the flashmemory plugin but it will never be the recommended media.

      Anyway, you are missing my point. I was trying to say that if you can't handle having downtime and want realtime mirroring, use something better than flash drives. You would be putting just as much wear & tear on the backup stick as the primary stick. I would use a scheduled dd or rsync to copy the running stick to the backup once a week or something. You could manually run the sync whenever something major changed on the primary as well.
      omv 4.1.6 arrakis | 64 bit | 4.16 backports kernel | omvextrasorg 4.1.7
      omv-extras.org plugins source code and issue tracker - github.com/OpenMediaVault-Plugin-Developers

      Please read this before posting a question.
      Please don't PM for support... Too many PMs!
    • I don't need realtime mirroring, but it would still be nice. Mostly, I want a copy of the OS and configuration so that if the Flash drive dies, I don't have to reinstall and reconfigure.

      1. Will rsync make the 2nd flash drive bootable?
      2. My motherboard comes with 4 SATA ports. If I am setting up RAID 5 with 4 drives, what other option do I have for boot drive other than Flash drive? The motherboard would have to have 6 or more SATA ports to be able to have a RAID 5 and 2 mirrored OS drives.
    • Here is the script that makes the dd backups. You will have to properly change it to reflect the id for your system drive and the location of the stored backups. I run this daily on cron at 3:00am.


      Shell-Script

      1. #!/bin/bash
      2. # Script to make dd image of system drive
      3. now=$(date +"%Y.%m.%d.%H.%M.%S")
      4. file="omv-4-$now.img"
      5. cd /srv/dev-disk-by-label-d1/omv-backup-imgs
      6. dd if=/dev/disk/by-id/usb-SAMSUNG_SSD_UM410_Series_100000000000-0:0 of=$file bs=1M
      Display All
      The shebang does not display in the above script, so please enter it when you create the file.


      Here is the script that moves the daily dd image into a directory that is created with today's date and deletes any directories and files that are older than 7 days. I run this daily on cron at 11:00am. You will have to properly change it to reflect the id for your system drive and the location of the stored backups.

      Source Code

      1. #! /bin/sh
      2. TEMP=`date +%F`
      3. cd /srv/dev-disk-by-label-d1/omv-backup-imgs
      4. mkdir $TEMP
      5. ls -1 | grep img | xargs -n 10 -i mv {} $TEMP/
      6. cd /srv/dev-disk-by-label-d1/omv-backup-imgs
      7. # now remove old directories and files
      8. cd /srv/dev-disk-by-label-d1/omv-backup-imgs
      9. for i in *
      10. do
      11. if [ -d $i ]
      12. then
      13. TODAY=`date +%s`
      14. FILETIME=`date +%s -d $i`
      15. if [ $? -eq 0 ]
      16. then
      17. DIFF=`expr $TODAY - $FILETIME`
      18. # The number below is seconds
      19. # 2592000s 30d
      20. # 7776000s 90d
      21. # 604800s 7d
      22. # 259200s 3d
      23. if [ $DIFF -gt 604800 ]
      24. then
      25. echo $i $DIFF "more than 7 days, say bye bye"
      26. cd $i
      27. # remove a lot of files, then remove dir
      28. ls -1 | xargs -n 10 -i rm -f {}
      29. cd ..
      30. rmdir $i
      31. else
      32. echo $i $DIFF "OK"
      33. fi
      34. else
      35. echo $i "is not a dated directory"
      36. fi
      37. fi
      38. done
      Display All
      The above scripts create the dd images of my OMV 16GB SSD system drive and store them on another large hard drive. The size of each dd image is the same size as the system drive, so make sure you have enough room on the other drive to hold as many images as you have decided to keep.

      To restore an image you can use dd. Just be very, very careful about the destination (of=) you chose to restore the image file to as it will be overwritten without any warning. The destination drive must be at least as large as the drive that the image was made from. Again, you will have to properly compose this command to reflect your drive id and image file name. This command must be run as root. Again, be very careful specifying the destination drive (of=), I am using /dev/sda here as an example only.

      dd if=omv-4-2018.05.01.07.00.01.img of=/dev/sda bs=1M
      OMV 4.x - ASRock Rack C2550D4I - 16GB ECC - Silverstone DS380

      The post was edited 2 times, last by gderf ().

    • You have to create a text file in an editor and paste the script contents into it. Give the file a descriptive name and perhaps the .sh extension. Then make it executable by running chmod 777 scriptname.sh

      Don't forget that missing shebang in the first script. It really is in my post but the forum will not display it.

      You can put the scripts anywhere, but I put all of mine in /usr/local/scripts

      You can use OMV's scheduled jobs to create the crons.

      Also, make absolutely sure you have made all the necessary changes to the scripts as required for your setup. And don't rely on these until you have tested the entire process from end to end - creating a backup image, restoring it to new media, and making sure it boots and works. Also, keep an eye on your image storage location to make sure older images are being deleted.
      OMV 4.x - ASRock Rack C2550D4I - 16GB ECC - Silverstone DS380
    • First, thank you so much.
      Second, sorry again, I work with Windows all day, I am new to linux.

      How do I use the OMV "scheduled jobs" to create the crons?




      changes to the 1st script:
      I need help with last line (dd if=/dev/disk/by-id/usb-SAMSUNG_SSD_UM410_Series_100000000000-0:0 of=$file bs=1M )
      1st - how do I get the drive UUID. I tried (ls -l /dev/disk/by-uuid) and my results didn't look like yours, i got
      f17ff563-9864-44fb-87d0-b4116d1fdd46

      2nd - do I replace this (usb-SAMSUNG_SSD_UM410_Series_100000000000-0:0) with my UUID?
    • See the attached screenshot for an example scheduled job. Adjust to suit.

      You can use

      /dev/disk/by-uuid/f17ff563-9864-44fb-87d0-b4116d1fdd46

      so long as that is your entire OMV system disk. If you have data disks in your machine they will also appear in that list.

      I could use similar, but since my OMV system disk is a SSD in an external USB case, I prefer to use what I have - it's clearer.

      If you run this command it might provide more details about your disks:

      tree /dev/disk -a

      What you want to be sure of is that your dd if= statement points to the entire disk and not any of the individual partitions.
      Images
      • Selection_003.jpg

        65.74 kB, 549×699, viewed 18 times
      OMV 4.x - ASRock Rack C2550D4I - 16GB ECC - Silverstone DS380
    • Using a UUID for a partition will not work for this purpose, you must image the entire disk.

      For the dd if= part, you can use /dev/sde but this is not the best practice because there is no guarantee that that physical disk will always be /dev/sde. If you add or remove disks or rearrange them by moving the cables around, it could change and you will have problems.

      Also, using dd if=/dev/disk/by-label/...... will not work either as it misses the boot track.

      Best practice is to use dd if=/dev/disk/by-id/.....

      You can install the tree command in a root shell like this:

      apt-get install tree

      Then use tree /dev/disk -a to get the required information.
      OMV 4.x - ASRock Rack C2550D4I - 16GB ECC - Silverstone DS380
    • I created the first script. When I ran it I received errors.

      ERRORS
      line 2: $'\r': command not found
      line 4: $'\r': command not found
      line 6: $'\r': command not found
      line 8: $'\r': command not found
      line 9: cd: $'/srv/dev-disk-by-label-backup/OSbackup\r': No such file or directory
      line 10: $'\r': command not found

      MY SCRIPT
      #!/bin/bash

      # Script to make dd image of system drive

      now=$(date +"%Y.%m.%d.%H.%M.%S")

      file="omv-4-$now.img"

      cd /srv/dev-disk-by-label-backup/OSbackup

      dd if=/dev/disk/by-id/usb-Lexar_USB_Flash_Drive_AASORI8PQW0H8O7E-0:0 of=$file bs=1M