How can I Mirror a Drive Without RAID

    • OMV 4.x
    • How can I Mirror a Drive Without RAID

      I just finished setting up my NAS. I finally fixed the bugs I was having in OMV with the system not updating and mounted the two 8TB hard drives as ext4.

      When I went to RAID options to do a Mirror config, my drives don't show, mounted or unmounted. I'm guessing this is because I formatted them to ext4 ???, not really sure as I only ever mirrored drives on an LSI Raid card in an IBM 3850 M2 server, so this is new to me.

      Anyway from things I read, I have gotten the impression that setting these 2 drives in a mirror RAID would be a bad idea and not a good way to protect my data.

      Is there a way I can mirror the data on the drive without RAID ?, I want to use one 8TB drive for all my shares, and the other to be an exact copy of all data (new, updated, deleted files)

      Thanks,
    • Elochai wrote:

      Is there a way I can mirror the data on the drive without RAID ?, I want to use one 8TB drive for all my shares, and the other to be an exact copy of all data (new, updated, deleted files)
      rsync would be perfect. It isn't a realtime mirror but I actually think it is better.
      omv 4.1.13 arrakis | 64 bit | 4.15 proxmox kernel | omvextrasorg 4.1.13
      omv-extras.org plugins source code and issue tracker - github

      Please read this before posting a question and this and this for docker questions.
      Please don't PM for support... Too many PMs!
    • Following is the command line, with switches, and scheduled job that I'm using to rsync one disk to another. (At 05:00AM, once a day.) Yours will be different, depending on your disk labels.

      rsync -av --delete /srv/dev-disk-by-label-EXT401/ /srv/dev-disk-by-label-EXT402/

      **In the above, the source is on the left, the destination is on the right. Since the above command line deletes files in the destination that are not in the source (I.E. mirroring), it's important to get that right.**


      The scheduled job:

      Good backup takes the "drama" out of computing
      ____________________________________
      Primary: OMV 3.0.99, ThinkServer TS140, 12GB ECC, 32GB USB boot, 4TB+4TB zmirror, 3TB client backup.
      Backup: OMV 4.1.9, Acer RC-111, 4GB, 32GB USB boot, 3TB+3TB zmirror, 4TB Rsync'ed disk
      2nd Data Backup: OMV 3.0.99, R-PI 2B, 16GB boot, 4TB WD USB MyPassport - direct connect (no hub)

      The post was edited 2 times, last by flmaxey: edit ().

    • Ok so this seems like a good fit, do I have to create the folders on the backup drive or will it mirror all the shares ?

      Also will other users in the network be able to see the backup drive ? With the way Windows 10 is, all users have to manually type in //NASBOX/ when mapping a share folder. So I only want them to see the shares that are the ones to be used, not the ones that are backups.
    • Elochai wrote:

      1. Ok so this seems like a good fit, do I have to create the folders on the backup drive or will it mirror all the shares ?

      2. Also will other users in the network be able to see the backup drive ? With the way Windows 10 is, all users have to manually type in //NASBOX/ when mapping a share folder. So I only want them to see the shares that are the ones to be used, not the ones that are backups.
      The answer is no to both.

      1. The rsync command will copy all data (folders, files, etc.) to the second drive. Also, with the switches shown in the example, if something is deleted on the first drive, it gets deleted on the second drive (during the next Rsync Job).

      (EDIT) In essence with the command line above, modified for your drive names, the second disk becomes a mirror of the first that's updated when the scheduled job runs. You might want to give thought to the update interval, or when the job is set to run. If there's a data "delete" disaster on the first drive, with enough time to discover it, you could turn off the rsync job and copy the deleted file(s) or folder(s) back. Setting the interval to somewhere between one day or even one week would help in this regard. (And if you're inclined to do so, @macom 's advice on rsnapshot is very good. OMV has a plugin for rsnapshot. If it seems like a lot to do right now, you can set it up later. )

      2. The second drive is an independent data back up. While it's contents will be identical, it's contents are not shared. It's simply a full data copy of the first drive. If the first drive crashes and burns, it would be necessary to share the contents of the second drive.
      Good backup takes the "drama" out of computing
      ____________________________________
      Primary: OMV 3.0.99, ThinkServer TS140, 12GB ECC, 32GB USB boot, 4TB+4TB zmirror, 3TB client backup.
      Backup: OMV 4.1.9, Acer RC-111, 4GB, 32GB USB boot, 3TB+3TB zmirror, 4TB Rsync'ed disk
      2nd Data Backup: OMV 3.0.99, R-PI 2B, 16GB boot, 4TB WD USB MyPassport - direct connect (no hub)

      The post was edited 2 times, last by flmaxey: added content ().

    • If you can spend some time, also have a look at rsnapshot. It is also working with rsync but it creates snapshots of the data. Advantage is that you can go back in time. If you delete a file it is only deleted in the snapshots created after you have deleted the file. You are still able to access the file in older snapshots.

      If you change a file, the change will also only take effect in snapshots created after the change. You can always find previous versions of the file in earlier snapshots.

      For more details on rsnapshot use google, yahoo or whatever search engine you prefer.

      There is a plugin available for rsnapshot in omv-extras.
      Odroid HC2 - armbian - Seagate ST4000DM004 - OMV4.x
      Asrock Q1900DC-ITX - 16GB - 2x Seagate ST3000VN000 - Intenso SSD 120GB - OMV4.x
      :!: Backup - Solutions to common problems - OMV setup videos - OMV4 Documentation - user guide :!:

      The post was edited 1 time, last by macom ().

    • See the Edit, in the post above.
      Good backup takes the "drama" out of computing
      ____________________________________
      Primary: OMV 3.0.99, ThinkServer TS140, 12GB ECC, 32GB USB boot, 4TB+4TB zmirror, 3TB client backup.
      Backup: OMV 4.1.9, Acer RC-111, 4GB, 32GB USB boot, 3TB+3TB zmirror, 4TB Rsync'ed disk
      2nd Data Backup: OMV 3.0.99, R-PI 2B, 16GB boot, 4TB WD USB MyPassport - direct connect (no hub)
    • Ok I’ll look into it more on Tuesday, I have been working 12 hour shifts since and haven’t had time to work on anything home.

      I’m not 100% on Linux so how do I get my drive names and locations for the command line ? I’m assuming it not the labels I put on them in OMV.
    • The labels you put on them, when you format drives in OMV, will be part of the name. Look for the names, on the command line, under /srv.

      The easiest way to deal with Linux directory structure, look at it, etc., is to install WinSCP. This -> guide will show you how to install and use WinSCP.


      I'll be back next weekend.
      Good backup takes the "drama" out of computing
      ____________________________________
      Primary: OMV 3.0.99, ThinkServer TS140, 12GB ECC, 32GB USB boot, 4TB+4TB zmirror, 3TB client backup.
      Backup: OMV 4.1.9, Acer RC-111, 4GB, 32GB USB boot, 3TB+3TB zmirror, 4TB Rsync'ed disk
      2nd Data Backup: OMV 3.0.99, R-PI 2B, 16GB boot, 4TB WD USB MyPassport - direct connect (no hub)
    • You can show the mountpoint in the filesystem view.
      of the GUI

      Move the curser over the title row until you see a small arrow. Click
      the arrow and move to "column" now choose the additional columns you
      like to see.


      Drives are mounted in /srv/.
      Odroid HC2 - armbian - Seagate ST4000DM004 - OMV4.x
      Asrock Q1900DC-ITX - 16GB - 2x Seagate ST3000VN000 - Intenso SSD 120GB - OMV4.x
      :!: Backup - Solutions to common problems - OMV setup videos - OMV4 Documentation - user guide :!:
    • Ok so I got my mount point from the GUI using macom advice and using the command line from flmaxey I setup drive 1 to backup to drive 2. I picked Daily but there was noting for setting the time. I saved it and then click run. It seems to have copied the files to disk 2 which is great. But how do I know it gonna run on it's own and when it's going to run.

      These are the settings I picked in Scheduled Jobs

      Enable: ON
      Time of Execution: Daily
      User: root
      Command: rsync -av --delete /srv/dev-disk-by-label-WD8DISK1/ /srv/dev-disk-by-label-WD8DISK2/
      send email: OFF
      Comments: Mirror WD 8TB Disk One to WD 8TB Disk Two as backup.
    • I've never used "Daily" but it might run at 00:01AM, when the new day starts. That's a guess. I've never tested it.

      I use, "Certain Date", leave everything with the default stars / astericks and set a time in Hour and Minute. Hour 21 and Minute 30 would run at 9:30PM, everyday.
      Good backup takes the "drama" out of computing
      ____________________________________
      Primary: OMV 3.0.99, ThinkServer TS140, 12GB ECC, 32GB USB boot, 4TB+4TB zmirror, 3TB client backup.
      Backup: OMV 4.1.9, Acer RC-111, 4GB, 32GB USB boot, 3TB+3TB zmirror, 4TB Rsync'ed disk
      2nd Data Backup: OMV 3.0.99, R-PI 2B, 16GB boot, 4TB WD USB MyPassport - direct connect (no hub)
    • Elochai wrote:

      /---/


      But how do I know it gonna run on it's own and when it's going to run.

      These are the settings I picked in Scheduled Jobs

      Enable: ON
      Time of Execution: Daily
      User: root
      Command: rsync -av --delete /srv/dev-disk-by-label-WD8DISK1/ /srv/dev-disk-by-label-WD8DISK2/
      send email: OFF
      Comments: Mirror WD 8TB Disk One to WD 8TB Disk Two as backup.

      If you're looking for verification that the Rsync job occurred:
      Add your E-mail information in System, Notification, and enable it. (Use the send test E-mail option to check that all is correct.)

      Then in the scheduled job set Send E-mail: to ON. When the job runs, an E-mail will be sent to you with the command line output. Note, if a lot has changed on the source drive, the E-mail may be quite long but it would serve as verification of the event.
      Good backup takes the "drama" out of computing
      ____________________________________
      Primary: OMV 3.0.99, ThinkServer TS140, 12GB ECC, 32GB USB boot, 4TB+4TB zmirror, 3TB client backup.
      Backup: OMV 4.1.9, Acer RC-111, 4GB, 32GB USB boot, 3TB+3TB zmirror, 4TB Rsync'ed disk
      2nd Data Backup: OMV 3.0.99, R-PI 2B, 16GB boot, 4TB WD USB MyPassport - direct connect (no hub)
    • flmaxey wrote:

      If you're looking for verification that the Rsync job occurred:
      Add your E-mail information in System, Notification, and enable it. (Use the send test E-mail option to check that all is correct.)
      I receive all sorts of notifications from my server including those from the USB backup plugin. However, I never managed to get a notification for a finished rsync job. Is this actually working for you?
      Odroid HC2 - armbian - Seagate ST4000DM004 - OMV4.x
      Asrock Q1900DC-ITX - 16GB - 2x Seagate ST3000VN000 - Intenso SSD 120GB - OMV4.x
      :!: Backup - Solutions to common problems - OMV setup videos - OMV4 Documentation - user guide :!:
    • macom wrote:

      \---\ I never managed to get a notification for a finished rsync job. Is this actually working for you?

      Yeah - it works for me. Here's an E-mail example:
      (There's no files list because the disks are up to date. Otherwise, it would be a long note.)



      **Note that I'm not using the "Send E-mail" option for an Rsync Job under Services, Rsync. (I haven't tried that.)


      The Rsync operation is executed as a command line (as shown above), under System, Scheduled Jobs. Send E-mail works from there.
      Good backup takes the "drama" out of computing
      ____________________________________
      Primary: OMV 3.0.99, ThinkServer TS140, 12GB ECC, 32GB USB boot, 4TB+4TB zmirror, 3TB client backup.
      Backup: OMV 4.1.9, Acer RC-111, 4GB, 32GB USB boot, 3TB+3TB zmirror, 4TB Rsync'ed disk
      2nd Data Backup: OMV 3.0.99, R-PI 2B, 16GB boot, 4TB WD USB MyPassport - direct connect (no hub)
    • Sorry for the late reply, work and all.

      I don't want an email sent as I get enough of them, I can tell it is working as I see the size of the drives change.

      I am more interested now in being able to set a time to do the Rsync. I'd like it to do it at 5 AM when no one is on. But I don't see an option under daily for setting the time.
    • I don't know when a job runs with "Daily", but I'd assume it kicks off at 00:01AM.

      In any case, if you want to set a specific time (05:00AM), use the following. It will run everyday at 05:00AM:
      ( The * means "all" - as in all days of the month, all months, all days of the week.)


      Good backup takes the "drama" out of computing
      ____________________________________
      Primary: OMV 3.0.99, ThinkServer TS140, 12GB ECC, 32GB USB boot, 4TB+4TB zmirror, 3TB client backup.
      Backup: OMV 4.1.9, Acer RC-111, 4GB, 32GB USB boot, 3TB+3TB zmirror, 4TB Rsync'ed disk
      2nd Data Backup: OMV 3.0.99, R-PI 2B, 16GB boot, 4TB WD USB MyPassport - direct connect (no hub)
    • flmaxey wrote:

      I don't know when a job runs with "Daily", but I'd assume it kicks off at 00:01AM.

      In any case, if you want to set a specific time (05:00AM), use the following. It will run everyday at 05:00AM:
      ( The * means "all" - as in all days of the month, all months, all days of the week.)



      Alright I'll give that a try, eventually I'll change backups to once a week once I got all main data moved over. Silly question tho, under this setup now, if Drive A failed, and Rsync kicked in to mirror the data of Drive A to Drive B. What would happen. Would it mirror a failed drive and wipe out drive B or would it see Drive A has failed and do nothing?
    • Elochai wrote:

      Alright I'll give that a try, eventually I'll change backups to once a week once I got all main data moved over. Silly question tho, under this setup now, if Drive A failed, and Rsync kicked in to mirror the data of Drive A to Drive B. What would happen. Would it mirror a failed drive and wipe out drive B or would it see Drive A has failed and do nothing?
      Really, it depends on how the source drive "fails":
      - If the source failed completely, with no source drive, the copy couldn't take place.
      - If the source slowly and silently corrupted data (but not the file names, dir's, or time stamps), data on the target would be still be OK.
      - If the sources' partition is intact but file and directory structure is partially lost (files and dir's begin to disappear), that would be a data disaster that would replicate to the destination - "if" the admin (you) didn't discover the problem before the next Rsync event. (And cancel the job.)
      (Protection from this possibility could be had by removing the --delete switch from the command line. This would also provide user delete protection, if the data is older than the backup/replication interval. The destination could be cleaned up by adding the --delete switch back into the command line on occasion, every few months or so.)

      With all things considered (failure modes, etc.), after completely copying the source drive, I'd go with a minimum backup interval of a week. And to allow more time for intervention, if needed, 2 weeks is not a bad idea.
      ______________________________________________________________________

      I'm going to try to come up with a way (hopefully) to use rsnapshot, for drive-to-drive backup. (The trick will be coming up with something that's simple for new users.)

      Rsnapshot (based on Rsync) would be a much better approach for backup because it allows restoration from several different time intervals (snapshots), and it would allow recovery from user deleted files, with potential for a year (or more) worth of protection.
      Similarly, rsnapshot can protect data that's lost from a source drive that's failing in a dirty way.
      Good backup takes the "drama" out of computing
      ____________________________________
      Primary: OMV 3.0.99, ThinkServer TS140, 12GB ECC, 32GB USB boot, 4TB+4TB zmirror, 3TB client backup.
      Backup: OMV 4.1.9, Acer RC-111, 4GB, 32GB USB boot, 3TB+3TB zmirror, 4TB Rsync'ed disk
      2nd Data Backup: OMV 3.0.99, R-PI 2B, 16GB boot, 4TB WD USB MyPassport - direct connect (no hub)