How can I Mirror a Drive Without RAID

  • I just finished setting up my NAS. I finally fixed the bugs I was having in OMV with the system not updating and mounted the two 8TB hard drives as ext4.


    When I went to RAID options to do a Mirror config, my drives don't show, mounted or unmounted. I'm guessing this is because I formatted them to ext4 ???, not really sure as I only ever mirrored drives on an LSI Raid card in an IBM 3850 M2 server, so this is new to me.


    Anyway from things I read, I have gotten the impression that setting these 2 drives in a mirror RAID would be a bad idea and not a good way to protect my data.


    Is there a way I can mirror the data on the drive without RAID ?, I want to use one 8TB drive for all my shares, and the other to be an exact copy of all data (new, updated, deleted files)


    Thanks,

    • Offizieller Beitrag

    Is there a way I can mirror the data on the drive without RAID ?, I want to use one 8TB drive for all my shares, and the other to be an exact copy of all data (new, updated, deleted files)

    rsync would be perfect. It isn't a realtime mirror but I actually think it is better.

    omv 7.0.4-2 sandworm | 64 bit | 6.5 proxmox kernel

    plugins :: omvextrasorg 7.0 | kvm 7.0.10 | compose 7.1.2 | k8s 7.0-6 | cputemp 7.0 | mergerfs 7.0.3


    omv-extras.org plugins source code and issue tracker - github


    Please try ctrl-shift-R and read this before posting a question.

    Please put your OMV system details in your signature.
    Please don't PM for support... Too many PMs!

    • Offizieller Beitrag

    Following is the command line, with switches, and scheduled job that I'm using to rsync one disk to another. (At 05:00AM, once a day.) Yours will be different, depending on your disk labels.


    rsync -av --delete /srv/dev-disk-by-label-EXT401/ /srv/dev-disk-by-label-EXT402/


    **In the above, the source is on the left, the destination is on the right. Since the above command line deletes files in the destination that are not in the source (I.E. mirroring), it's important to get that right.**



    The scheduled job:


  • Ok so this seems like a good fit, do I have to create the folders on the backup drive or will it mirror all the shares ?


    Also will other users in the network be able to see the backup drive ? With the way Windows 10 is, all users have to manually type in //NASBOX/ when mapping a share folder. So I only want them to see the shares that are the ones to be used, not the ones that are backups.

    • Offizieller Beitrag

    1. Ok so this seems like a good fit, do I have to create the folders on the backup drive or will it mirror all the shares ?


    2. Also will other users in the network be able to see the backup drive ? With the way Windows 10 is, all users have to manually type in //NASBOX/ when mapping a share folder. So I only want them to see the shares that are the ones to be used, not the ones that are backups.

    The answer is no to both.


    1. The rsync command will copy all data (folders, files, etc.) to the second drive. Also, with the switches shown in the example, if something is deleted on the first drive, it gets deleted on the second drive (during the next Rsync Job).


    (EDIT) In essence with the command line above, modified for your drive names, the second disk becomes a mirror of the first that's updated when the scheduled job runs. You might want to give thought to the update interval, or when the job is set to run. If there's a data "delete" disaster on the first drive, with enough time to discover it, you could turn off the rsync job and copy the deleted file(s) or folder(s) back. Setting the interval to somewhere between one day or even one week would help in this regard. (And if you're inclined to do so, @macom 's advice on rsnapshot is very good. OMV has a plugin for rsnapshot. If it seems like a lot to do right now, you can set it up later. )


    2. The second drive is an independent data back up. While it's contents will be identical, it's contents are not shared. It's simply a full data copy of the first drive. If the first drive crashes and burns, it would be necessary to share the contents of the second drive.

    • Offizieller Beitrag

    If you can spend some time, also have a look at rsnapshot. It is also working with rsync but it creates snapshots of the data. Advantage is that you can go back in time. If you delete a file it is only deleted in the snapshots created after you have deleted the file. You are still able to access the file in older snapshots.


    If you change a file, the change will also only take effect in snapshots created after the change. You can always find previous versions of the file in earlier snapshots.


    For more details on rsnapshot use google, yahoo or whatever search engine you prefer.


    There is a plugin available for rsnapshot in omv-extras.

  • Ok I’ll look into it more on Tuesday, I have been working 12 hour shifts since and haven’t had time to work on anything home.


    I’m not 100% on Linux so how do I get my drive names and locations for the command line ? I’m assuming it not the labels I put on them in OMV.

    • Offizieller Beitrag

    The labels you put on them, when you format drives in OMV, will be part of the name. Look for the names, on the command line, under /srv.

    The easiest way to deal with Linux directory structure, look at it, etc., is to install WinSCP. This -> guide will show you how to install and use WinSCP.


    I'll be back next weekend.

    • Offizieller Beitrag

    You can show the mountpoint in the filesystem view.
    of the GUI


    Move the curser over the title row until you see a small arrow. Click
    the arrow and move to "column" now choose the additional columns you
    like to see.



    Drives are mounted in /srv/.

  • Ok so I got my mount point from the GUI using macom advice and using the command line from flmaxey I setup drive 1 to backup to drive 2. I picked Daily but there was noting for setting the time. I saved it and then click run. It seems to have copied the files to disk 2 which is great. But how do I know it gonna run on it's own and when it's going to run.


    These are the settings I picked in Scheduled Jobs


    Enable: ON
    Time of Execution: Daily
    User: root
    Command: rsync -av --delete /srv/dev-disk-by-label-WD8DISK1/ /srv/dev-disk-by-label-WD8DISK2/
    send email: OFF
    Comments: Mirror WD 8TB Disk One to WD 8TB Disk Two as backup.

    • Offizieller Beitrag

    I've never used "Daily" but it might run at 00:01AM, when the new day starts. That's a guess. I've never tested it.


    I use, "Certain Date", leave everything with the default stars / astericks and set a time in Hour and Minute. Hour 21 and Minute 30 would run at 9:30PM, everyday.

    • Offizieller Beitrag


    If you're looking for verification that the Rsync job occurred:
    Add your E-mail information in System, Notification, and enable it. (Use the send test E-mail option to check that all is correct.)


    Then in the scheduled job set Send E-mail: to ON. When the job runs, an E-mail will be sent to you with the command line output. Note, if a lot has changed on the source drive, the E-mail may be quite long but it would serve as verification of the event.

    • Offizieller Beitrag

    If you're looking for verification that the Rsync job occurred:
    Add your E-mail information in System, Notification, and enable it. (Use the send test E-mail option to check that all is correct.)

    I receive all sorts of notifications from my server including those from the USB backup plugin. However, I never managed to get a notification for a finished rsync job. Is this actually working for you?

    • Offizieller Beitrag

    \---\ I never managed to get a notification for a finished rsync job. Is this actually working for you?


    Yeah - it works for me. Here's an E-mail example:
    (There's no files list because the disks are up to date. Otherwise, it would be a long note.)



    **Note that I'm not using the "Send E-mail" option for an Rsync Job under Services, Rsync. (I haven't tried that.)



    The Rsync operation is executed as a command line (as shown above), under System, Scheduled Jobs. Send E-mail works from there.

  • Sorry for the late reply, work and all.


    I don't want an email sent as I get enough of them, I can tell it is working as I see the size of the drives change.


    I am more interested now in being able to set a time to do the Rsync. I'd like it to do it at 5 AM when no one is on. But I don't see an option under daily for setting the time.

    • Offizieller Beitrag

    I don't know when a job runs with "Daily", but I'd assume it kicks off at 00:01AM.


    In any case, if you want to set a specific time (05:00AM), use the following. It will run everyday at 05:00AM:
    ( The * means "all" - as in all days of the month, all months, all days of the week.)



  • Alright I'll give that a try, eventually I'll change backups to once a week once I got all main data moved over. Silly question tho, under this setup now, if Drive A failed, and Rsync kicked in to mirror the data of Drive A to Drive B. What would happen. Would it mirror a failed drive and wipe out drive B or would it see Drive A has failed and do nothing?

    • Offizieller Beitrag

    Alright I'll give that a try, eventually I'll change backups to once a week once I got all main data moved over. Silly question tho, under this setup now, if Drive A failed, and Rsync kicked in to mirror the data of Drive A to Drive B. What would happen. Would it mirror a failed drive and wipe out drive B or would it see Drive A has failed and do nothing?

    Really, it depends on how the source drive "fails":
    - If the source failed completely, with no source drive, the copy couldn't take place.
    - If the source slowly and silently corrupted data (but not the file names, dir's, or time stamps), data on the target would be still be OK.
    - If the sources' partition is intact but file and directory structure is partially lost (files and dir's begin to disappear), that would be a data disaster that would replicate to the destination - "if" the admin (you) didn't discover the problem before the next Rsync event. (And cancel the job.)
    (Protection from this possibility could be had by removing the --delete switch from the command line. This would also provide user delete protection, if the data is older than the backup/replication interval. The destination could be cleaned up by adding the --delete switch back into the command line on occasion, every few months or so.)


    With all things considered (failure modes, etc.), after completely copying the source drive, I'd go with a minimum backup interval of a week. And to allow more time for intervention, if needed, 2 weeks is not a bad idea.
    ______________________________________________________________________


    I'm going to try to come up with a way (hopefully) to use rsnapshot, for drive-to-drive backup. (The trick will be coming up with something that's simple for new users.)


    Rsnapshot (based on Rsync) would be a much better approach for backup because it allows restoration from several different time intervals (snapshots), and it would allow recovery from user deleted files, with potential for a year (or more) worth of protection.
    Similarly, rsnapshot can protect data that's lost from a source drive that's failing in a dirty way.

Jetzt mitmachen!

Sie haben noch kein Benutzerkonto auf unserer Seite? Registrieren Sie sich kostenlos und nehmen Sie an unserer Community teil!