Cannot create RAID, new install

There are two ways this could be done, that come to mind, with all drives in one box.

For good fault protection and ease of restoration:

- UnionFs one 10TB drive together with one 6TB drive, for a total of 16TB.
- Do the same with the second drive pair.
- Rsync UnionFs 1 to UnionFs 2
(Other than the extra piece of combining 2 drives into a UnionFS drive, the Rsync command line is outlined in the guide. It would be a matter of setting up the command line with UnionFs1 as the source and UnionFS2 as the destination.)

This give you 100% data backup and restorations are easy. If something in the primary pair fails, simply re-point your shared folders to UnionFs 2, with identical content, and you're back on line. (This is in the guide.) However, there's no bitrot protection.
_________________________________________________________

For the maximum disk real-estate, with protection and recovery options:

- UnionFS all drives together 10+6+6, saving one 10TB drive for SNAPRAID. (For SNAPRAID to work, the parity drive must be equal to, or larger, than the largest drive in the array.)
- This would give you a UnionFS drive of 22Tb
- SNAPRAID provides decent options for the recovery of the loss of a drive, scrubbing for bitrot and more. Take a look-> here.

While getting started is easy, recovery is more involved than an Rsync backup. ((In any case, when recovering, it's time to slow down, take a breath and do some reading before doing anything, in any case.))

The trade off is more disk space and bitrot protection, versus ease of restoration.

_________________________________________________________

I'm not telling you what to do, but it makes sense to investigate the options before committing. And remember, as you consider the your options; it's not a matter of "if" a drive will fail, it's a matter of "when".

Zitat von telijah

Also, I am gonna start this over with blank 10TB drives, which command is best to wipe those drives clean to follow your guide properly? I will remove the share first

Since you've already done Wipefs, I think the guide (quick wipe) process would do it. But, if you've already set up mdadm RAID, maybe you should do the wipefs command again just to be sure.

From there, follow the guide. An extra quick wipe takes no time at all.
(Using the EXT4 format, as it is in the guide, is probably best - that will take a bit of time.) Once you get a 10TB drive setup and your data copied over from the synology; I'll try to provide some pointers for what you decide on.

1. Put the EXT4 on both 10TB drives (sdb and sdc).Yes.

2. Create shared folders and SMB service to themYes. (On the primary drive only.)

3. Stop PlexMaybe? I don't know the in's and outs of Plex, but this makes sense.

4. Rsync from 6TB Synology share to primary 10TB driveYes. You'll set up a remote mount, to the Synlogy share (already described).
When it shows up in Storage, File Systems, then create two shared folders as previously described, one on OMV and one on the Synolgy box.
Finally, create a LOCAL Rsync job in the GUI, under Services, Rsync, with the source shared folder being the Synology share and the destination shared folder being the OMV share. (Take a look at the Rsync Options I use for a box to box transfer, below.)


5. Setup rsync job between sdb and sdc Yes. From the guide, you'll set up your own custom Rsync command line, and execute the drive-to-drive command line under System, Scheduled Jobs. Per guidance in the guide, give yourself enough time to intervene if something goes wrong.

6. Complete Plex migration and restart Plex; point Plex database to new share(s)(Makes sense. You know more about this than I do.)

Take your time and make sure all is working on OMV the way you want it to, before touching the Synology box. (And I mean like a few weeks to a month or more to be sure of your OMV configuration, all updates are applied, the boot drive is cloned, all is well and it's working as a media and file server the way you want it to.) During that period, the synology box will be a backup/fall back, in the event that something goes wrong. As we all know, "stuff" happens. Be certain before taking the synology box down.

7. Move 6TB drives to OMV; Full wipe; set EXT4; setup rsync between them (presumably sdd and sde)If you get the first set of drives working, this step will be easy.

___________________________________________________________

Again you could use the synology box as a fully independent backup to OMV, using Remote Mount and (push) Rsync' network shares to synology box. Throw a 10TB drive in it and, with one 6TB drive, you can duplicate 16TB of data on two different machines. I don't know what model you have but a couple synology models can be converted to OMV. (Do a google search with your model and OMV as search terms.)
____________________________________________________________

Rsync Options:
If the job gets interrupted for some reason, "keep partially transferred files" will allow the next job to pick up where the first job was interrupted. Important for a really large transfer.
Also, if you add files to the OMV destination, but not the synology box, turn Delete files off. Otherwise if the files don't exist on the synology box (the source) they'll be deleted from the destination (OMV). Once the transfer is complete, this job should be deleted or reversed to push files the other way (for backup).

That should get you started.

I hope you didn't erase the synology share - which is what would happen if you got it bass ackwards with the delete switch on. The delete cycle is brutally fast.

Check the synology share to see if it exists - with files in it.

Zitat von telijah

I did not get it backwards, everything is still there. Like I said, I placed the test file onto the local 10tb drive, and the rsync job did delete the test file as it does not exist on the Synology drive, so delete action direction was correct.

Bro - you don't know how relieved I am, for you... That would have been a disaster. So, let's change things - don't turn the delete switch on in Rsync, until you're sure you have two full copies!

Zitat von telijah

Please wait, syncing </srv/4fe94ee8-2e58-4596-bca6-4990fbf9a771/SynPlexMedia/> to </srv/dev-disk-by-label-plex01data/PlexMedia01> ..

Since you have WinSCP installed, use it to navigate to /srv/4fe94ee8-2e58-4596-bca6-4990fbf9a771/SynPlexMedia/. In taking a guess maybe this is the problem. (/srv/4fe94ee8-2e58-4596-bca6-4990fbf9a771/SynPlexMedia/ ) You might be looking into an empty or nonexistent folder on the source side.

Remember, the path in a shared folder, that's using Remote mount, is "/" This requires editing the default path entry when created.
In the following, the Device is the Remote Mount. Note the Path.

Zitat von telijah

Any reason I should not just do a copy from WinSCP on my Windows machine? I have the Synology share and the OMV share mapped in Windows...

No, you could but WinSCP doesn't do large copy jobs well. You could also simply open the two shares, on a windows machine, and copy and paste in chunks but that will take a long time and is subject to errors. My concern, in either case, would be that files or folders might be missed.

If going the copy route and you're comfortable on the CLI, I'd use "MC" (midnight commander) with OMV. (It's pretty easy to use, menu driven, and point and click with a mouse works as well.)
On the command line do:

apt-get install mc
then
mc

MC copies left to right. Since the remote mount exists, you should be able to navigate to the the following locations and copy files, left to right.

/srv/4fe94ee8-2e58-4596-bca6-4990fbf9a771/SynPlexMedia/ /srv/dev-disk-by-label-plex01data/PlexMedia01

Zitat von telijah

Son of a bitch, it was the Path, I didn't think of it the way you spelled it out but yes, in the Shared Folder setup, I had put in SynPlexMedia thinking it needed the name I gave it. Once I changed the Path to just be the forward slash, I can see it creating the file list now (using Preview mode first).

So the problem was,,, it worked EXACTLY the way you configured it.... (Just kidding - glad it worked out.)

Remote Mount takes you "there", to the root of the remote share or "/". No path is required for the shared folder to find the location. If something is added, it goes nowhere. (The default path, that auto-fills based on the name, doesn't help.)

Zitat von telijah

It is running now, with Delete turned off (although I am pretty confident in recognizing the direction of changes :)). I just wish there was a better way of knowing the overall status/progress than the window the OMV gui provides.

If you leave the GUI interface up, with the popup Rsync window, it will stay up and scroll away. Rsync is fast, efficient, and far better than WinSCP. Go away,, don't watch it. Depending on what you have it may take hours (1GB) or even a day or two (100mbs). There's a chance that something might happen that interrupts the transfer. (A reboot?) Not a big deal. Just run the job again. The only way to know that all is transferred is to run the job, later, and see nothing transferred.
(Much like what you saw in the beginning. )

Really, other than the manual pop-up box, there's no need for a progress indicator. It will happen. When automated, after awhile, you'll take it for granted - it's that reliable. And after the first transfer, it's very fast - changes only.
__________________________________________________________________________________

Now, I know you want to pull drives out of the Synology box, but (after all is transferred) give some thought to using it as an independent backup device. All you have to do now after the full transfer has taken place (make sure of that), is change the source and destination and (with all due caution) turn delete on.

After that, on a schedule you specify, Rsync will replicate the additions, deletions, changes, etc., you make on the OMV share, to the Synology share. As of the time of sync, the two shares will be a mirror.

If you want to use it as cold storage, you could turn the Synology box off, then bring it up and manually run the Rsyn job, from time to time. In such a case, your OMV box could crash and burn, and you'd only lose changes or additions since the last sync.

BTW - assuming you're running a Windows workgroup LAN:
If you don't want to see the Synology Box on your network, change its' workgroup name to "backup" (or something different from your current workgroup name.) I will still be on the network, but not visible to users in Network Explorer.

Cheers.

Well, you have a handle on it and know what you want to do.

My preference is to replicate the server entirely on a second device. If something goes wrong with the first, there's zero down time. Switch to the second. It's already there with identical content.
___________________________________________________________

BTW: you can Rsync Windows network shares in the same manner. It just takes a username and password with at least read access, if pulling, or write access, if pushing.
I don't use clients for storage because, with UrBackup storing client images, I'd be storing the same thing twice. (Eats up hard disk space on the server side.)

When configuring a Remote Mount, user names and passwords with "read only" access to a remote network share produce an error, but they still work. "Read only" is my preference because I don't want a backup device to have the ability to change anything at the main server.