New Boot SSD, Reconnect Old HDDs

  • I have successfully installed a new SSD for my boot drive (my previous boot had been on a USB stick under version 3, which failed after a power outage) and reconnected my hard drives. I have mounted the hard drives from the Web GUI, but they are not yet referenced. With the previous installation, I think I had all the drives mapped under a single virtual drive, but can't remember completely, or how I did that (I'm not Linux proficient). And all the shared folders were there as well. So, I need help on what to do to see those folders, and put the drives back into a virtual drive.

    • Offizieller Beitrag

    The unionfilesystems plugin aggregates 2 or more drives under a single mount point, which gives the appearance of a single drive.


    Is that what you used?


    After creating the common mount point (virtual drive) with unionfs, adding shared folders to it is the same as it would be with a physical drive.

  • Hey, thanks for replying FLMAXEY. The unionfilesystem plugin sounds VERY familiar, so I'm sure I used that. I have four drives: sdadisk1, sdadisk2, sdadisk3, sdadisk4. I found all the directories on sdadisk2 thru sdadisk4 (that is, identically named directories) so I'm assuming I unioned those three drives. So:

    • I also seem to remember that sdadisk1 was not in the union because the system wanted an available drive outside the union for something. Wish I could remember that. Any ideas?
    • I don't see the unionfilesystem in the plugins in version 4.1.3. Where is it now? Or maybe the question is, "how do I access it"?
    • If the identical directories currently exist on the three hard drives, and I "unionize" them, will I then have a single set of directories for access, and all the data will be there?

    Appreciate the help.

    • Offizieller Beitrag

    The only drive that would need to be outside the union would be the boot drive, but there's nothing wrong with having a data drive outside the union. There are use cases for it.
    _______________________________________________________________


    The unionfs plugin is available on OMV4. If you did a fresh build, you'll need to upload a file to activate OMV-Extras and go through a process that will enable all the available plugins, to get to the Unionfs plugin. The process is detailed in the User's Guide. It starts on page 36.


    (BTW; So that you're not doing disaster recovery again, take a look at the section on cloning boot drives.)

    If the identical directories currently exist on the three hard drives, and I "unionize" them, will I then have a single set of directories for access, and all the data will be there?

    Saying that you haven't changed anything on the previously unioned drives (moved, duplicated, or deleted data), and assuming you put all drives back into the union that were there before; I believe all data and the folder structures will be there as it was before your boot drive crashed.


    Unionfs (also known as mergerfs) does a logical "addition" of the contents (files and folders) of all drives, but does not duplicate data. By extension, when all drives are back in a union, all data should be there as it was before.


    Thereafter, it would be a matter of sharing folders that would exist in the union and doing SMB shares of those folders.


    (After the drives are merged) If you're not sure of your previous folder locations, look at the section in the User Guide on WinSCP. WinSCP is great for looking at directory structure.


    Using WinSCP - when you browse to drives under /srv , remember that you do not want to share data on your individual drives. Set up your shares ONLY from merged unionfs drive.

    • Offizieller Beitrag

    Is LMV2 the version 4.1.3 replacement for unionfilesystem?

    No. LVM2 is another drive merging scheme entirely. While some use it, I wouldn't.


    BTW: If you have an extra disk, that's not part of the Union AND it's the same size or larger than any disk in the union, you might want to give SNAPRAID a look.

  • Thanks for the feedback, and instruction.
    I have uninstalled LMV2, just to be safe.
    I now have the OMV-Extras installed, enabled the OMV-extras.org Testing, and have installed unionfilesystems 4.0.2.


    The drives are exactly where they were before the boot-drive crash. No shares or any other impact on those drives has taken place yet. The three drives I mentioned, sdadisk2, sdadisk3, and sdadisk4 all have identical folders on them. sdadisk1 does not, and that's why I think it was outside of the unionfs, although I am not sure why at this moment. But given that setup, is it possible I might have maybe done a RAID setup? I realize you don't know for sure, but was curious if that might also be a possibility.


    Because I was guided down the unionfs/or RAID path two years ago by members of the forum, I'm going to go back and read through all the postings. Maybe I'll get clarity there.


    thanks

    • Offizieller Beitrag

    It's good that you did away with the LVM2 idea. In the way that it works, it reminds me of traditional RAID without any benefits, other than aggregating drives.

    The drives are exactly where they were before the boot-drive crash. No shares or any other impact on those drives has taken place yet. The three drives I mentioned, sdadisk2, sdadisk3, and sdadisk4 all have identical folders on them. sdadisk1 does not, and that's why I think it was outside of the unionfs, although I am not sure why at this moment. But given that setup, is it possible I might have maybe done a RAID setup? I realize you don't know for sure, but was curious if that might also be a possibility.
    Because I was guided down the unionfs/or RAID path two years ago by members of the forum, I'm going to go back and read through all the postings. Maybe I'll get clarity there.


    thanks

    Those 3 drives with identical folders (but with different files in them if you looked close) would almost have to be a unionfs setup.


    If you assembled your 3 disks in a union (with unionfs), and your data structure is readable (files and folders) the only plausible reason why you've be remembering "RAID" would be if you were using SNAPRAID. (If using traditional RAID, you wouldn't see anything in single drive outside of an the array.) SNAPRAID is the only thing that would work with unionfs and would explain why there's one drive is outside union. The outside drive is the SNAPRAID parity drive. With unaltered drives, it should be easy to recreate your old setup. Unionfs, using the same folder structure, will simply add the files of the 3 disks together into a common larger drive.


    BTW: Searching the forum for old posts may be easier with Google, or a similar search engine, if you use the term OMV and maybe your forum user name, as part of the search criteria.


    I'd say "Good Luck", but I think you have a handle on it. :)

  • On Oct 24, 2016 after many installation issues were finally resolved, I wrote this (to ryecoarron, who was really helpful): "But, now that I have BTRFS formatted on all the drives, I can implement SnapRaid, set up Users, Backup, ..."


    So I'm thinking that I must have used Snapraid. I haven't found any reference to unionfilesystem in those old posts.
    But does unionfs work in conjunction with Snapraid?
    Or just use Snapraid by itself?
    Also, in the old posts, I was advised to use AHCI for my SATA configuration. But this time, after I installed OMV v4.1.3, I couldn't boot with the HDDs reconnected until I changed the configuration to ATA.


    So it looks like I'm supposed to do the unionfs on the three drives (sdadisk2, 3, 4) and then Snapraid the sdadisk1 with the union drive. Is that right?

    Einmal editiert, zuletzt von curious1 () aus folgendem Grund: Clarification

    • Offizieller Beitrag

    So it looks like I'm supposed to do the unionfs on the three drives (sdadisk2, 3, 4) and then Snapraid the sdadisk1 with the union drive. Is that right?

    Based on everything you'd said so far, your background checks, etc., I thing you're spot on doing exactly what you've outlined above. @ryecoaaron would have recommended this setup.


    The good thing about unionfs is, there's no penalty for using it. It won't change the drives or their contents in any way. It merely merges their contents together under a common mount point. You could do the union first, and setup SNAPRAID later, if you like.
    ___________________________________________________________________________________________


    If sdadisk1 is not part of the union (with the common directory structure you've noted on the other drives) it was most likely the SNAPRAID parity drive. SNAPRAID is an excellent choice as well, in that if you decide not to use it in the future, the contents of the protected union drive will be unaffected. (Their's no penalty if you decide to back out.)
    The only rule that must be observed is, to be a SNAPRAID parity drive; sdadisk 1 must be equal to, or larger, than the largest disk in the union (any one of sdadisk2,3, or 4). I'm guessing this is the case.

  • Thanks so much for your feedback FLMAXEY. Yes, the sdadisk1 that would be used for parity with Snapraid is the same size as the other three ... 1TB each.


    I will do the unionfs and then setup Snapraid. Will let you know how it turns out. BTW, is there a HOW TO for doing Snapraid from the Web GUI? I saw the manual that provides all the command line guidance, but would rather work from the GUI.


    See in the WebGUI that there are a number of parameters for setting up Snapraid.

    • Do I just take all the defaults?
    • Is there a guide for Snapraid in the WebGUI?
    • The "How to setupSnapRAID in OMV" plugin guide advises to setup a content file on each data-disk. Should I do that first on each drive, and then do the union, or do I just need a content file for the unioned file system (the unioned three disks)?

    5 Mal editiert, zuletzt von curious1 () aus folgendem Grund: clarification

    • Offizieller Beitrag

    Sorry, I'm out of town on the weekdays. I'm only home on the weekends.


    Given that time has passed, I'll assume you got the unionfs part set up.
    _________________________________________________________________________


    Here it is, a brief: SNAPRAID HOW-TO :)


    The following is from one of my backups servers. I'm not using unionfs on this box, but the SNAPRAID principle is the same.


    I label my disks, when I format them, to reflect their function. I have two disks, EXT401 and EXT402, to protect. (These disks are formatted EXT4, and they are disk 01 and 02. My "parity" disk is labeled SNAPRAID.


    **Note, in this case, you will not use the merged unionfs drive. You're protecting each of the individual disks in the union. SNAPRAID will give you rebuild protection for 1 disk failure. That's important if you think about it - unionfs is a merger of individual drives with a common directory structure, but unique content. Accordingly, individual disk protection is needed.**
    _________________________________________________________________________


    (After installing the SNAPRAID PLUGIN)


    - In the Drives tab, using the + Add button, add each of the disks in your unionfs setup, individually. sdadisk2, sdadisk3, and sdadisk4. Toggle the switches on (green) for content and data for each of these drives.
    - sdadisk1 is (as I understand it) your SNAPRAID parity drive. Toggle on content and parity for sdadisk1.



    Since it doesn't happen automatically, I trigger a "sync" once a week (on Monday) with the following scheduled job. (The following will work for you as well.) It can be run more or less frequent as you like. However, since it will work your drives (depending on how much has changed) and a few CPU cycles will be needed, it's probably best to schedule it for after-hours. In my case it's 00:01AM.



    That's is how I do it. Others, with better ideas, please chime in.
    ____________________________________________________________________


    With the basic set up done, the SNAPRAID page will make more sense.
    If there's a command line you want to run, from inside the GUI, the above will let you schedule it or (using the run button) run it on a manual basis.

  • Welcome back FLMAXEY!! Thanks for the comprehensive response. You're the best!


    So yes, I did do the unionfs and implemented SnapRAID. Parity is on sdadisk1. But I did the content on the unionfs of sdadisk2, sdadisk3, and sdadisk4. So if I'm reading you right, I should remove that particular content item and then do each of the disks individually. I'll do that now.

    • Offizieller Beitrag

    I suppose you could put a content file on the union but it would be a single file and subject to loss if the disk on which it's stored (in the union) fails.


    In the event of failure, multiple copies of the content file on individual disks will ensure a copy will be available in the event of any single disk failure.

  • OK FLMAXEY. Took care of the individual content/data disks in SnapRAID. AND went through the procedure for Windows 10 not recognizing OMV in the Network list of File Explorer. It still won't show up in my Network list, but the Shortcut I created did show me the folders. But, I can't access them ... ACCESS DENIED. I have gone through and updated the ACL for the shares to enable permission inheritance, and set recursive, but this didn't seem to help. Any ideas?


    Actually, I have gone back in and looked at privileges on the shared folders, and they had been altered somewhere along the line. So as I reset privileges, I can access them on my Windows 10 now. But sure wish the doggone thing would show up in my File Explorer.

    • Offizieller Beitrag

    So as I reset privileges, I can access them on my Windows 10 now. But sure wish the doggone thing would show up in my File Explorer.

    I am not kidding when I say, I tried to find the reason(s) why it's not possible to browse to an OMV server (in some cases). Hours were spent in research, examination (sniffing traffic), various test scenarios, etc. One of the more serious problems in trying to approach this is that I only own one Win 10 box and with the optional software OEM's install along with custom security settings (in the registry and other), no "one" Win 10 box is representative of what's out there.


    I quickly discovered that the problem is not a protocol issue (SMB / Samba) - it's a network discovery services issue.


    The conclusion was:
    There didn't seem to be a consistent way to guarantee network discovery of servers and clients, even with outlandish processes. (If you followed the link to the Windows 10 forum, running for years now, there are a lot of unhappy campers - who would stand one leg, wearing aluminum foil hats, in attempts to make it happen. :) )
    The reason, I believe, is that Microsoft is throwing wrenches into the machine intentionally, trying to force all (individual users and businesses) onto their unified workstation platform - Windows 10. Further, it appears that they're forcing older M$ server versions out the door as well.


    As you found, even the Win10 connect How-To is pretty extensive but it solves most of the potential issues with the SMB protocol for a broad cross section of users and it helps to prevent M$ meddling, in the background, as their numerous security updates are applied.

    • Offizieller Beitrag

    Now that you're on line again - don't forget backup. Your setup could be duplicated, easily, on a single 3TB drive. (USB connected external?)


    SNAPRAID is very good, but everything is fallible. The solution is a second full copy (I.E. backup).

  • - In the Drives tab, using the + Add button, add each of the disks in your unionfs setup, individually. sdadisk2, sdadisk3, and sdadisk4. Toggle the switches on (green) for content and data for each of these drives.
    - sdadisk1 is (as I understand it) your SNAPRAID parity drive. Toggle on content and parity for sdadisk1.


    If I can make a suggestion, I would not put the content file on parity disks, especially if your parity disk and data disks are all the same size (rather than the parity disk being larger). My reason for this is that the content file on the data disks takes up a relatively small amount of space on each disk that actual data cannot occupy. This allows the available space of the parity disk to be just slightly larger than that of the data disks, ensuring that you can never "max out" the parity drive.


    However, if you write that content file to all drives including parity, you could potentially end up maxing out the available space of the parity drive. Maybe it's inconsequential with the way SnapRAID is designed; I don't know how the program would react if there is no more space available for the parity file, but it sounds like an easy problem to avoid. Also, with three data disks, you'll probably have a sufficient number of copies of the content file that a fourth is not likely to become useful for rebuilding lost data.

  • Yes, next is backup. Wonder what I'll run into there, LOL. Think I'll get a 6TB drive ... they're cheap enough.


    I do wonder about why we would need content on the parity drive. If you could expand on that thought, FLMAXEY, I would appreciate hearing your take.


    Also, there is a Directory Service under Access Rights Management. Will that do anything to help?


    Thanks

Jetzt mitmachen!

Sie haben noch kein Benutzerkonto auf unserer Seite? Registrieren Sie sich kostenlos und nehmen Sie an unserer Community teil!