Newb in need of some help removing disks with snapraid and mergerfs

    • OMV 3.x

    This site uses cookies. By continuing to browse this site, you are agreeing to our Cookie Policy.

    • Newb in need of some help removing disks with snapraid and mergerfs

      Gallery of settings for reference

      I discovered 2 of my old smaller drives are pretty bad off according to snapraid's SMART report. I built my first server with snapraid and mergerfs to pool a bunch of old drives. Then I added a new pair of 8tb drives after scouring Best Buy, one for storage and one for the new parity. Now I want to remove the 2 risky drives (device names sdd and sdb) permanently by copying their data to the sdc disk (still an old smaller disk but in good shape).

      Since I pooled the disks and made a snapraid array, I have no idea how to copy data from individual disks before removing them. Any suggestions?
    • Interesting that your drives have 0 errors and yet a high failure %. Almost all my drives have a handful of errors and yet the largest failure % is 10%.

      Anyway, I'd ssh into the box and copy the files using good ole cp as root. You could install midnight commander or use x11 port forwarding and a gui filemanager like Thunar. The options are many.

      Once the files are copied over, you can remove the drives from snapraid using the plugin gui and then resync.
      snapraid diff
      snapraid -h sync
      snapraid scrub
    • chuccck wrote:



      Since I pooled the disks and made a snapraid array, I have no idea how to copy data from individual disks before removing them. Any suggestions?
      in OMV you can share the pool that mergerFS do with all disk , but you can share disk individually too ( in fact only can share folders inside the disk, but can be enought).

      EG: You have 3 disk in a mergerFS pool to share the folder "Video", this mean that you have a folder named "Video" in each disk, that you can share by SMB to copy to other (new) disk.

      Sorry if this is not true en SnapRaid (I talk about MergerFS only)
      OMV 3.0.96 x64 on a HP T510, 16GB CF as Boot Disk & 32GB SSD 2,5" disk for Data, 4 GB RAM, CPU VIA EDEN X2 U4200 is x64 at 1GHz

      Post: HPT510 SlimNAS ; HOWTO Install Pi-Hole ; HOWTO install MLDonkey ; HOHTO Install ZFS-Plugin ; OMV_OldGUI ; ShellinaBOX ;
      Dockers: MLDonkey ; PiHole ;
      Videos: @TechnoDadLife
    • jollyrogr wrote:

      Interesting that your drives have 0 errors and yet a high failure %. Almost all my drives have a handful of errors and yet the largest failure % is 10%.

      Anyway, I'd ssh into the box and copy the files using good ole cp as root. You could install midnight commander or use x11 port forwarding and a gui filemanager like Thunar. The options are many.

      Once the files are copied over, you can remove the drives from snapraid using the plugin gui and then resync.
      snapraid diff
      snapraid -h sync
      snapraid scrub
      When I try this, I get a content error that says I am missing the disk that I deleted from the Snapraid setup

      Source Code

      1. Loading state from /srv/dev-disk-by-label-VOLUME2/snapraid.content...
      2. Decoding error in '/srv/dev-disk-by-label-VOLUME2/snapraid.content' at offset 89
      3. The file CRC is correct!
      4. Disk 'Data1' with uuid '04897033-cee2-4d9b-a4a2-fc4334d076b4' not present in the configuration file!
      5. If you have removed it from the configuration file, please restore it
    • Here's what the snapraid FAQ says about removing a disk:
      snapraid.it/faq#remdatadisk
      How can I remove a data disk from an existing array?

      To remove a data disk from the array do:
      • Change in the configuration file the related "disk" option to point to an empty directory
      • Remove from the configuration file any "content" option pointing to such disk
      • Run a "sync" command with the "-E, --force-empty" option:
        • snapraid sync -E
        • The "-E" option tells at SnapRAID to proceed even when detecting an empty disk.
      • When the "sync" command terminates, remove the "disk" option from the configuration file.
      Your array is now without any reference to the removed disk.

      So I'm not sure how that jives with using the plugin since the plugin edits the config file.
    • jollyrogr wrote:

      Here's what the snapraid FAQ says about removing a disk:
      snapraid.it/faq#remdatadisk
      How can I remove a data disk from an existing array?

      To remove a data disk from the array do:
      • Change in the configuration file the related "disk" option to point to an empty directory
      • Remove from the configuration file any "content" option pointing to such disk
      • Run a "sync" command with the "-E, --force-empty" option:
        • snapraid sync -E
        • The "-E" option tells at SnapRAID to proceed even when detecting an empty disk.
      • When the "sync" command terminates, remove the "disk" option from the configuration file.
      Your array is now without any reference to the removed disk.
      So I'm not sure how that jives with using the plugin since the plugin edits the config file.
      Yeah, that is where I get lost because I have done everything via the GUI plugin. If I knew how to get to the content files on the disks, I could just delete them and start new, but I have no idea. I only really know how to create a shared drive to access on windows later. I am not worried about having to rebuild the sync.
    • jollyrogr wrote:

      The content files are listed in snapraid.conf which you can see from the plugin gui. In order to do what you want to do, you need to be able to ssh to the box and perform some tasks on the remote terminal.
      OK, I have only tested the ssh once when I set it up but i think i know how to run it again. I was looking at something like WinSCP as well to find the content files.
    • OK, I used WinSCP to access the content files on each disk and delete them. Then the snapraid sync -E finally ran. I had to run normal sync again to get the content files to reappear. The 2nd sync didn't actually do anything but create those content files. Only took a few seconds. I suppose the parity drive didn't repeat those files I copied over right? I didn't wipe the parity disk before redoing the sync
    • I just did something similar, except I was replacing old small drives with newer, large drives. These are the relevant steps from my process and it seems to have worked so far:

      • Remove references to the "bad" drives in both SnapRAID and MergerFS (through the GUI to make sure no data is being written to these drives)
      • rsync -avxHAWX --numeric-ids --info=progress2 /srv/olddiskpartition /srv/newdiskpartition
      • Run a SnapRAID diff (or check to be extra careful) to ensure all files are present
      • When you're sure that all data has transferred successfully without corruption, etc, run a sync with your new SnapRAID config (old drives gone)
      • Unmount the old drives in the Filesystem GUI





      My process was a bit longer because I was adding new drives, not using one already present on my server, but I think this should be simple enough. Either run the rsync locally or make sure to use screen if via SSH.
    • I guess I could have looked at the official FAQ before starting my process:

      How can I replace a data disk?

      If you lost the disk, see the recovering section in the manual. Otherwise, copy all the files in the new disk, maintaining the same directory structure and names.


      In Linux use the command:


      cp -av /from_dir/. /to_dir


      WARNING! If you want to use the 'rsync' or 'mc' commands, ensure that they are recent ones. 'rsync' must be at least version 3.1.0 (2013), and 'mc' at least version 4.8.19 (2017). Older versions are not able to copy the sub-second timestamps.


      In Windows use the command:


      robocopy F:\from_dir T:\to_dir /e /copyall


      Then change the SnapRAID configuration to point the disk at the new mount point and run a "diff" command to ensure that everything is placed correctly.


      snapraid diff


      If "diff" reports some "added" or "removed" files likely you have copied files with a different directory structure. In such case you have to fix it, until "diff" reports only "equal", "restored" or "copied" files.


      Now you can check that the files were really copied correctly, you can run a "check" audit command limited to the replaced disk to check the check-sum of all the files.


      snapraid check -a -d DISK_NAME


      Finally you can run "sync" to update the SnapRAID state. This command will be almost immediate.


      snapraid sync