Big Data Loss with OMV and Snapraid

    • OMV 1.0

    This site uses cookies. By continuing to browse this site, you are agreeing to our Cookie Policy.

    • Big Data Loss with OMV and Snapraid

      Hello Guys,

      i really need your help, cause since yesterday im missing about 6 GB of Data ( Mostly *.mkv & *.jpg) on my system.

      Here is my setup:

      1x 60 GB SSD with OMV installed
      1x WD RED 3 TB as Parity
      3x WD Red 3 TB as Content + Data

      The diskspace on my 3 Content + Data drive has been like: Disk 1: 92% Usage Disk 2: 94% Usage Disk 3: 50 % Usage.
      The diskspace on my Parity Drive is: 90 % Usage

      On Saturday i did a full sync with Snapraid. The result was 12 writing erros. Therefor i run scrub in Snapraid. Scrub ran without any problem and i still had access to all my data. Yesterday i installed Calibre and made a mistake while choosing the correct folder. Therefore caliber changed the ownership for half of my files and in the end my system crashed while uninstalling calibre.

      For me it was no problem, i just run clonezilla and restored an backup of the OS before the installation of calibre.

      After the backup booted i was pretty shocked, as on my Datadrivers nearly everything was lost. Disk Usage on Disk 1: 20 % ond Disk 2: 25 % and ond Disk 3: 50 %. Parity is still 90%

      Only the Data on Disk 3 still fully exists...the rest on the other two disks is more or less fully gone.

      I then tried with snapraid FIX to recover the data. After 12 hours of running it finished today, with restoring about 100 gb of 6 TB !!


      I've checked the disks and according to Crystal Disk they are fine so i don't know where the data has gone or what happened to it.

      Is there any other way to recover my data then using snapraid FIX ?
      Software:
      OMV 3.0 / Plex / Snapraid / Tvheadend / ecoDMS / sonar / sabnzbd
      Hardware:
      ASRock C2750D4I / 2x Kingston KVR16E11/8 / Dell PERC H200 Crossflashed / 8x WD Red 3 TB / 60GB TeamGroup Dark L3 SSD / Samsung 750 EVO 256 GB SSD / Sundtek Sky TV Ultimate 6 DVB-S2
    • you can use "extundelete" and see, what it can bring back to you

      What exactly did you choose wrong in the callibre install?
      "Glowing days. Don't cry because they are over. Smile because they happened." - Confucius

      Server: 1x 32GB SSD (system) - 5x 2TB Data - 1x 2TB Snapraid-Parity - latest OMV 1.x
      No Support through PM
      Tutorials --- Howto install OMV-Extras --- Upgrade/Update-Problems --- If autoshutdown doesn' -work
    • I am wondering the same thing?? Did you use the same shared folder for everything? It does change the owner of the files in its shared folder but never deletes anything.

      ** edit - I take that back. It does delete the user which would delete its home folder. If you had other stuff in that folder, it would delete it... Guess I will be removing the user deletion from all plugins. Very sorry that your data was deleted :(
      omv 4.1.13 arrakis | 64 bit | 4.15 proxmox kernel | omvextrasorg 4.1.13
      omv-extras.org plugins source code and issue tracker - github

      Please read this before posting a question and this and this for docker questions.
      Please don't PM for support... Too many PMs!

      The post was edited 2 times, last by ryecoaaron ().

    • Hi Guys,

      i guess it happend like this:

      i have a pool folder called storage. In this folder i have different subfolders like: , movies, anime etc.

      After the calibre installation i wanted to created a new shared folder in this subfolder called ebook. but instead of putting the path like: //storage/ebook it was //storage.

      So Calibre took over the ownership of everything in the subfolder.

      When i realized my mistake i first removed the shared folder created wrongly for calibre. Then im asked to only delete the share or also recrusivly the whole folder. I've chosen only the remove the share.

      Till this point everythin was still ok and the data still existing.

      But then when i uninstalled Calibre i guess while deleting the User Calibre it also deleted everything with the ownership of calibre ?!?


      Maybe this was my mistake or problem :(

      But i thought with Snapraid it would be no problem to get the data back...but maybe i missunderstood something as i only received a few gigs back :-/

      Is "extundelete" similar to photorec ?
      Software:
      OMV 3.0 / Plex / Snapraid / Tvheadend / ecoDMS / sonar / sabnzbd
      Hardware:
      ASRock C2750D4I / 2x Kingston KVR16E11/8 / Dell PERC H200 Crossflashed / 8x WD Red 3 TB / 60GB TeamGroup Dark L3 SSD / Samsung 750 EVO 256 GB SSD / Sundtek Sky TV Ultimate 6 DVB-S2
    • When you change the folder in calibre, that new folder becomes calibre's new home folder. When the plugin is uninstalled (changed now), it removed the user (and its home directory). Since you had everything in it, it removed everything. The plugin no longer removes the home folder so this should not happen again. While the plugin should have had its own shared folder (which you tried to do), it really shouldn't delete the user on uninstallation from this experience. So, it won't anymore.

      I don't use snapraid but the fix command should've recovered the data from what I read. Maybe some help from the snapraid author would be beneficial.

      extundelete is like photorec except it recovers filenames/directory names.
      omv 4.1.13 arrakis | 64 bit | 4.15 proxmox kernel | omvextrasorg 4.1.13
      omv-extras.org plugins source code and issue tracker - github

      Please read this before posting a question and this and this for docker questions.
      Please don't PM for support... Too many PMs!
    • I will write to the Snapraid Author aswell, but friends told me Snapraid is only usefull i've i loose data on one hard disk. I've i loose data on two hard disk at the same time, there is no chance to revert it properly with only one parity drive. I don't know why, but i now tried the second time FIX command and it always gives me unreacoverable.....


      As im not familiar with the linux comands ( therefore i asked about photorec) do you know a good tutorial for extundelete ?


      Anyway thank you for your help and the explanation.
      Software:
      OMV 3.0 / Plex / Snapraid / Tvheadend / ecoDMS / sonar / sabnzbd
      Hardware:
      ASRock C2750D4I / 2x Kingston KVR16E11/8 / Dell PERC H200 Crossflashed / 8x WD Red 3 TB / 60GB TeamGroup Dark L3 SSD / Samsung 750 EVO 256 GB SSD / Sundtek Sky TV Ultimate 6 DVB-S2
    • I would boot systemrescuecd and do the following:

      - mount a drive to recover files TO: mount /dev/sdXY /mnt/backup
      - change to that directory: cd /mnt/backup
      - extundelete --restore-all /dev/sdWZ

      Substitute XY and WZ for the actual partition letters and numbers. You may have to run it more than once if the files were in a pool covering multiple disks. If you post fdisk -l and cat /etc/fstab, I can help more.
      omv 4.1.13 arrakis | 64 bit | 4.15 proxmox kernel | omvextrasorg 4.1.13
      omv-extras.org plugins source code and issue tracker - github

      Please read this before posting a question and this and this for docker questions.
      Please don't PM for support... Too many PMs!
    • SnapRAID can only restore files from 1 lost HDD, if one Parity-drive is used.
      RAID is no backup: petemarovichimages.com/2013/11…id-as-your-backup-system/
      but: your data shouldn't be deleted by uninstalling a plugin
      This is fixed on github

      "Glowing days. Don't cry because they are over. Smile because they happened." - Confucius

      Server: 1x 32GB SSD (system) - 5x 2TB Data - 1x 2TB Snapraid-Parity - latest OMV 1.x
      No Support through PM
      Tutorials --- Howto install OMV-Extras --- Upgrade/Update-Problems --- If autoshutdown doesn' -work

      The post was edited 1 time, last by Solo0815 ().

    • The "data being deleted by plugin" issue has been thoroughly covered and I mentioned it a few posts back when I explained what happened. This plugin had plenty of time and testing and this issue still slipped through. It is fixed on github and new version already in repo.
      omv 4.1.13 arrakis | 64 bit | 4.15 proxmox kernel | omvextrasorg 4.1.13
      omv-extras.org plugins source code and issue tracker - github

      Please read this before posting a question and this and this for docker questions.
      Please don't PM for support... Too many PMs!
    • ryecoaaron wrote:

      I would boot systemrescuecd and do the following:

      - mount a drive to recover files TO: mount /dev/sdXY /mnt/backup
      - change to that directory: cd /mnt/backup
      - extundelete --restore-all /dev/sdWZ

      Substitute XY and WZ for the actual partition letters and numbers. You may have to run it more than once if the files were in a pool covering multiple disks. If you post fdisk -l and cat /etc/fstab, I can help more.


      So i downloaded Systemrescuecd, bootet and used commands fdsik - l and cat /etc/fstab

      I also installed another 3 TB HDD ( Toshiba) to put the backup files onto.



      fdisk -l

      Source Code

      1. Disk /dev/sda: 55.9 GiB, 60022480896 bytes, 117231408 sectors
      2. Units: sectors of 1 * 512 = 512 bytes
      3. Sector size (logical/physical): 512 bytes / 512 bytes
      4. I/O size (minimum/optimal): 512 bytes / 512 bytes
      5. Disklabel type: dos
      6. Disk identifier: 0x000397bf
      7. Device Boot Start End Blocks Id System
      8. /dev/sda1 * 2048 112383999 56190976 83 Linux
      9. /dev/sda2 112386046 117229567 2421761 5 Extended
      10. /dev/sda5 112386048 117229567 2421760 82 Linux swap / Solaris
      11. Disk /dev/sdb: 2.7 TiB, 3000592982016 bytes, 5860533168 sectors
      12. Units: sectors of 1 * 512 = 512 bytes
      13. Sector size (logical/physical): 512 bytes / 4096 bytes
      14. I/O size (minimum/optimal): 4096 bytes / 4096 bytes
      15. Disklabel type: gpt
      16. Disk identifier: E23727CE-DDE8-4D5E-ABAF-6DFBD05BD8ED
      17. Device Start End Size Type
      18. /dev/sdb1 2048 5860533134 2.7T Linux filesystem
      19. Disk /dev/sdc: 2.7 TiB, 3000592982016 bytes, 5860533168 sectors
      20. Units: sectors of 1 * 512 = 512 bytes
      21. Sector size (logical/physical): 512 bytes / 4096 bytes
      22. I/O size (minimum/optimal): 4096 bytes / 4096 bytes
      23. Disklabel type: gpt
      24. Disk identifier: DA740759-66BA-4BD0-B76A-D63DA1649CB7
      25. Device Start End Size Type
      26. /dev/sdc1 2048 5860533134 2.7T Linux filesystem
      27. Disk /dev/sdd: 2.7 TiB, 3000592982016 bytes, 5860533168 sectors
      28. Units: sectors of 1 * 512 = 512 bytes
      29. Sector size (logical/physical): 512 bytes / 4096 bytes
      30. I/O size (minimum/optimal): 4096 bytes / 4096 bytes
      31. Disklabel type: gpt
      32. Disk identifier: 6CB05F8C-AB47-4528-ADC5-F406A418535B
      33. Device Start End Size Type
      34. /dev/sdd1 2048 5860533134 2.7T Linux filesystem
      35. Disk /dev/sde: 2.7 TiB, 3000592982016 bytes, 5860533168 sectors
      36. Units: sectors of 1 * 512 = 512 bytes
      37. Sector size (logical/physical): 512 bytes / 4096 bytes
      38. I/O size (minimum/optimal): 4096 bytes / 4096 bytes
      39. Disklabel type: gpt
      40. Disk identifier: EA47C666-4F69-4485-B40D-BCB6201B9C7A
      41. Device Start End Size Type
      42. /dev/sde1 2048 5860533134 2.7T Linux filesystem
      43. Disk /dev/sdf: 2.7 TiB, 3000592982016 bytes, 5860533168 sectors
      44. Units: sectors of 1 * 512 = 512 bytes
      45. Sector size (logical/physical): 512 bytes / 4096 bytes
      46. I/O size (minimum/optimal): 4096 bytes / 4096 bytes
      47. Disklabel type: gpt
      48. Disk identifier: D33D82C9-FF0A-434A-B910-6BF9DBD3275B
      49. Device Start End Size Type
      50. /dev/sdf1 2048 5860533134 2.7T Linux filesystem
      Display All


      cat /etc/fstab

      Source Code

      1. ####################################################
      2. ## ATTENTION: THIS IS THE FSTAB ON THE LIVECD ##
      3. ## PLEASE EDIT THE FSTAB at /mnt/gentoo/etc/fstab ##
      4. ####################################################
      5. tmpfs / tmpfs defaults 0 0
      6. /dev/fd0 /mnt/floppy auto noauto 0 0

      Software:
      OMV 3.0 / Plex / Snapraid / Tvheadend / ecoDMS / sonar / sabnzbd
      Hardware:
      ASRock C2750D4I / 2x Kingston KVR16E11/8 / Dell PERC H200 Crossflashed / 8x WD Red 3 TB / 60GB TeamGroup Dark L3 SSD / Samsung 750 EVO 256 GB SSD / Sundtek Sky TV Ultimate 6 DVB-S2

      The post was edited 1 time, last by writzthecat ().

    • Need a little more info...

      mount /dev/sda1 /mnt/gentoo
      cat /mnt/gentoo/etc/fstab
      cat /mnt/gentoo/etc/snapraid.conf
      omv 4.1.13 arrakis | 64 bit | 4.15 proxmox kernel | omvextrasorg 4.1.13
      omv-extras.org plugins source code and issue tracker - github

      Please read this before posting a question and this and this for docker questions.
      Please don't PM for support... Too many PMs!

    • cat /mnt/gentoo/etc/fstab

      Source Code

      1. # /etc/fstab: static file system information.
      2. #
      3. # Use 'blkid' to print the universally unique identifier for a
      4. # device; this may be used with UUID= as a more robust way to name devices
      5. # that works even if disks are added and removed. See fstab(5).
      6. #
      7. # <file system> <mount point> <type> <options> <dump> <pass>
      8. # / was on /dev/sda1 during installation
      9. UUID=be149662-abd5-4d5d-b532-14d091321812 / ext4 errors=remount-ro 0 1
      10. # swap was on /dev/sda5 during installation
      11. UUID=5d31da78-38c9-48e7-8aa0-bfa5c4b18235 none swap sw 0 0
      12. /dev/sr0 /media/cdrom0 udf,iso9660 user,noauto 0 0
      13. tmpfs /tmp tmpfs defaults 0 0
      14. # >>> [openmediavault]
      15. UUID=a7f543ef-6a8f-41f9-ac83-942085c9bfca /media/a7f543ef-6a8f-41f9-ac83-942085c9bfca ext4 defaults,nofail,acl,user_xattr,noexec,usrjquota=aquota.user,grpjquota=aquota.group,jqfmt=vfsv0 0 2
      16. UUID=0722de2e-cf4b-4a63-aece-f01f502b7c27 /media/0722de2e-cf4b-4a63-aece-f01f502b7c27 ext4 defaults,nofail,acl,user_xattr,noexec,usrjquota=aquota.user,grpjquota=aquota.group,jqfmt=vfsv0 0 2
      17. UUID=2f914808-85f6-4b9b-aee3-43cfad784960 /media/2f914808-85f6-4b9b-aee3-43cfad784960 ext4 defaults,nofail,acl,user_xattr,noexec,usrjquota=aquota.user,grpjquota=aquota.group,jqfmt=vfsv0 0 2
      18. UUID=858099db-5680-44d7-8cd3-79134be2157c /media/858099db-5680-44d7-8cd3-79134be2157c ext4 defaults,nofail,acl,user_xattr,noexec,usrjquota=aquota.user,grpjquota=aquota.group,jqfmt=vfsv0 0 2
      19. UUID=c7692c8e-2a3e-4ac2-b954-29c71ac43df9 /media/c7692c8e-2a3e-4ac2-b954-29c71ac43df9 ext4 defaults,nofail,acl,user_xattr,noexec,usrjquota=aquota.user,grpjquota=aquota.group,jqfmt=vfsv0 0 2
      20. none /media/storage aufs br:/media/0722de2e-cf4b-4a63-aece-f01f502b7c27/d1=rw:/media/2f914808-85f6-4b9b-aee3-43cfad784960/d2=rw:/media/858099db-5680-44d7-8cd3-79134be2157c/d3=rw,sum,create=mfs,udba=reval 0 0
      21. /media/storage /media/0722de2e-cf4b-4a63-aece-f01f502b7c27/storage none bind 0 0
      22. # <<< [openmediavault]
      Display All


      cat /mnt/gentoo/etc/snapraid.conf

      Source Code

      1. # this file was automatically generated from
      2. # OpenMediaVault Kralizec 1.0.29
      3. # and 'openmediavault-snapraid'
      4. block_size 256
      5. autosave 0
      6. #####################################################################
      7. # OMV-Name: Parity1 Drive Label: Parity1
      8. parity /media/a7f543ef-6a8f-41f9-ac83-942085c9bfca/snapraid.parity
      9. #####################################################################
      10. # OMV-Name: Disk1 Drive Label: DISK1
      11. content /media/0722de2e-cf4b-4a63-aece-f01f502b7c27/snapraid.content
      12. disk Disk1 /media/0722de2e-cf4b-4a63-aece-f01f502b7c27
      13. #####################################################################
      14. # OMV-Name: Disk2 Drive Label: DISK2
      15. content /media/2f914808-85f6-4b9b-aee3-43cfad784960/snapraid.content
      16. disk Disk2 /media/2f914808-85f6-4b9b-aee3-43cfad784960
      17. #####################################################################
      18. # OMV-Name: Disk3 Drive Label: DISK3
      19. content /media/858099db-5680-44d7-8cd3-79134be2157c/snapraid.content
      20. disk Disk3 /media/858099db-5680-44d7-8cd3-79134be2157c
      21. exclude *.bak
      22. exclude *.unrecoverable
      23. exclude /tmp/
      24. exclude lost+found/
      25. exclude .content
      26. exclude aquota.group
      27. exclude aquota.user
      28. exclude snapraid.conf*
      29. # Rule selected from /media/0722de2e-cf4b-4a63-aece-f01f502b7c27
      30. exclude /storage/
      31. # Rule selected from /media/0722de2e-cf4b-4a63-aece-f01f502b7c27
      32. exclude /plexmediaserver/
      Display All

      Software:
      OMV 3.0 / Plex / Snapraid / Tvheadend / ecoDMS / sonar / sabnzbd
      Hardware:
      ASRock C2750D4I / 2x Kingston KVR16E11/8 / Dell PERC H200 Crossflashed / 8x WD Red 3 TB / 60GB TeamGroup Dark L3 SSD / Samsung 750 EVO 256 GB SSD / Sundtek Sky TV Ultimate 6 DVB-S2

      The post was edited 1 time, last by writzthecat ().

    • From sysresccd. You won't need to leave sysresccd until the data is recovered.
      omv 4.1.13 arrakis | 64 bit | 4.15 proxmox kernel | omvextrasorg 4.1.13
      omv-extras.org plugins source code and issue tracker - github

      Please read this before posting a question and this and this for docker questions.
      Please don't PM for support... Too many PMs!
    • Here we go :)

      blkid

      Source Code

      1. /dev/loop0: TYPE="squashfs"
      2. /dev/sda1: UUID="be149662-abd5-4d5d-b532-14d091321812" TYPE="ext4" PARTUUID="000397bf-01"
      3. /dev/sda5: UUID="5d31da78-38c9-48e7-8aa0-bfa5c4b18235" TYPE="swap" PARTUUID="000397bf-05"
      4. /dev/sdb1: LABEL="Backup" UUID="c7692c8e-2a3e-4ac2-b954-29c71ac43df9" TYPE="ext4" PARTUUID="abbbbcb7-4831-4c62-b4ce-b8eefa757578"
      5. /dev/sdc1: LABEL="DISK3" UUID="858099db-5680-44d7-8cd3-79134be2157c" TYPE="ext4" PARTUUID="571fd11d-2fe9-46a0-a45e-32bfd5a677ef"
      6. /dev/sdd1: LABEL="DISK2" UUID="2f914808-85f6-4b9b-aee3-43cfad784960" TYPE="ext4" PARTUUID="e563015a-431e-488a-93ea-30960db9df19"
      7. /dev/sde1: LABEL="DISK1" UUID="0722de2e-cf4b-4a63-aece-f01f502b7c27" TYPE="ext4" PARTUUID="da3e312c-be1c-42b3-bd40-810f5be6ce4b"
      8. /dev/sdf1: LABEL="Parity1" UUID="a7f543ef-6a8f-41f9-ac83-942085c9bfca" TYPE="ext4" PARTUUID="c206d942-0953-4b93-bb1b-2239a8c42fe7"
      Software:
      OMV 3.0 / Plex / Snapraid / Tvheadend / ecoDMS / sonar / sabnzbd
      Hardware:
      ASRock C2750D4I / 2x Kingston KVR16E11/8 / Dell PERC H200 Crossflashed / 8x WD Red 3 TB / 60GB TeamGroup Dark L3 SSD / Samsung 750 EVO 256 GB SSD / Sundtek Sky TV Ultimate 6 DVB-S2
    • Ok.

      Source Code

      1. umount /mnt/gentoo
      2. mount /dev/sdb1 /mnt/backup
      3. mkdir -p /mnt/backup/recover
      4. cd /mnt/backup/recover
      5. mkdir disk1 disk2 disk3
      6. cd disk1
      7. extundelete --restore-all /dev/sde1
      8. cd ../disk2
      9. extundelete --restore-all /dev/sdd1
      10. cd ../disk3
      11. extundelete --restore-all /dev/sdc1
      Display All


      If there are no missing files from disk3, I guess you don't need to do the last two steps.
      omv 4.1.13 arrakis | 64 bit | 4.15 proxmox kernel | omvextrasorg 4.1.13
      omv-extras.org plugins source code and issue tracker - github

      Please read this before posting a question and this and this for docker questions.
      Please don't PM for support... Too many PMs!