can't bind in Raid5 after reinstall from OMV3

    • OMV 3.x

    This site uses cookies. By continuing to browse this site, you are agreeing to our Cookie Policy.

    • can't bind in Raid5 after reinstall from OMV3

      Hy guys,

      after an reinstall from my NAS to OMV3, I can't bind in/mount my old Raid5.

      What have I done before, after the reinstall my Physical Disks were shown but not in the Raid
      Management. So I start with the commands from this Post to find the issue. The result was that the superblock on devices has an issue.
      This I solve with the below commads for my devices sda,sdb & sdd


      Source Code

      1. mdadm --zero-superblock /dev/sda[bd]
      2. and
      3. dd if=/dev/zero of=/dev/sda[bd] bs=1K count=1024
      now I see my Raid5 in the Raid management, but I can't mount it in the file system. the mount button is grey

      So how must i proceed?


      Below the current status



      cat /proc/mdstat output

      Source Code

      1. root@DoggiNas:~# cat /proc/mdstat
      2. Personalities : [raid6] [raid5] [raid4]
      3. md127 : active (auto-read-only) raid5 sdd[0] sda[2] sdb[1]
      4. 5860270080 blocks super 1.2 level 5, 512k chunk, algorithm 2 [3/3] [UUU]
      5. bitmap: 0/22 pages [0KB], 65536KB chunk
      6. unused devices: <none>

      blkid output

      Source Code

      1. root@DoggiNas:~# blkid
      2. /dev/sdb: UUID="de78e9c6-c5a2-647e-c01c-f1092ad82856" UUID_SUB="b159821d-29ff-751b-d736-63e0bb49cf05" LABEL="DoggiNas:doggi" TYPE="linux_raid_member"
      3. /dev/sda: UUID="de78e9c6-c5a2-647e-c01c-f1092ad82856" UUID_SUB="7f0aba23-2979-7069-f5ac-a365ec4c5f55" LABEL="DoggiNas:doggi" TYPE="linux_raid_member"
      4. /dev/sdd: UUID="de78e9c6-c5a2-647e-c01c-f1092ad82856" UUID_SUB="89f2e673-dc6d-e87e-d4f3-13de30e2973d" LABEL="DoggiNas:doggi" TYPE="linux_raid_member"
      5. /dev/sdc1: UUID="6a490532-3b24-46aa-a584-ae49e6a07abe" TYPE="ext4" PARTUUID="a21f89a7-01"
      6. /dev/sdc5: UUID="90d69f5d-117d-47ca-81a5-436ef36083e2" TYPE="swap" PARTUUID="a21f89a7-05"


      fdisk -l output

      Source Code

      1. root@DoggiNas:~# fdisk -l
      2. Disk /dev/sdb: 2,7 TiB, 3000592982016 bytes, 5860533168 sectors
      3. Units: sectors of 1 * 512 = 512 bytes
      4. Sector size (logical/physical): 512 bytes / 4096 bytes
      5. I/O size (minimum/optimal): 4096 bytes / 4096 bytes
      6. Disk /dev/sda: 2,7 TiB, 3000592982016 bytes, 5860533168 sectors
      7. Units: sectors of 1 * 512 = 512 bytes
      8. Sector size (logical/physical): 512 bytes / 4096 bytes
      9. I/O size (minimum/optimal): 4096 bytes / 4096 bytes
      10. Disk /dev/sdd: 2,7 TiB, 3000592982016 bytes, 5860533168 sectors
      11. Units: sectors of 1 * 512 = 512 bytes
      12. Sector size (logical/physical): 512 bytes / 4096 bytes
      13. I/O size (minimum/optimal): 4096 bytes / 4096 bytes
      14. Disk /dev/sdc: 55,9 GiB, 60022480896 bytes, 117231408 sectors
      15. Units: sectors of 1 * 512 = 512 bytes
      16. Sector size (logical/physical): 512 bytes / 512 bytes
      17. I/O size (minimum/optimal): 512 bytes / 512 bytes
      18. Disklabel type: dos
      19. Disk identifier: 0xa21f89a7
      20. Device Boot Start End Sectors Size Id Type
      21. /dev/sdc1 * 2048 112383999 112381952 53,6G 83 Linux
      22. /dev/sdc2 112386046 117229567 4843522 2,3G 5 Extended
      23. /dev/sdc5 112386048 117229567 4843520 2,3G 82 Linux swap / Solaris
      24. Disk /dev/md127: 5,5 TiB, 6000916561920 bytes, 11720540160 sectors
      25. Units: sectors of 1 * 512 = 512 bytes
      26. Sector size (logical/physical): 512 bytes / 4096 bytes
      27. I/O size (minimum/optimal): 524288 bytes / 1048576 bytes
      Display All

      cat /etc/mdadm/mdadm.conf output

      Source Code

      1. root@DoggiNas:~# cat /etc/mdadm/mdadm.conf# mdadm.conf
      2. ## Please refer to mdadm.conf(5) for information about this file.
      3. #
      4. # by default, scan all partitions (/proc/partitions) for MD superblocks.
      5. # alternatively, specify devices to scan, using wildcards if desired.
      6. # Note, if no DEVICE line is present, then "DEVICE partitions" is assumed.
      7. # To avoid the auto-assembly of RAID devices a pattern that CAN'T match is
      8. # used if no RAID devices are configured.
      9. DEVICE partitions
      10. # auto-create devices with Debian standard permissions
      11. CREATE owner=root group=disk mode=0660 auto=yes
      12. # automatically tag new arrays as belonging to the local system
      13. HOMEHOST <system>
      14. # definitions of existing MD arrays
      Display All

      mdadm --detail --scan --verbose output

      Source Code

      1. root@DoggiNas:~# mdadm --detail --scan --verbose
      2. ARRAY /dev/md/doggi level=raid5 num-devices=3 metadata=1.2 name=DoggiNas:doggi UUID=de78e9c6:c5a2647e:c01cf109:2ad82856
      3. devices=/dev/sda,/dev/sdb,/dev/sdd

      The post was edited 3 times, last by vandog ().

    • ok THX
      I have done this, but the raid5 is still not avaible in the filesystem.
      and my device sdc is shown but I can do nothing


      Where make I an thinking mistake ?

      by the way when I restart the NAS md127 is again as active (auto-read-only) :(
      so I repeat mdadm --readwrite /dev/md127



      mdadm --assemble --scan o utput

      Source Code

      1. root@DoggiNas:~# mdadm --assemble --scan
      2. mdadm: No arrays found in config file or automatically

      The post was edited 1 time, last by vandog ().

    • Did you look at the output of cat /proc/mdstat to see if it came out of auto-read-only mode? Look at dmesg to check for any errors.
      omv 4.0.11 arrakis | 64 bit | 4.13 backports kernel | omvextrasorg 4.1.0
      omv-extras.org plugins source code and issue tracker - github.com/OpenMediaVault-Plugin-Developers

      Please don't PM for support... Too many PMs!
    • Now I'm back
      You see, after start from the NAS , it is in the auto-read-only mode,
      than I type the command "...--rewrite...." , and the output is active Raid.

      Source Code

      1. root@DoggiNas:~# cat /proc/mdstat
      2. Personalities : [raid6] [raid5] [raid4]
      3. md127 : active (auto-read-only) raid5 sdd[0] sda[2] sdb[1]
      4. 5860270080 blocks super 1.2 level 5, 512k chunk, algorithm 2 [3/3] [UUU]
      5. bitmap: 0/22 pages [0KB], 65536KB chunk
      6. unused devices: <none>
      7. root@DoggiNas:~# mdadm --readwrite /dev/md127
      8. root@DoggiNas:~# cat /proc/mdstat
      9. Personalities : [raid6] [raid5] [raid4]
      10. md127 : active raid5 sdd[0] sda[2] sdb[1]
      11. 5860270080 blocks super 1.2 level 5, 512k chunk, algorithm 2 [3/3] [UUU]
      12. bitmap: 0/22 pages [0KB], 65536KB chunk
      13. unused devices: <none>
      Display All


      And here the output from dmesg
      Range from usd to systemd
      the complete log is in the attachment

      Source Code

      1. [ 1.523408] ata3.00: ATA-10: TEAM L3 SSD 60GB, S9FM01.8, max UDMA/133
      2. [ 1.523419] ata3.00: 117231408 sectors, multi 16: LBA48 NCQ (depth 31/32), AA
      3. [ 1.523509] ata4.00: ATA-9: WDC WD30EFRX-68EUZN0, 82.00A82, max UDMA/133
      4. [ 1.523519] ata4.00: 5860533168 sectors, multi 16: LBA48 NCQ (depth 31/32), A
      5. A
      6. [ 1.523538] ata2.00: ATA-9: WDC WD30EFRX-68EUZN0, 82.00A82, max UDMA/133
      7. [ 1.523552] ata2.00: 5860533168 sectors, multi 16: LBA48 NCQ (depth 31/32), A
      8. A
      9. [ 1.523815] ata1.00: ATA-9: WDC WD30EFRX-68EUZN0, 82.00A82, max UDMA/133
      10. [ 1.523823] ata1.00: 5860533168 sectors, multi 16: LBA48 NCQ (depth 31/32), A
      11. A
      12. [ 1.523884] ata3.00: configured for UDMA/133
      13. [ 1.524887] ata2.00: configured for UDMA/133
      14. [ 1.524923] ata4.00: configured for UDMA/133
      15. [ 1.524952] ata1.00: configured for UDMA/133
      16. [ 1.525246] scsi 0:0:0:0: Direct-Access ATA WDC WD30EFRX-68E 0A82 PQ
      17. : 0 ANSI: 5
      18. [ 1.547546] scsi 1:0:0:0: Direct-Access ATA WDC WD30EFRX-68E 0A82 PQ
      19. : 0 ANSI: 5
      20. [ 1.575595] scsi 2:0:0:0: Direct-Access ATA TEAM L3 SSD 60GB 01.8 PQ
      21. : 0 ANSI: 5
      22. [ 1.603340] scsi 3:0:0:0: Direct-Access ATA WDC WD30EFRX-68E 0A82 PQ
      23. : 0 ANSI: 5
      24. [ 1.611337] usb 1-1: New USB device found, idVendor=8087, idProduct=8009
      25. [ 1.611340] usb 1-1: New USB device strings: Mfr=0, Product=0, SerialNumber=0
      26. [ 1.611743] hub 1-1:1.0: USB hub found
      27. [ 1.611871] hub 1-1:1.0: 6 ports detected
      28. [ 1.627341] usb 2-1: New USB device found, idVendor=8087, idProduct=8001
      29. [ 1.627355] usb 2-1: New USB device strings: Mfr=0, Product=0, SerialNumber=0
      30. [ 1.627603] hub 2-1:1.0: USB hub found
      31. [ 1.627661] hub 2-1:1.0: 8 ports detected
      32. [ 1.634928] sd 0:0:0:0: [sda] 5860533168 512-byte logical blocks: (3.00 TB/2.
      33. 73 TiB)
      34. [ 1.634930] sd 0:0:0:0: [sda] 4096-byte physical blocks
      35. [ 1.634958] sd 0:0:0:0: [sda] Write Protect is off
      36. [ 1.634960] sd 0:0:0:0: [sda] Mode Sense: 00 3a 00 00
      37. [ 1.634971] sd 0:0:0:0: [sda] Write cache: enabled, read cache: enabled, does
      38. n't support DPO or FUA
      39. [ 1.635093] sd 1:0:0:0: [sdb] 5860533168 512-byte logical blocks: (3.00 TB/2.
      40. 73 TiB)
      41. [ 1.635095] sd 1:0:0:0: [sdb] 4096-byte physical blocks
      42. [ 1.635120] sd 1:0:0:0: [sdb] Write Protect is off
      43. [ 1.635122] sd 1:0:0:0: [sdb] Mode Sense: 00 3a 00 00
      44. [ 1.635132] sd 1:0:0:0: [sdb] Write cache: enabled, read cache: enabled, does
      45. n't support DPO or FUA
      46. [ 1.635318] sd 3:0:0:0: [sdd] 5860533168 512-byte logical blocks: (3.00 TB/2.
      47. 73 TiB)
      48. [ 1.635319] sd 2:0:0:0: [sdc] 117231408 512-byte logical blocks: (60.0 GB/55.
      49. 9 GiB)
      50. [ 1.635321] sd 3:0:0:0: [sdd] 4096-byte physical blocks
      51. [ 1.635349] sd 3:0:0:0: [sdd] Write Protect is off
      52. [ 1.635351] sd 3:0:0:0: [sdd] Mode Sense: 00 3a 00 00
      53. [ 1.635355] sd 2:0:0:0: [sdc] Write Protect is off
      54. [ 1.635357] sd 2:0:0:0: [sdc] Mode Sense: 00 3a 00 00
      55. [ 1.635364] sd 3:0:0:0: [sdd] Write cache: enabled, read cache: enabled, does
      56. n't support DPO or FUA
      57. [ 1.635368] sd 2:0:0:0: [sdc] Write cache: enabled, read cache: enabled, does
      58. n't support DPO or FUA
      59. [ 1.635583] sd 0:0:0:0: Attached scsi generic sg0 type 0
      60. [ 1.635612] sd 1:0:0:0: Attached scsi generic sg1 type 0
      61. [ 1.635637] sd 2:0:0:0: Attached scsi generic sg2 type 0
      62. [ 1.635658] sd 3:0:0:0: Attached scsi generic sg3 type 0
      63. [ 1.636092] sdc: sdc1 sdc2 < sdc5 >
      64. [ 1.636297] sd 2:0:0:0: [sdc] Attached SCSI disk
      65. [ 1.641954] sd 1:0:0:0: [sdb] Attached SCSI disk
      66. [ 1.643626] sd 0:0:0:0: [sda] Attached SCSI disk
      67. [ 1.645705] sd 3:0:0:0: [sdd] Attached SCSI disk
      68. [ 1.687769] md: md127 stopped.
      69. [ 1.688378] md: bind<sdb>
      70. [ 1.688708] md: bind<sda>
      71. [ 1.688788] md: bind<sdd>
      72. [ 1.758879] raid6: sse2x1 gen() 11472 MB/s
      73. [ 1.826883] raid6: sse2x1 xor() 9006 MB/s
      74. [ 1.894886] raid6: sse2x2 gen() 14450 MB/s
      75. [ 1.962882] raid6: sse2x2 xor() 9832 MB/s
      76. [ 2.030888] raid6: sse2x4 gen() 16731 MB/s
      77. [ 2.098889] raid6: sse2x4 xor() 11895 MB/s
      78. [ 2.166890] raid6: avx2x1 gen() 22374 MB/s
      79. [ 2.234894] raid6: avx2x2 gen() 25729 MB/s
      80. [ 2.302896] raid6: avx2x4 gen() 29795 MB/s
      81. [ 2.302896] raid6: using algorithm avx2x4 gen() 29795 MB/s
      82. [ 2.302897] raid6: using avx2x2 recovery algorithm
      83. [ 2.302932] tsc: Refined TSC clocksource calibration: 3598.955 MHz
      84. [ 2.302934] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x33e077d5
      85. 718, max_idle_ns: 440795204040 ns
      86. [ 2.303284] xor: automatically using best checksumming function:
      87. [ 2.342897] avx : 36210.000 MB/sec
      88. [ 2.343223] async_tx: api initialized (async)
      89. [ 2.345744] md: raid6 personality registered for level 6
      90. [ 2.345745] md: raid5 personality registered for level 5
      91. [ 2.345746] md: raid4 personality registered for level 4
      92. [ 2.345889] md/raid:md127: device sdd operational as raid disk 0
      93. [ 2.345891] md/raid:md127: device sda operational as raid disk 2
      94. [ 2.345891] md/raid:md127: device sdb operational as raid disk 1
      95. [ 2.363081] md/raid:md127: allocated 3316kB
      96. [ 2.363097] md/raid:md127: raid level 5 active with 3 out of 3 devices, algor
      97. ithm 2
      98. [ 2.363098] RAID conf printout:
      99. [ 2.363099] --- level:5 rd:3 wd:3
      100. [ 2.363100] disk 0, o:1, dev:sdd
      101. [ 2.363101] disk 1, o:1, dev:sdb
      102. [ 2.363101] disk 2, o:1, dev:sda
      103. [ 2.363180] created bitmap (22 pages) for device md127
      104. [ 2.363522] md127: bitmap initialized from disk: read 2 pages, set 0 of 44711
      105. bits
      106. [ 2.401122] md127: detected capacity change from 0 to 6000916561920
      107. [ 2.494943] device-mapper: uevent: version 1.0.3
      108. [ 2.507037] device-mapper: ioctl: 4.34.0-ioctl (2015-10-28) initialised: dm-d
      109. evel@redhat.com
      110. [ 2.508422] random: lvm: uninitialized urandom read (4 bytes read, 81 bits of
      111. entropy available)
      112. [ 2.691103] Btrfs loaded, crc32c=crc32c-intel
      113. [ 2.691107] BTRFS: selftest: sectorsize: 4096 nodesize: 4096
      114. [ 2.691109] BTRFS: selftest: sectorsize: 4096 nodesize: 8192
      115. [ 2.691111] BTRFS: selftest: sectorsize: 4096 nodesize: 16384
      116. [ 2.691113] BTRFS: selftest: sectorsize: 4096 nodesize: 32768
      117. [ 2.691115] BTRFS: selftest: sectorsize: 4096 nodesize: 65536
      118. [ 2.839025] PM: Starting manual resume from disk
      119. [ 2.839036] PM: Hibernation image partition 8:37 present
      120. [ 2.839039] PM: Looking for hibernation image.
      121. [ 2.839337] PM: Image not found (code -22)
      122. [ 2.839343] PM: Hibernation image not present or could not be loaded.
      123. [ 3.000887] EXT4-fs (sdc1): mounted filesystem with ordered data mode. Opts:
      124. (null)
      Display All
      Hm I don't see any errors, but I don't really know what I must looking for :)
      Files
      • dmesglog.txt

        (66.65 kB, downloaded 98 times, last: )
    • I'm guessing you shut down the server all the time? Do you let it finishing syncing once it is in readwrite mode? Not sure why it is going into readwrite mode. I would try update-initramfs -u before rebooting too.
      omv 4.0.11 arrakis | 64 bit | 4.13 backports kernel | omvextrasorg 4.1.0
      omv-extras.org plugins source code and issue tracker - github.com/OpenMediaVault-Plugin-Developers

      Please don't PM for support... Too many PMs!
    • OMG I think I have done a big thinking error. Correct me ryecoaaron if I not right.

      I have setup a new OMV3 system and I want set my old Raid5 in this new system.
      The new system has no array.
      Would I choose my old Raid5 to be create a new file sytem , all my datas will be erase, correct ?





      I thought I could setup a new system and then I can mount my old Raid5 with all of my data.

      This thread LINK open my eyes about my issue.


      Is there an way to create a new array with the old Raid5 without to loose the data?
    • Your array should still have the old filesystem on it. If you create a new filesystem, it erases everything. I would try rebooting to see if the array comes backup correctly first. If it does and you haven't create a new filesystem already, post the output of blkid.
      omv 4.0.11 arrakis | 64 bit | 4.13 backports kernel | omvextrasorg 4.1.0
      omv-extras.org plugins source code and issue tracker - github.com/OpenMediaVault-Plugin-Developers

      Please don't PM for support... Too many PMs!
    • Ok I have reboot
      and this is the output from blkid

      Source Code

      1. root@DoggiNas:~# blkid
      2. /dev/sda: UUID="de78e9c6-c5a2-647e-c01c-f1092ad82856" UUID_SUB="7f0aba23-2979-70
      3. 69-f5ac-a365ec4c5f55" LABEL="DoggiNas:doggi" TYPE="linux_raid_member"
      4. /dev/sdb: UUID="de78e9c6-c5a2-647e-c01c-f1092ad82856" UUID_SUB="b159821d-29ff-75 1b-d736-63e0bb49cf05" LABEL="DoggiNas:doggi" TYPE="linux_raid_member"
      5. /dev/sdd: UUID="de78e9c6-c5a2-647e-c01c-f1092ad82856" UUID_SUB="89f2e673-dc6d-e8 7e-d4f3-13de30e2973d" LABEL="DoggiNas:doggi" TYPE="linux_raid_member"
      6. /dev/sdc1: UUID="6a490532-3b24-46aa-a584-ae49e6a07abe" TYPE="ext4" PARTUUID="a21f89a7-01"
      7. /dev/sdc5: UUID="90d69f5d-117d-47ca-81a5-436ef36083e2" TYPE="swap" PARTUUID="a21f89a7-05"
      8. root@DoggiNas:~#
      Display All