Adding storage drives and changing RAID setup to suit

    • OMV 1.0
    • Resolved
    • Adding storage drives and changing RAID setup to suit

      Hi there

      I will first of all say I am not an IT pro, but can follow instructions and generally not brick stuff!

      I have OMV 0.5.60 installed on an HP Micro Server. At the moment I have a pair of 3TB WD red drives installed, but these are nearing capacity in a straight mirrored RAID config.

      I'm intending to buy 2 more 3TB WD reds to fill the available bays. The server gets used to backup a desktop and laptop on a weekly basis and is largely used for media server duties around the house, so lots of music and large video files.

      Is there a walk through guide to adding additional drives and what steps I need to take in what order for OMV to recognise them?

      Given the above usage what RAID config is recommended? I'm happy to live with 1 drive failure to give maximum storage, but not sure what other factors I need to take into account?

      Grateful for any guidance - I haven't yet ordered the drives but will be doing so shortly.

      Adrian
    • There is plenty of guides and howto's by searching in google. You're looking at marking the mirror as degraded (an option that is not available in the webUI) so you'll have to use the CLI a little bit

      As I understand the general plan would be:

      1) Mark your mirror raid1 as degraded. This is removing one the HDD from the mirror. It will still be working and data available.
      2) Build a new RAID5 array with 3 disks (2 new ones and the other one you'd remove from the mirror before)
      3) Format raid5, mount and move all data from the RAID1 degraded array
      3) Once the data is moved. Destroy RAID1 array, and add that drive to the raid5 array (grow), grow filesystem and your done.

      All of this steps (except from number 1) can be done from the webUI. Also I would recommend you to backup important data if you don't feel to sure and also practice before in a VM environment. A pre-installed one is available here OpenMediaVault 1.0 VirtualBox Appliances online
      New wiki
      chat support at #openmediavault@freenode IRC | Spanish & English | GMT+10
      telegram.me/openmediavault broadcast channel
      openmediavault discord server
    • Thanks for the advice - I'm probably searching with the wrong terms in google. I had to look up CLI to understand what you're suggesting I need to do lol

      My friend who did the install was messing about with software called Putty - is that the sort of thing you mean for CLI?

      If there's an obvious and simple guide to performing at least the 1st step, or the whole lot a link would be gratefully bookmarked :)

      Adrian
    • CLI is command line interface, you can access OMV CLI with putty by enabling ssh access in the webUI. From there you will use the mdadm tool. Which is used for managing raid arrays. Take a look here and here for related documentation.

      You can come to IRC freenode if you want a more detailed explanation.

      As I'd recommend you before. Practice before in a virtual machine (VM)
      New wiki
      chat support at #openmediavault@freenode IRC | Spanish & English | GMT+10
      telegram.me/openmediavault broadcast channel
      openmediavault discord server
    • Thanks - looking at this again I'm wondering if it would be easier to pull all the data off the drives and start from scratch? I have backed up the content I need and I'm not bothered about losing the backups for my desktop and laptop as I can set the backups up again easily enough.

      I've upgraded OMV to 1.0.31 using Putty CLI so had a small amount of practice with that now!

      Quite keen not to have to setup all the folder shares side of things again, but OMV is on a USB stick (I know! Going to move it to a 5th HDD once I've sorted this out) so none of the storage drives are used for OMV installation so could be wiped.

      Is that a simpler and potentially less time consuming option? If so what order steps do I need to take?
    • I mean the reshaping and moving data does take a lot of time. I guess it would be faster to wipe the disks, build the new raid5 clean and push data into the new array.

      Is your option, it can help you practice with mdadm the option i suggested. But please backup important data before you commit real changes
      New wiki
      chat support at #openmediavault@freenode IRC | Spanish & English | GMT+10
      telegram.me/openmediavault broadcast channel
      openmediavault discord server
    • If I follow the original suggestion I presume this command will be needed to degrade one HDD? I presume removing it is also necessary otherwise it'll still 'see' it?

      mdadm /dev/md0 --fail /dev/sda1 --remove /dev/sda1

      Do I need to delete the data off either of the existing drives at any point, or will adding them to the new RAID5 do that anyway?

      What's required to move data from the RAID1 disc to the new RAID5? I presume this is independant of the folder shares etc but once created the new RAID5 is empty? Is that a simple CLI instruction to copy or am I missing something obvious in the GUI?

      Have made a note of the UUID string. Presume tune2fs runs in CLI? Not looked for that yet!

      Thanks for you patience :)

      Adrian
    • Yes i think thats correct. The old drive will need to be wiped, thats from the webUI to be added to the new raid5.
      From there format the new raid5 to ext4, then mount the raid5. from there is time to move data from raid1 to raid5 array. For that i would use rsync.
      Once finished you can dismiss the raid1 array, delete the fs, destroy the array, wipe the disk and add to the new raid5

      Take a look at my signature for a virtual machine download of OMV
      New wiki
      chat support at #openmediavault@freenode IRC | Spanish & English | GMT+10
      telegram.me/openmediavault broadcast channel
      openmediavault discord server
    • Hiya

      I chickened out of trying to move my files around to accommodate the change in RAID so decided to start again and rebuild :)

      Has gone OK so far, although I should've deleted the RAID before wiping the physical drives, judging by the carnage that caused! Couldn't get OMV to boot until I ejected the drives first, then inserted one at a time so OMV could recognise them. Managed to find a command in CLI to clear the superblock info from the 2 original drives, which then allowed me to add them to my RAID5.

      I've now finished creating the RAID5 (took about 6 hours) and my 4x3TB drives are showing a total capacity of 8.19TB, but now having issues with the file systems. I've created a new ext4 file system which is attributed to /dev/md0 - it says it's online but it's not mounted. Does the file system creation take hours to finish and I'm just trying to move things along too quickly? Any way to monitor if it's doing anything?

      My original shared folders are present and I've updated the volume to point to md0 but they're not accessible yet, presumably because the file system isn't mounted?

      If I try to mount I get an error:

      Failed to mount 'f608fd07-91e0-4ef2-931d-525a287d093c': mount: no such partition found

      Error #6000:
      exception 'OMVException' with message 'Failed to mount 'f608fd07-91e0-4ef2-931d-525a287d093c': mount: no such partition found' in /usr/share/openmediavault/engined/rpc/filesystemmgmt.inc:887
      Stack trace:
      #0 [internal function]: OMVRpcServiceFileSystemMgmt->mount(Array, Array)
      #1 /usr/share/php/openmediavault/rpcservice.inc(125): call_user_func_array(Array, Array)
      #2 /usr/share/php/openmediavault/rpc.inc(79): OMVRpcServiceAbstract->callMethod('mount', Array, Array)
      #3 /usr/sbin/omv-engined(501): OMVRpc::exec('FileSystemMgmt', 'mount', Array, Array, 1)
      #4 {main}

      I also still have my original file system (I presume) sitting with lots of n/a markings and status of 'missing' - is that an issue or can it be ignored? I'm assuming in CLI I could potentially clear this? There's no device marked and deleting it is greyed out.

      ETA - another error I get if I try to apply config changes is:

      File contains invalid JSON (filename=/var/lib/openmediavault/fstab_tasks.json):

      Error #6000:
      exception 'OMVException' with message 'File contains invalid JSON (filename=/var/lib/openmediavault/fstab_tasks.json): ' in /usr/share/php/openmediavault/util.inc:274
      Stack trace:
      #0 /usr/share/php/openmediavault/module.inc(141): OMVJsonFile->read()
      #1 /usr/share/openmediavault/engined/module/fstab.inc(52): OMVModuleAbstract->execTasks('delete')
      #2 /usr/share/openmediavault/engined/rpc/config.inc(164): OMVModuleFsTab->stopService()
      #3 [internal function]: OMVRpcServiceConfig->applyChanges(Array, Array)
      #4 /usr/share/php/openmediavault/rpcservice.inc(125): call_user_func_array(Array, Array)
      #5 /usr/share/php/openmediavault/rpcservice.inc(158): OMVRpcServiceAbstract->callMethod('applyChanges', Array, Array)
      #6 /usr/share/openmediavault/engined/rpc/config.inc(224): OMVRpcServiceAbstract->callMethodBg('applyChanges', Array, Array)
      #7 [internal function]: OMVRpcServiceConfig->applyChangesBg(Array, Array)
      #8 /usr/share/php/openmediavault/rpcservice.inc(125): call_user_func_array(Array, Array)
      #9 /usr/share/php/openmediavault/rpc.inc(79): OMVRpcServiceAbstract->callMethod('applyChangesBg', Array, Array)
      #10 /usr/sbin/omv-engined(501): OMVRpc::exec('Config', 'applyChangesBg', Array, Array, 1)
      #11 {main}

      The post was edited 1 time, last by Adrian E ().

    • Is it best if I post the details in the relevent section on here to get some help? I've managed to get into config.sys using nano after doing some searching but I'm confused by the UUIDs I've seen. I wrote down the UUID before I changed anything but it's different from the GUI and Putty.

      I have the UUID for the new RAID5 from within OMV.

      I'm assuming I need to remove the UUID for my old RAID1 file system as well as sorting out the folder shares?

      The folder shares allowed me to assign them to the new volume although if I revert the config it goes back to a long string that look a bit like a UUID but isn't the same as the one I wrote down from the RAID1 config or Putty!
    • In the config.xml there is a section <fstab></fstab> in between you'll find the file system entries like <mntent></mntent>. They look like this

      Source Code

      1. ​ <mntent>
      2. <uuid>d62735ae-5f85-4a35-88bd-bbf2f6ff5a81</uuid>
      3. <fsname>f2ccc44c-1ef4-4e50-b36b-cc71d86ebf31</fsname>
      4. <dir>/media/f2ccc44c-1ef4-4e50-b36b-cc71d86ebf31</dir>
      5. <type>ext4</type>
      6. <opts>defaults,nofail,acl,user_xattr,noexec,usrjquota=aquota.user,grpjquota=aquota.group,jqfmt=vfsv0</opts>
      7. <freq>0</freq>
      8. <passno>2</passno>
      9. <hidden>0</hidden>
      10. </mntent>


      From there you can find the old ones and delete entries, After fstab you'll find the sharedfolder entries referenced to the old media mounts also.
      New wiki
      chat support at #openmediavault@freenode IRC | Spanish & English | GMT+10
      telegram.me/openmediavault broadcast channel
      openmediavault discord server
    • This is what I find - any chance you could highlight the bits I should be deleting? I'm wanting to retain the shared folder structures I had but I can always replace anyway!

      Source Code

      1. <fstab>
      2. <!--
      3. <mntent>
      4. <uuid>xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx</uuid>
      5. <fsname>xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx|xxxx-xxxx|/dev/xxx</fsname>
      6. <dir>/xxx/yyy/zzz</dir>
      7. <type>ext2|ext3|ext4|xfs|jfs|iso9660|udf</type>
      8. <opts></opts>
      9. <freq>0</freq>
      10. <passno>0|1|2</passno>
      11. </mntent>
      12. -->
      13. <mntent>
      14. <uuid>c4987f3d-a8fd-4504-bb8d-6374e1eb59e1</uuid>
      15. <fsname>c5d9f005-5195-4a03-917a-97b271d7fd0e</fsname>
      16. <dir>/media/c5d9f005-5195-4a03-917a-97b271d7fd0e</dir>
      17. <type>ext4</type>
      18. <opts>defaults,nofail,acl,user_xattr,noexec,usrjquota=aquota.user,grpjquota=aquota.group,jqfmt=vfsv0</opts>
      19. <freq>0</freq>
      20. <passno>2</passno>
      21. <hidden>0</hidden>
      22. </mntent>
      23. </fstab>
      24. <shares>
      25. <!--
      26. <sharedfolder>
      27. <uuid>xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx</uuid>
      28. <name>xxx</name>
      29. <comment>xxx</comment>
      30. <mntentref>xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx</mntentref>
      31. <reldirpath>xxx</reldirpath>
      32. <privileges>
      33. <privilege>
      34. <type>user|group</type>
      35. <name>xxx</name>
      36. <perms>0 = no permission|5 = read and execute|7 = read, write and execute</perms>
      37. </privilege>
      38. </privileges>
      39. </sharedfolder>
      40. -->
      41. <sharedfolder>
      42. <uuid>f60884ce-0a75-40f5-9ef3-20034dc032aa</uuid>
      43. <name>videos</name>
      44. <comment></comment>
      45. <mntentref>c4987f3d-a8fd-4504-bb8d-6374e1eb59e1</mntentref>
      46. <reldirpath>videos</reldirpath>
      47. <privileges>
      48. <privilege>
      49. <type>user</type>
      50. <name>adrian</name>
      51. <perms>7</perms>
      52. </privilege>
      53. </privileges>
      54. </sharedfolder>
      55. <sharedfolder>
      56. <uuid>778c6d8a-51f2-438b-acb7-bc69b3777338</uuid>
      57. <name>music</name>
      58. <comment></comment>
      59. <mntentref>c4987f3d-a8fd-4504-bb8d-6374e1eb59e1</mntentref>
      60. <reldirpath>music</reldirpath>
      61. <privileges></privileges>
      62. </sharedfolder>
      63. <sharedfolder>
      64. <uuid>8fa60a62-1c8d-4039-90e8-e77aa350c823</uuid>
      65. <name>public</name>
      66. <comment></comment>
      67. <mntentref>c4987f3d-a8fd-4504-bb8d-6374e1eb59e1</mntentref>
      68. <reldirpath>public/</reldirpath>
      69. <privileges></privileges>
      70. </sharedfolder>
      71. <sharedfolder>
      72. <uuid>c74afc05-5bef-4968-be40-484b53dc9cbb</uuid>
      73. <name>pictures</name>
      74. <comment></comment>
      75. <mntentref>c4987f3d-a8fd-4504-bb8d-6374e1eb59e1</mntentref>
      76. <reldirpath>pictures/</reldirpath>
      77. <privileges></privileges>
      78. </sharedfolder>
      79. <sharedfolder>
      80. <uuid>a6cfad20-7ee1-4b9a-ab1e-11bcb4103a96</uuid>
      81. <name>Desktop_backups</name>
      82. <comment></comment>
      83. <mntentref>c4987f3d-a8fd-4504-bb8d-6374e1eb59e1</mntentref>
      84. <reldirpath>Desktop_backups</reldirpath>
      85. <privileges></privileges>
      86. </sharedfolder>
      87. <sharedfolder>
      88. <uuid>4efe9cbc-3d3b-4688-a70c-7b32d57faf1a</uuid>
      89. <name>Laptop_backups</name>
      90. <comment></comment>
      91. <mntentref>c4987f3d-a8fd-4504-bb8d-6374e1eb59e1</mntentref>
      92. <reldirpath>Laptop_backups</reldirpath>
      93. <privileges></privileges>
      94. </sharedfolder>
      95. <sharedfolder>
      96. <uuid>7ae22c1c-934b-4721-b5b5-ab07f24850fb</uuid>
      97. <name>torrents</name>
      98. <comment></comment>
      99. <mntentref>c4987f3d-a8fd-4504-bb8d-6374e1eb59e1</mntentref>
      100. <reldirpath>torrents</reldirpath>
      101. <privileges>
      102. <privilege>
      103. <type>user</type>
      104. <name>adrian</name>
      105. <perms>7</perms>
      106. </privilege>
      107. </privileges>
      108. </sharedfolder>
      109. </shares>
      110. <apt>
      Display All
    • Hiya

      Thanks - think this is what you need?

      Source Code

      1. /dev/sda1: UUID="669bba33-60fb-4a80-af21-48ff4ebf5fbd" TYPE="ext4"
      2. /dev/sda5: UUID="f6d54fb9-fa39-4db9-a2b8-bc3af5a80757" TYPE="swap"
      3. /dev/sdb: UUID="ad8a6655-7f10-69ff-8c4d-39b762405a83" UUID_SUB="5814d8c2-f768-c462-9349-8377bc5c7d70" LABEL="openmediavault:0" TYPE="linux_raid_member"
      4. /dev/sdc: UUID="ad8a6655-7f10-69ff-8c4d-39b762405a83" UUID_SUB="8d409e92-ae12-3de6-39d4-166c8e1e695b" LABEL="openmediavault:0" TYPE="linux_raid_member"
      5. /dev/sdd: UUID="ad8a6655-7f10-69ff-8c4d-39b762405a83" UUID_SUB="983153b1-e200-f787-78f3-9e77b9d9ec2a" LABEL="openmediavault:0" TYPE="linux_raid_member"
      6. /dev/sde: UUID="ad8a6655-7f10-69ff-8c4d-39b762405a83" UUID_SUB="10a0b44c-f96a-eeb7-0f43-e3ae50817fb2" LABEL="openmediavault:0" TYPE="linux_raid_member"
      7. /dev/md0: UUID="f608fd07-91e0-4ef2-931d-525a287d093c" TYPE="ext4"
    • Ok you can assign the previous UUID with:

      tune2fs -U c4987f3d-a8fd-4504-bb8d-6374e1eb59e1 /dev/md0

      run omv-mkconf fstab and reboot

      I am assuming that this uuid is the old one that still exist in the config file by mntent and shares
      New wiki
      chat support at #openmediavault@freenode IRC | Spanish & English | GMT+10
      telegram.me/openmediavault broadcast channel
      openmediavault discord server
    • Hiya

      OK - done all that. Just rebooted and still can't mount the new file system

      blkid gives this now (not sure if you'd expect that to change?):

      Source Code

      1. /dev/sdb: UUID="ad8a6655-7f10-69ff-8c4d-39b762405a83" UUID_SUB="8d409e92-ae12-3de6-39d4-166c8e1e695b" LABEL="openmediavault:0" TYPE="linux_raid_member"
      2. /dev/sdc: UUID="ad8a6655-7f10-69ff-8c4d-39b762405a83" UUID_SUB="983153b1-e200-f787-78f3-9e77b9d9ec2a" LABEL="openmediavault:0" TYPE="linux_raid_member"
      3. /dev/sdd: UUID="ad8a6655-7f10-69ff-8c4d-39b762405a83" UUID_SUB="10a0b44c-f96a-eeb7-0f43-e3ae50817fb2" LABEL="openmediavault:0" TYPE="linux_raid_member"
      4. /dev/md0: UUID="c4987f3d-a8fd-4504-bb8d-6374e1eb59e1" TYPE="ext4"
      5. /dev/sda: UUID="ad8a6655-7f10-69ff-8c4d-39b762405a83" UUID_SUB="5814d8c2-f768-c462-9349-8377bc5c7d70" LABEL="openmediavault:0" TYPE="linux_raid_member"
      6. /dev/sde1: UUID="669bba33-60fb-4a80-af21-48ff4ebf5fbd" TYPE="ext4"
      7. /dev/sde5: UUID="f6d54fb9-fa39-4db9-a2b8-bc3af5a80757" TYPE="swap"


      and error message if I try to mount the dev/md0 file system is:

      Source Code

      1. Error #6000:
      2. exception 'OMVException' with message 'File contains invalid JSON (filename=/var/lib/openmediavault/fstab_tasks.json): ' in /usr/share/php/openmediavault/util.inc:274
      3. Stack trace:
      4. #0 /usr/share/php/openmediavault/module.inc(141): OMVJsonFile->read()
      5. #1 /usr/share/openmediavault/engined/module/fstab.inc(52): OMVModuleAbstract->execTasks('delete')
      6. #2 /usr/share/openmediavault/engined/rpc/config.inc(164): OMVModuleFsTab->stopService()
      7. #3 [internal function]: OMVRpcServiceConfig->applyChanges(Array, Array)
      8. #4 /usr/share/php/openmediavault/rpcservice.inc(125): call_user_func_array(Array, Array)
      9. #5 /usr/share/php/openmediavault/rpc.inc(79): OMVRpcServiceAbstract->callMethod('applyChanges', Array, Array)
      10. #6 /usr/share/openmediavault/engined/rpc/filesystemmgmt.inc(872): OMVRpc::exec('Config', 'applyChanges', Array, Array)
      11. #7 [internal function]: OMVRpcServiceFileSystemMgmt->mount(Array, Array)
      12. #8 /usr/share/php/openmediavault/rpcservice.inc(125): call_user_func_array(Array, Array)
      13. #9 /usr/share/php/openmediavault/rpc.inc(79): OMVRpcServiceAbstract->callMethod('mount', Array, Array)
      14. #10 /usr/sbin/omv-engined(501): OMVRpc::exec('FileSystemMgmt', 'mount', Array, Array, 1)
      15. #11 {main}
      Display All


      Have taken a few screen shots in case it helps:

      [IMG:http://i5.photobucket.com/albums/y180/BX16VALVE/OMV/th_raid_zps44683ac4.jpg]
      [IMG:http://i5.photobucket.com/albums/y180/BX16VALVE/OMV/th_filesystem_zps8e0725d8.jpg]
      [IMG:http://i5.photobucket.com/albums/y180/BX16VALVE/OMV/th_physicaldiscs_zps43e8bf2f.jpg]
      [IMG:http://i5.photobucket.com/albums/y180/BX16VALVE/OMV/th_sharedfolders_zpsbc3a7a91.jpg]
      [IMG:http://i5.photobucket.com/albums/y180/BX16VALVE/OMV/th_array_zps49b26daf.jpg]

      If I try to mount the file system and generate the error above, I can go into the shared folders and select dev/md0 but if I revert it shows the volume as c4987f3d-a8fd-4504-bb8d-6374e1eb59e1