Pool Hard Drives

    • jcschwartz wrote:

      I will give this all a try. Like I said I haven't tried anything else because using windows was what I knew how to do, and yes the command line stuff is scary. I would have no idea how to do that. Is there some sort of a guide I can review?

      Personally, since I had no experience with Linux, I just set up a Linux Mint virtual machine on my Windows PC that I played around with for a few weeks. It helped me learn how to do a lot of basic things from the command line like moving and deleting files. Then I set up an OMV VM and played around with that for a month or so. You can practice as much or little as you want and never have to worry about messing anything up, since if you do, you can delete the VM and start over again.
    • ryecoaaron wrote:

      jollyrogr wrote:

      You can pool at the folder level but requires manual configuration.
      Most people don't really need this and I recommend against manual configuration.
      I need it. I did it. It works great.

      jcschwartz,
      I'd recommend some other way of keeping track of what you've watched rather than physically moving files around. That seems very cumbersome to me. I use Kodi to watch my movies and it has a little icon next to the file name to show if a file's been watched or not.
    • jollyrogr wrote:

      I need it. I did it. It works great.
      That is why I said "most". While you have it working, I don't think it is a good idea to recommend it for newbs.

      jollyrogr wrote:

      I use Kodi to watch my movies and it has a little icon next to the file name to show if a file's been watched or not.
      That is what I do as well.

      jcschwartz wrote:

      How do you go about setting up OMV VM?
      That needs a more information like what virtualization platform.
      omv 5.0.14 usul | 64 bit | 5.0 proxmox kernel | omvextrasorg 5.1.4
      omv-extras.org plugins source code and issue tracker - github

      Please read this before posting a question and this and this for docker questions.
      Please don't PM for support... Too many PMs!
    • jollyrogr wrote:

      I need it. I did it. It works great.
      jcschwartz,
      I'd recommend some other way of keeping track of what you've watched rather than physically moving files around. That seems very cumbersome to me. I use Kodi to watch my movies and it has a little icon next to the file name to show if a file's been watched or not.

      Kodi and Plex also have Trakt plugins that sync your watched status to a trakt.tv account, which is nice because it will sync again if you ever have to reinstall or reset your kodi database.
    • The OP was asking about methods to move files across drives, which was slow because the user was doing it through a network share.

      Given the description of the setup and problem, the user may not be aware of all options, and eXtplorer may resolve the simple issue of quicker file transfer without having to do much else to the server.

      Not saying this is unquestionably the best solution, just that it is a way to easily resolve file transfer speeds.
      OMV 4.1.23-1 (Arrakis); Shuttle XPC SH67H3; Intel Core i5-2390T; 8 GB DDR3-1333 RAM; 128GB SanDisk Z400s SSD (OS); Samsung 860 EVO 1TB (primary storage).
    • eXtplorer is not available as plug-in for OMV3 and OMV4. So installation is more complex.

      IMO easiest way to copy files on the server is midnight commander.

      Install with
      sudo apt install mc

      start with
      sudo mc

      To get started with CLI, I recommend "The Linux Command Line", which can be downloaded for free here:
      linuxcommand.org/tlcl.php/
      Odroid HC2 - armbian - OMV4.x | Asrock Q1900DC-ITX - Intenso SSD 120GB - OMV4.x
      :!: Backup - Solutions to common problems - OMV setup videos - OMV4 Documentation - user guide :!:
    • ryecoaaron wrote:

      mergerfs works on top of existing filesystems. Creating a pool will not do anything to the files on the current filesystems. You can add/remove drives whenever you want. If a drive fails, you will just lose access to the files on that drive.

      OP, this is the problem with mergerfs. Here is what happens, if you have 3 drives and you create a pool with them and then you decide to remove one of the drives your data on that drive will be removed from the pool. I would suggest installing Rockstor instead of OMV due to a fact that when you create a pool in Rockstor and you want to remove the drive but keep all the data Rockstor will move the data to the drives you are keeping. OMV does not seem to have anything like this unless you go with RAID.
    • ZanosG wrote:

      ryecoaaron wrote:

      mergerfs works on top of existing filesystems. Creating a pool will not do anything to the files on the current filesystems. You can add/remove drives whenever you want. If a drive fails, you will just lose access to the files on that drive.
      OP, this is the problem with mergerfs. Here is what happens, if you have 3 drives and you create a pool with them and then you decide to remove one of the drives your data on that drive will be removed from the pool. I would suggest installing Rockstor instead of OMV due to a fact that when you create a pool in Rockstor and you want to remove the drive but keep all the data Rockstor will move the data to the drives you are keeping. OMV does not seem to have anything like this unless you go with RAID.
      Ok troll. I don’t know why you are recommending something else in the OMV forum due to one filesystem choice. There are plenty of filesystem choices in OMV that dont behave this way. Pretty sure Rockstor doesn’t do anything magical with mergerfs to do something else (if it even supports mergerfs). And last time I checked, there isn’t a filesystem or raid type supported by Rockstor that isn’t supported by OMV.
      omv 5.0.14 usul | 64 bit | 5.0 proxmox kernel | omvextrasorg 5.1.4
      omv-extras.org plugins source code and issue tracker - github

      Please read this before posting a question and this and this for docker questions.
      Please don't PM for support... Too many PMs!
    • ryecoaaron wrote:

      ZanosG wrote:

      ryecoaaron wrote:

      mergerfs works on top of existing filesystems. Creating a pool will not do anything to the files on the current filesystems. You can add/remove drives whenever you want. If a drive fails, you will just lose access to the files on that drive.
      OP, this is the problem with mergerfs. Here is what happens, if you have 3 drives and you create a pool with them and then you decide to remove one of the drives your data on that drive will be removed from the pool. I would suggest installing Rockstor instead of OMV due to a fact that when you create a pool in Rockstor and you want to remove the drive but keep all the data Rockstor will move the data to the drives you are keeping. OMV does not seem to have anything like this unless you go with RAID.
      Ok troll. I don’t know why you are recommending something else in the OMV forum due to one filesystem choice. There are plenty of filesystem choices in OMV that dont behave this way. Pretty sure Rockstor doesn’t do anything magical with mergerfs to do something else (if it even supports mergerfs). And last time I checked, there isn’t a filesystem or raid type supported by Rockstor that isn’t supported by OMV.

      OMV has nothing to loose if I recommend something else that will make things easier for those people. OMV is free so they are not loosing any revenue. No, Rockstor uses BTRFS but to me thats all besides the point. When I asked on here about a filesystem that would be able to do what I want I was told that mergerfs can not do things this way. I have 8TB and 4TB drives in my system and I have them set up in Liner Raid right now for total of 12TB of space. I had 8TB and 4TB merged together too with mergerfs before I found out that I just cant add or remove drives without loosing data so for me there was no difference between setting up Linear Raid with these two drives or using mergerfs. Lets say I want to add another 8TB and then remove the old 8TB. Under Rockstor I can create pool and select Linear Raid. Then I can add and remove drives without data being removed as long as there is enough room for the data on the drives that are staying in the system. And I can do all this with few clicks because the system is set up so you dont have to be a rocket scientist to figure things out. Im not knocking OMV. Im just pointing out that some other system might be better for some people.



      flmaxey wrote:

      ZanosG wrote:

      OP, this is the problem with mergerfs. Here is what happens, if you have 3 drives and you create a pool with them and then you decide to remove one of the drives your data on that drive will be removed from the pool. I would suggest installing Rockstor instead of OMV due to a fact that when you create a pool in Rockstor and you want to remove the drive but keep all the data Rockstor will move the data to the drives you are keeping. OMV does not seem to have anything like this unless you go with RAID.
      Are you comparing mergerfs to BTRFS RAID?
      Do you know what the status of BTRFS RAID is? In the versions of BTRFS RAID where it's possible to remove a drive AND consolidate data reducing the array in size (RAID 5 and 6), the current status is "unstable". On the other hand, what you're describing is not possible in BTRFS RAID5. Removing 1 drive of 3, would result in fewer drives than is possible in a RAID 5 array. That would require a minimum of 4 block devices.
      ((The command in question is btrfs device delete . In theory, the command triggers a data rebalancing before the drive is removed from the array. What I find interesting is that the command doesn't work for bad drives because the ability to "read" the drive is a requirement.))

      With that noted, did you know that you can set up BTRFS RAID in OMV, in any implementation available in ROCKSTOR? What you may not be aware of are the files systems and RAID flavors that OMV has, that are NOT available in ROCKSTOR. Further, OMV has more value added plugins than ROCKSTOR and and an easy to use, thoroughly tested, Docker GUI.

      Of the serious "out of the box" candidates for a Home or a Small Business NAS, during the last 4 years or so, I've tested them all. If you look closely at Rockstor's "features", currently, there's more in "Beta" than in "production". These features have been stable and available in OMV for years and I'm using several of them of them right now.

      For these reasons and many more, ROCKSTOR can't even be compared to OMV.

      Im talking Linear RAID. See above.
    • ZanosG wrote:

      And I can do all this with few clicks because the system is set up so you dont have to be a rocket scientist to figure things out. Im not knocking OMV. Im just pointing out that some other system might be better for some people.
      You can do that with OMV too. Just not with btrfs (yet). You can do it with lvm, zfs, or mdadm. OMV may be a bit more difficult some times but that is because it is more flexible.
      omv 5.0.14 usul | 64 bit | 5.0 proxmox kernel | omvextrasorg 5.1.4
      omv-extras.org plugins source code and issue tracker - github

      Please read this before posting a question and this and this for docker questions.
      Please don't PM for support... Too many PMs!
    • So I tried LVM2. I was able to add physical disks under Volume Groups but I was not able to remove any. When I clicked on Reduce, all I got was the page with no disks listed. zfs didnt want to install at all on OMV4. There were countless errors. mdadm I didnt try.

      One suggestion, If I cant do it through GUI its simply not going to work for me and for millions of other average users. Hence why I was saying that Rockstor might be more helpful.
    • Hi ZanosG,

      You can have linear RAID with mdadm.
      For example, I have 4x2To RAID 5 for main storage, and make backups of that datas every night with rsnapshot on a 3x2To Linear RAID.

      That works perfect for me, and very easy to configure with OMV GUI.
      Lian Li PC-V354 with Be Quiet fans | Gigabyte GA-G33M-DS2R | Intel E8400@3,6Ghz | 6GB DDR2 RAM
      1x500MB SSD for System/Backup/Downloads | 7x2To HDD RAID 6 for Datas/Snapshots
      Powered by OMV v4.1.18 / Kernel 4.19
    • ZanosG wrote:

      So I tried LVM2. I was able to add physical disks under Volume Groups but I was not able to remove any. When I clicked on Reduce, all I got was the page with no disks listed. zfs didnt want to install at all on OMV4. There were countless errors. mdadm I didnt try.
      With LVM, you have to know how to use it. You can't reduce a volume group if there is an LV on that physical volume.
      zfs has installed many, many times on OMV 4 for me. What are these countless errors?

      ZanosG wrote:

      One suggestion, If I cant do it through GUI its simply not going to work for me and for millions of other average users. Hence why I was saying that Rockstor might be more helpful.
      How is that a suggestion? The suggestions I gave do work in the GUI. I never said they are for the millions of average users. The only reason Rockstor is easier in your case is because of btrfs. lvm and mdadm are much older and more complicated.

      You are not an average user if you add and remove disks all the time. Average users add a couple of disks and just want it to work. They *might* add a disk but very, very few people remove disks. Adding works very well with mergerfs.
      omv 5.0.14 usul | 64 bit | 5.0 proxmox kernel | omvextrasorg 5.1.4
      omv-extras.org plugins source code and issue tracker - github

      Please read this before posting a question and this and this for docker questions.
      Please don't PM for support... Too many PMs!
    • sbocquet wrote:

      Hi ZanosG,

      You can have linear RAID with mdadm.
      For example, I have 4x2To RAID 5 for main storage, and make backups of that datas every night with rsnapshot on a 3x2To Linear RAID.

      That works perfect for me, and very easy to configure with OMV GUI.
      Thanks for your input but I cant even find anything on google on how to install this mdadm at all. If I had that many drives and had RAID 5 I would have no problem with growing it. The problem is that with Linear you cant grow it or remove disks. You have to break the RAID and start all over again and thats when all your data is gone.
    • Hi,

      mdadm is part of the basic packages on debian, so OMV has it from the box else...

      Source Code

      1. apt-get install mdadm

      For the growing of a linear RAID, no problem...

      Source Code

      1. fdisk /dev/sdb
      2. n,1,enter,enter,t,1,fd,w
      3. fdisk /dev/sdc
      4. n,1,enter,enter,t,1,fd,w
      5. mdadm -C /dev/md0 -llinear -n2 /dev/sdb1 /dev/sdc1
      6. cat /proc/mdstat:
      7. Personalities : [linear]
      8. md0 : active linear sdc1[1] sdb1[0]
      9. 192512 blocks 64k rounding
      10. unused devices: <none>
      11. Next, I created a third device:
      12. fdisk /dev/sdd
      13. n,1,enter,enter,t,1,fd,w
      Display All
      I then added it to the array:

      Source Code

      1. mdadm --grow /dev/md0 --add /dev/sdd1
      2. cat /proc/mdstat:
      3. Personalities : [linear]
      4. md0 : active linear sdd1[2] sdc1[1] sdb1[0]
      5. 288768 blocks 64k rounding
      6. unused devices: <none>

      Then resize your File System... and mount it with OMV GUI. ;)

      Done ;)
      Lian Li PC-V354 with Be Quiet fans | Gigabyte GA-G33M-DS2R | Intel E8400@3,6Ghz | 6GB DDR2 RAM
      1x500MB SSD for System/Backup/Downloads | 7x2To HDD RAID 6 for Datas/Snapshots
      Powered by OMV v4.1.18 / Kernel 4.19
    • ryecoaaron wrote:

      ZanosG wrote:

      So I tried LVM2. I was able to add physical disks under Volume Groups but I was not able to remove any. When I clicked on Reduce, all I got was the page with no disks listed. zfs didnt want to install at all on OMV4. There were countless errors. mdadm I didnt try.
      With LVM, you have to know how to use it. You can't reduce a volume group if there is an LV on that physical volume.zfs has installed many, many times on OMV 4 for me. What are these countless errors?

      ZanosG wrote:

      One suggestion, If I cant do it through GUI its simply not going to work for me and for millions of other average users. Hence why I was saying that Rockstor might be more helpful.
      How is that a suggestion? The suggestions I gave do work in the GUI. I never said they are for the millions of average users. The only reason Rockstor is easier in your case is because of btrfs. lvm and mdadm are much older and more complicated.
      You are not an average user if you add and remove disks all the time. Average users add a couple of disks and just want it to work. They *might* add a disk but very, very few people remove disks. Adding works very well with mergerfs.
      I dont know what errors were. As soon as I encounter errors I ditch everything and move onto something that is proven to work. I dont have a know how on how to monkey with things in CLI nor do I want to. And Im not sure why we are going back and forth with this. From the beginning I have only claimed that Rockstor might be easier for people like me and you have said it yourself that any options in OMV will be harder to set up for someone like me. And for the last time, yes mergerfs will remove and add disks fine however the data does not get moved over prior to disk being removed. Im not sure how hard is to understand that when in GUI I select to remove disk and once the disk is removed I check the folder in question and the data is gone.
    • sbocquet wrote:

      Hi,

      mdadm is part of the basic packages on debian, so OMV has it from the box else...

      Source Code

      1. apt-get install mdadm
      For the growing of a linear RAID, no problem...

      Source Code

      1. fdisk /dev/sdb
      2. n,1,enter,enter,t,1,fd,w
      3. fdisk /dev/sdc
      4. n,1,enter,enter,t,1,fd,w
      5. mdadm -C /dev/md0 -llinear -n2 /dev/sdb1 /dev/sdc1
      6. cat /proc/mdstat:
      7. Personalities : [linear]
      8. md0 : active linear sdc1[1] sdb1[0]
      9. 192512 blocks 64k rounding
      10. unused devices: <none>
      11. Next, I created a third device:
      12. fdisk /dev/sdd
      13. n,1,enter,enter,t,1,fd,w
      Display All
      I then added it to the array:

      Source Code

      1. mdadm --grow /dev/md0 --add /dev/sdd1cat /proc/mdstat:Personalities : [linear] md0 : active linear sdd1[2] sdc1[1] sdb1[0] 288768 blocks 64k rounding unused devices: <none>
      Done ;)


      But OMV can do the same with a nice GUI... ;)

      Sorry but that's all gibberish to me. I do appreciate your input however I don't know what to do with any of that or if its going to mess my system if i start typing stuff in CLI without knowing what exactly it is doing.