Pool Hard Drives

    • jcschwartz wrote:

      I will give this all a try. Like I said I haven't tried anything else because using windows was what I knew how to do, and yes the command line stuff is scary. I would have no idea how to do that. Is there some sort of a guide I can review?

      Personally, since I had no experience with Linux, I just set up a Linux Mint virtual machine on my Windows PC that I played around with for a few weeks. It helped me learn how to do a lot of basic things from the command line like moving and deleting files. Then I set up an OMV VM and played around with that for a month or so. You can practice as much or little as you want and never have to worry about messing anything up, since if you do, you can delete the VM and start over again.
    • ryecoaaron wrote:

      jollyrogr wrote:

      You can pool at the folder level but requires manual configuration.
      Most people don't really need this and I recommend against manual configuration.
      I need it. I did it. It works great.

      jcschwartz,
      I'd recommend some other way of keeping track of what you've watched rather than physically moving files around. That seems very cumbersome to me. I use Kodi to watch my movies and it has a little icon next to the file name to show if a file's been watched or not.
    • jollyrogr wrote:

      I need it. I did it. It works great.
      That is why I said "most". While you have it working, I don't think it is a good idea to recommend it for newbs.

      jollyrogr wrote:

      I use Kodi to watch my movies and it has a little icon next to the file name to show if a file's been watched or not.
      That is what I do as well.

      jcschwartz wrote:

      How do you go about setting up OMV VM?
      That needs a more information like what virtualization platform.
      omv 4.1.17 arrakis | 64 bit | 4.15 proxmox kernel | omvextrasorg 4.1.13
      omv-extras.org plugins source code and issue tracker - github

      Please read this before posting a question and this and this for docker questions.
      Please don't PM for support... Too many PMs!
    • jollyrogr wrote:

      I need it. I did it. It works great.
      jcschwartz,
      I'd recommend some other way of keeping track of what you've watched rather than physically moving files around. That seems very cumbersome to me. I use Kodi to watch my movies and it has a little icon next to the file name to show if a file's been watched or not.

      Kodi and Plex also have Trakt plugins that sync your watched status to a trakt.tv account, which is nice because it will sync again if you ever have to reinstall or reset your kodi database.
    • jcschwartz wrote:

      How do you go about setting up OMV VM?
      If you're looking to learn and have Windows experience, give Virtual Box a try. If you have a Windows machine with 6MB (8MB or more preferred) load Virtual Box on it.

      To get you started:
      After the Virtual Box install, select "New" (OVM is Debian Linux), assign 2GB, create an 16GB boot drive and a small data drive or maybe 3 or 4 small data drives in 5 to 10GB sizes. (BTW dynamic disks work fine and take up less space). In network, check "Enable Network Adapter" and in "Attached to:" select Bridged Adapter. With an OMV ISO/CD in the host DVD drive, install OMV in a VM.

      If you've never used Virtual Box or some other virtualization app., it may take some time to familiarize but it's well worth it. There are examples on-line of how to build a guest. (This one is old but it will point you in the right direction. -> Using VB.)

      VM's are great learning tools and it only takes about 10-15 minutes to do an OMV build from scratch. Configuration takes longer, depending on what you want to try.

      Testing with VM's can be done without consequence. When the VM build is complete, snapshot or clone the VM, test it, break it, revive it.

      Video Guides :!: New User Guide :!: Docker Guides :!: Pi-hole in Docker
      Good backup takes the "drama" out of computing.
      ____________________________________
      Primary: OMV 3.0.99, ThinkServer TS140, 12GB ECC, 32GB USB boot, 4TB+4TB zmirror, 3TB client backup.
      OMV 4.1.13, Intel Server SC5650HCBRP, 32GB ECC, 16GB USB boot, UnionFS+SNAPRAID
      Backup: OMV 4.1.9, Acer RC-111, 4GB, 32GB USB boot, 3TB+3TB zmirror, 4TB Rsync'ed disk
    • eXtplorer is a file manager. Virtual Box is a virtualization platform.

      For "visualizing" Linux file systems and some file management tasks (permissions, direct editing of config files), I'd recommend WinSCP.

      Video Guides :!: New User Guide :!: Docker Guides :!: Pi-hole in Docker
      Good backup takes the "drama" out of computing.
      ____________________________________
      Primary: OMV 3.0.99, ThinkServer TS140, 12GB ECC, 32GB USB boot, 4TB+4TB zmirror, 3TB client backup.
      OMV 4.1.13, Intel Server SC5650HCBRP, 32GB ECC, 16GB USB boot, UnionFS+SNAPRAID
      Backup: OMV 4.1.9, Acer RC-111, 4GB, 32GB USB boot, 3TB+3TB zmirror, 4TB Rsync'ed disk
    • The OP was asking about methods to move files across drives, which was slow because the user was doing it through a network share.

      Given the description of the setup and problem, the user may not be aware of all options, and eXtplorer may resolve the simple issue of quicker file transfer without having to do much else to the server.

      Not saying this is unquestionably the best solution, just that it is a way to easily resolve file transfer speeds.
      OMV 4.1.14-1 (Arrakis); Shuttle XPC SH67H3; Intel Core i5-2390T; 8 GB DDR3-1333 RAM; 128GB SanDisk Z400s SSD (OS); Samsung 860 EVO 1TB (primary storage).
    • eXtplorer is not available as plug-in for OMV3 and OMV4. So installation is more complex.

      IMO easiest way to copy files on the server is midnight commander.

      Install with
      sudo apt install mc

      start with
      sudo mc

      To get started with CLI, I recommend "The Linux Command Line", which can be downloaded for free here:
      linuxcommand.org/tlcl.php/
      Odroid HC2 - armbian - Seagate ST4000DM004 - OMV4.x
      Asrock Q1900DC-ITX - 16GB - 2x Seagate ST3000VN000 - Intenso SSD 120GB - OMV4.x
      :!: Backup - Solutions to common problems - OMV setup videos - OMV4 Documentation - user guide :!:
    • openletter wrote:

      /---/
      Given the description of the setup and problem, the user may not be aware of all options, and eXtplorer may resolve the simple issue of quicker file transfer without having to do much else to the server.

      Not saying this is unquestionably the best solution, just that it is a way to easily resolve file transfer speeds.
      Point taken. (I was focusing on the VM question.) For that purpose, WinSCP would not be the best solution.

      Video Guides :!: New User Guide :!: Docker Guides :!: Pi-hole in Docker
      Good backup takes the "drama" out of computing.
      ____________________________________
      Primary: OMV 3.0.99, ThinkServer TS140, 12GB ECC, 32GB USB boot, 4TB+4TB zmirror, 3TB client backup.
      OMV 4.1.13, Intel Server SC5650HCBRP, 32GB ECC, 16GB USB boot, UnionFS+SNAPRAID
      Backup: OMV 4.1.9, Acer RC-111, 4GB, 32GB USB boot, 3TB+3TB zmirror, 4TB Rsync'ed disk
    • ryecoaaron wrote:

      mergerfs works on top of existing filesystems. Creating a pool will not do anything to the files on the current filesystems. You can add/remove drives whenever you want. If a drive fails, you will just lose access to the files on that drive.

      OP, this is the problem with mergerfs. Here is what happens, if you have 3 drives and you create a pool with them and then you decide to remove one of the drives your data on that drive will be removed from the pool. I would suggest installing Rockstor instead of OMV due to a fact that when you create a pool in Rockstor and you want to remove the drive but keep all the data Rockstor will move the data to the drives you are keeping. OMV does not seem to have anything like this unless you go with RAID.
    • ZanosG wrote:

      ryecoaaron wrote:

      mergerfs works on top of existing filesystems. Creating a pool will not do anything to the files on the current filesystems. You can add/remove drives whenever you want. If a drive fails, you will just lose access to the files on that drive.
      OP, this is the problem with mergerfs. Here is what happens, if you have 3 drives and you create a pool with them and then you decide to remove one of the drives your data on that drive will be removed from the pool. I would suggest installing Rockstor instead of OMV due to a fact that when you create a pool in Rockstor and you want to remove the drive but keep all the data Rockstor will move the data to the drives you are keeping. OMV does not seem to have anything like this unless you go with RAID.
      Ok troll. I don’t know why you are recommending something else in the OMV forum due to one filesystem choice. There are plenty of filesystem choices in OMV that dont behave this way. Pretty sure Rockstor doesn’t do anything magical with mergerfs to do something else (if it even supports mergerfs). And last time I checked, there isn’t a filesystem or raid type supported by Rockstor that isn’t supported by OMV.
      omv 4.1.17 arrakis | 64 bit | 4.15 proxmox kernel | omvextrasorg 4.1.13
      omv-extras.org plugins source code and issue tracker - github

      Please read this before posting a question and this and this for docker questions.
      Please don't PM for support... Too many PMs!
    • ZanosG wrote:

      OP, this is the problem with mergerfs. Here is what happens, if you have 3 drives and you create a pool with them and then you decide to remove one of the drives your data on that drive will be removed from the pool. I would suggest installing Rockstor instead of OMV due to a fact that when you create a pool in Rockstor and you want to remove the drive but keep all the data Rockstor will move the data to the drives you are keeping. OMV does not seem to have anything like this unless you go with RAID.
      Are you comparing mergerfs to BTRFS RAID?

      Do you know what the status of BTRFS RAID is? In the versions of BTRFS RAID where it's possible to remove a drive AND consolidate data reducing the array in size (RAID 5 and 6), the current status is "unstable". On the other hand, what you're describing is not possible in BTRFS RAID5. Removing 1 drive of 3, would result in fewer drives than is possible in a RAID 5 array. That would require a minimum of 4 block devices.
      ((The command in question is btrfs device delete . In theory, the command triggers a data rebalancing before the drive is removed from the array. What I find interesting is that the command doesn't work for bad drives because the ability to "read" the drive is a requirement.))

      With that noted, did you know that you can set up BTRFS RAID in OMV, in any implementation available in ROCKSTOR? What you may not be aware of are the files systems and RAID flavors that OMV has, that are NOT available in ROCKSTOR. Further, OMV has more value added plugins than ROCKSTOR and and an easy to use, thoroughly tested, Docker GUI.

      Of the serious "out of the box" candidates for a Home or a Small Business NAS, during the last 4 years or so, I've tested them all. If you look closely at Rockstor's "features", currently, there's more in "Beta" than in "production". These features have been stable and available in OMV for years and I'm using several of them of them right now.

      For these reasons and many more, ROCKSTOR can't even be compared to OMV.

      Video Guides :!: New User Guide :!: Docker Guides :!: Pi-hole in Docker
      Good backup takes the "drama" out of computing.
      ____________________________________
      Primary: OMV 3.0.99, ThinkServer TS140, 12GB ECC, 32GB USB boot, 4TB+4TB zmirror, 3TB client backup.
      OMV 4.1.13, Intel Server SC5650HCBRP, 32GB ECC, 16GB USB boot, UnionFS+SNAPRAID
      Backup: OMV 4.1.9, Acer RC-111, 4GB, 32GB USB boot, 3TB+3TB zmirror, 4TB Rsync'ed disk

      The post was edited 1 time, last by flmaxey: edit ().

    • ryecoaaron wrote:

      ZanosG wrote:

      ryecoaaron wrote:

      mergerfs works on top of existing filesystems. Creating a pool will not do anything to the files on the current filesystems. You can add/remove drives whenever you want. If a drive fails, you will just lose access to the files on that drive.
      OP, this is the problem with mergerfs. Here is what happens, if you have 3 drives and you create a pool with them and then you decide to remove one of the drives your data on that drive will be removed from the pool. I would suggest installing Rockstor instead of OMV due to a fact that when you create a pool in Rockstor and you want to remove the drive but keep all the data Rockstor will move the data to the drives you are keeping. OMV does not seem to have anything like this unless you go with RAID.
      Ok troll. I don’t know why you are recommending something else in the OMV forum due to one filesystem choice. There are plenty of filesystem choices in OMV that dont behave this way. Pretty sure Rockstor doesn’t do anything magical with mergerfs to do something else (if it even supports mergerfs). And last time I checked, there isn’t a filesystem or raid type supported by Rockstor that isn’t supported by OMV.

      OMV has nothing to loose if I recommend something else that will make things easier for those people. OMV is free so they are not loosing any revenue. No, Rockstor uses BTRFS but to me thats all besides the point. When I asked on here about a filesystem that would be able to do what I want I was told that mergerfs can not do things this way. I have 8TB and 4TB drives in my system and I have them set up in Liner Raid right now for total of 12TB of space. I had 8TB and 4TB merged together too with mergerfs before I found out that I just cant add or remove drives without loosing data so for me there was no difference between setting up Linear Raid with these two drives or using mergerfs. Lets say I want to add another 8TB and then remove the old 8TB. Under Rockstor I can create pool and select Linear Raid. Then I can add and remove drives without data being removed as long as there is enough room for the data on the drives that are staying in the system. And I can do all this with few clicks because the system is set up so you dont have to be a rocket scientist to figure things out. Im not knocking OMV. Im just pointing out that some other system might be better for some people.



      flmaxey wrote:

      ZanosG wrote:

      OP, this is the problem with mergerfs. Here is what happens, if you have 3 drives and you create a pool with them and then you decide to remove one of the drives your data on that drive will be removed from the pool. I would suggest installing Rockstor instead of OMV due to a fact that when you create a pool in Rockstor and you want to remove the drive but keep all the data Rockstor will move the data to the drives you are keeping. OMV does not seem to have anything like this unless you go with RAID.
      Are you comparing mergerfs to BTRFS RAID?
      Do you know what the status of BTRFS RAID is? In the versions of BTRFS RAID where it's possible to remove a drive AND consolidate data reducing the array in size (RAID 5 and 6), the current status is "unstable". On the other hand, what you're describing is not possible in BTRFS RAID5. Removing 1 drive of 3, would result in fewer drives than is possible in a RAID 5 array. That would require a minimum of 4 block devices.
      ((The command in question is btrfs device delete . In theory, the command triggers a data rebalancing before the drive is removed from the array. What I find interesting is that the command doesn't work for bad drives because the ability to "read" the drive is a requirement.))

      With that noted, did you know that you can set up BTRFS RAID in OMV, in any implementation available in ROCKSTOR? What you may not be aware of are the files systems and RAID flavors that OMV has, that are NOT available in ROCKSTOR. Further, OMV has more value added plugins than ROCKSTOR and and an easy to use, thoroughly tested, Docker GUI.

      Of the serious "out of the box" candidates for a Home or a Small Business NAS, during the last 4 years or so, I've tested them all. If you look closely at Rockstor's "features", currently, there's more in "Beta" than in "production". These features have been stable and available in OMV for years and I'm using several of them of them right now.

      For these reasons and many more, ROCKSTOR can't even be compared to OMV.

      Im talking Linear RAID. See above.
    • ZanosG wrote:

      Im talking Linear RAID. See above.
      OMV will support BTRFS linear RAID as well and, as noted, a whole lot more.
      _______________________________

      With a few exceptions, I've never understood the need to aggregate muti-terabyte drives. If it's done so that the contents a single folder can span multiple drives, the obvious remedy would be data organization. As a potential solution; the use of symlinks would give the functional appearance of an enormous drive without dealing with the issues associated with LVM, RAID, mergerfs, etc.

      Since the root reason for huge pools tend to be video files, well, that's what Plex and other media managers are all about - aggregating media files into a single interface.

      (Just an opinion.)

      Video Guides :!: New User Guide :!: Docker Guides :!: Pi-hole in Docker
      Good backup takes the "drama" out of computing.
      ____________________________________
      Primary: OMV 3.0.99, ThinkServer TS140, 12GB ECC, 32GB USB boot, 4TB+4TB zmirror, 3TB client backup.
      OMV 4.1.13, Intel Server SC5650HCBRP, 32GB ECC, 16GB USB boot, UnionFS+SNAPRAID
      Backup: OMV 4.1.9, Acer RC-111, 4GB, 32GB USB boot, 3TB+3TB zmirror, 4TB Rsync'ed disk

      The post was edited 1 time, last by flmaxey: edit ().

    • ZanosG wrote:

      And I can do all this with few clicks because the system is set up so you dont have to be a rocket scientist to figure things out. Im not knocking OMV. Im just pointing out that some other system might be better for some people.
      You can do that with OMV too. Just not with btrfs (yet). You can do it with lvm, zfs, or mdadm. OMV may be a bit more difficult some times but that is because it is more flexible.
      omv 4.1.17 arrakis | 64 bit | 4.15 proxmox kernel | omvextrasorg 4.1.13
      omv-extras.org plugins source code and issue tracker - github

      Please read this before posting a question and this and this for docker questions.
      Please don't PM for support... Too many PMs!
    • So I tried LVM2. I was able to add physical disks under Volume Groups but I was not able to remove any. When I clicked on Reduce, all I got was the page with no disks listed. zfs didnt want to install at all on OMV4. There were countless errors. mdadm I didnt try.

      One suggestion, If I cant do it through GUI its simply not going to work for me and for millions of other average users. Hence why I was saying that Rockstor might be more helpful.
    • Hi ZanosG,

      You can have linear RAID with mdadm.
      For example, I have 4x2To RAID 5 for main storage, and make backups of that datas every night with rsnapshot on a 3x2To Linear RAID.

      That works perfect for me, and very easy to configure with OMV GUI.
      Lian Li PC-V354 with Be Quiet fans | Gigabyte GA-G33M-DS2R | Intel E8400@3,6Ghz | 6GB DDR2 RAM
      1x500MB SSD for System/Backup | 7x2To HDD with ZFS RAIDz2 for Datas/Snapshots
      Powered by OMV v4.1.7 / Kernel 4.16.x / ZFS 0.7.9