Proxmox and OMV, Pool Lots of Resources?

seed · 17. Oktober 2019

Im doing a reorg of all my servers, cleaning some up, upgrading others and all that stuff and I'm looking for the best way to pool all of these resources for a refreshed lab. This is what I have.

asrock rack AMD 12C/24T - 32GB RAM - 120GB SSD
asrock rack XEON v3 12C/24T - 64GB - 120GB SSD
hp microserver gen 8 2C - 10GB 120GB SSD
hp microserver gen 8 4C/8T - 16GB - 120GB SSD
asrock rack avoton 8C- 32GB - 120GB SSD (Will probably retire)
asrock i5 3600 6C - 16GB - 120GB SSD (Current OMV 5 with external storage attached), 512GB NVMe drive(cache drive)
Large disk array over an LSI SAS9200-16E - Container storage and data storage
2nd large array with no disks. I may split the disks into both boxes and raid 10 them so I have better failover there. Currently they're all RAID 6 in one array box.

I am looking for interesting ways to be able to leverage all of these as a pool. I was originally thinking i'd use portainer and just have omv on a host natively that runs my core VMS and a portainer vm that maybe can manage the other nodes container services, but now I'm thinking maybe i'll use proxmox natively, and run omv inside as a vm. I want to be sure I dont suffer issues with my Plex container as well and perhaps can still use it for passthrough to the disk so I dont lose out on much if any overhead?

Is it wise to run proxmox across the board then setup OMV as a VM then setup my containers in the OMV VM? Will I have better resiliency this way? One area i'm concerned about is if my OMV node dies today i'll be left scrounging.

Any ideas welcome as I refresh everything.

ryecoaaron · 17. Oktober 2019

Zitat von seed

Is it wise to run proxmox across the board then setup OMV as a VM then setup my containers

I run a proxmox cluster at home. OMV is a VM. But my containers are on a separate VM that is only containers.

Zitat von seed

Will I have better resiliency this way?

Depends. You have to have shared storage between the three nodes to really make it resilient. You have enough nodes to do that though.

seed · 17. Oktober 2019

Zitat von ryecoaaron

But my containers are on a separate VM that is only containers.

Hi,

Can you elaborate on what you mean here? Running containers in a VM somewhat blows my mind sometimes.

Also, what's the preferred way to deal with shared storage? Should I just have the array as a data volume that I mount over NFS to all the nodes or do I break apart the array and mount them locally to each node?

My main need is really the media volume that Plex uses. How can I share this in a way that a second Plex container or failover container can use the volume if it's not attached to a different node that where the container resides.

Seed

ryecoaaron · 17. Oktober 2019

Zitat von seed

Can you elaborate on what you mean here? Running containers in a VM somewhat blows my mind sometimes.

OMV is installed on one VM and I have an Ubuntu 18 VM that is my docker host running all of the containers I use. Most apps don't know a VM isn't a real machine. So, there is no problem doing this. In fact, this is how VMware does containers in their implementations present and future. The VM is just as lightweight as possible to do nothing but run containers.

Zitat von seed

what's the preferred way to deal with shared storage?

That is a personal preference. I have an NFS server that servers my three proxmox nodes.

Zitat von seed

Should I just have the array as a data volume that I mount over NFS to all the nodes or do I break apart the array and mount them locally to each node?

I like having the storage on my shared nfs server. Kind of like a SAN in an enterprise setup. Works very well. I do have 10GBe connections. My nodes do have a little bit of local storage in case I run some unimportant VMs.

Zitat von seed

My main need is really the media volume that Plex uses. How can I share this in a way that a second Plex container or failover container can use the volume if it's not attached to a different node that where the container resides.

If you have the plex media on a shared nfs server and a container VM on each node, you could have nfs mounted on each VM. Then it would just be a matter of starting your docker image to get your plex up.

seed · 17. Oktober 2019

Ok thank you for the input!

I think what I'll do is put proxmox on all the nodes and:

1. Install an OMV VM
2. Install portainer
3. Create VMs on the other hosts with Ubuntu 18 LTS that takes all the resources
4. Run Swarm on all the VMs with one managed server in one of them
5. Setup portainer and remote agent on those VMs
6. Have main array local to the omv VM host
7. Profit?

Hmmm. Do I really need OMV for this? What does OMV provide if I just go with portainer and manage nodes with proxmox and portainer for container management?

ryecoaaron · 18. Oktober 2019

Zitat von seed

Do I really need OMV for this?

I don't know. Do you? I don't use OMV or portainer for my docker stuff. Maybe it would be easier to setup nfs and/or samba if you need it?

seed · 18. Oktober 2019

Zitat von ryecoaaron

I don't know. Do you? I don't use OMV or portainer for my docker stuff.

Ha,

I don't really, but I really want to keep using OMV because I love it so much, but part of me is thinking that if I use proxmox and portainer with swarm i wouldn't need OMV anymore?

ryecoaaron · 18. Oktober 2019

Zitat von seed

ut part of me is thinking that if I use proxmox and portainer with swarm i wouldn't need OMV anymore?

If you know how to setup all of that stuff, I wouldn't force yourself to use OMV. There are some things that it is easier to use OMV to do. Having it on the system doesn't really hurt anything.

donh · 18. Oktober 2019

I use omv as a vm for sharing to machines on the network. It allows easy admin of shares and Active directory for access.

Since you have 3 servers you could setup a cluster. And use the cluster file systems. Talk about redundancy. But that is over my head.

Good luck.

Jetzt mitmachen!