Rabbit hole - Advice needed - Promox or Image Backup solution?

  • So I've just bought new hardware for a server intended to run Openmediavault as a media server running Docker containers. It's almost the same hardware as the build here with an Asrock J4105 as the base but with 8GB ram, a 128GB SSD and two 1TB 3.5" hard drives.


    The main OMV installation was going to be installed on the SSD which I'd then repartition with Gparted GUI to give some of the SSD space as fast config "storage" for configuration files mostly related to Docker containers, particularly Homeassistant. One of the 1TB HDD's would be used for media file storage and the other would receive backup/copies of files from (I hadn't fully figured this out yet) the first HDD using rsync or similar (rather than raid). I'd then also use the Duplicati plugin to upload files to Backblaze.


    I've been tinkering with OMV with Docker containers (which are fantastic) on an old Intel Atom D525 build for a while and found that the Docker side of things in particular can throw up unexpected outcomes, errors, "ghost" containers appearing etc which combined with the fact that I'm inexperienced and getting used to how it all works and the linux commandline even though I do have a little beginner experience, meant that I really needed a good backup strategy which I hadn't figured out. Despite this, I'd got together a decent system configuration after hours and hours of quirks probably partly related to hardware issues.... and then my hardware died...


    A new system was needed and I eventually got hold of a J4015 motherboard (Intel is having supply issues and these things are like Hen's teeth). However, after searching the forums here, I just couldn't figure out the essential task of a reliable way of fairly easily backing up the entire system for when I wanted to test different settings or modify Docker containers or something went wrong etc. I'm used to using Macrium reflect in Windows to very easily create a differential image of the system drive which can just as easily be restored in case of any issues.


    Then I thought of virtualisation and initially ESXI and the idea of running OMV virtualised. This would give me the ability to snapshot and restore the system drive easily but I'm not so sure how setting up the drives would work although I could figure that out. From there I remembered Promox which would kind of make more sense since it's Debian (as with OMV) Linux based. That added questions however since apparently you can run Docker directly from the Promox GUI (I've seen Portainer which I think does/helps with this). At that point, I'm starting to wonder if it makes sense to use OMV at all. If i can just use Portainer to run and manage Docker containers then I could conceivably also just set up a SAMBA server within/from Promox which could serve the files accumulated by the Docker containers? Also there was the question of performance which was important. The J4015 is fairly powerful for what it is with Hardware Virtualisation support, but it's not a fully fledged computer in terms of speed and I couldn't find a definitive answer about the performance hit of running OMV virtualised.


    So I'm wondering if anyone has any thoughts on:


    • Is there a fairly straightforward method of backing up/snapshotting an OMV installation which is installed directly to hardware, preferably incrementally, with a GUI and which can similarly be restored? I've seen many posts about this question but all of them are pretty opaque in exactly how restores are accomplished and usually refer to Clonezilla or command line tools like DD. The DD based backup tool provided in the OMV GUI (maybe OMV-extras) interface isn't very configurable from what I can see (how to backup only used parts of the disk and enable compression) and again - how would one easily and reliably restore from a backup this way? I think the OMV FAQ is crying out for a super clear and step by step way of creating and restoring OMV installation backups as many people ask the same questions but get answers assuming a "figure it out yourself, here's the linux commands you need to use" approach.
    • If backing up and restoring an OMV installation (with snapshots or incrementally ideally) simply isn't an easy thing to do and necessarily requires either the use of existing tools "as-is" or command line expertise - fair enough. In that case - has anyone got experience running OMV as a Promox VM and ALSO on bare metal and knows the performance difference/penalty of doing it that way? How should storage be set up in this case? Does OMV need to be configured/updated with drivers etc for optimal performance on Promox? Are there any other things to consider?
    • If i'm going this direction - then would it even make sense to run OMV inside a Promox VM? Could I just as well use it to run Docker containers and then also set it up to serve files over the network, thereby sidestepping the overhead of running OMV virtualised? - although visualising OMV would bring the previously mentioned benefits of being able to snapshot the WHOLE VM, including Docker setup and configs.

    There's quite a few ways of doing this and I'm guessing drawbacks/benefits to each approach - any insight/feedback/advice would be great. Thanks

  • Is there a fairly straightforward method of backing up/snapshotting an OMV installation which is installed directly to hardware, preferably incrementally, with a GUI and which can similarly be restored?

    There are many ways to backup OMV (as in the OS) but the easiest I've found and the absolute fastest to restore is, using and cloning thumbdrives. ( Here, on page 75 in the current version.) While it's an offline process, it's simple and effective and would allow you to gracefully back out of a configuration mistake. (As you're well aware, Dockers can be finicky.)
    Lastly the Docker plugin allows for relocating Docker images under settings. Moving them from the boot-thumbdrive, to a data drive, is a good idea. Due to the potential for constant activity, USB thumbdrives is not a good place for most Dockers to live. Also, if you go with the thumbdrive option, you'll need to install the flashmemory plugin. That's in the guide as well.
    ______________________________________________


    If you want to use SSD's to boot, there's another option that would work along similar lines but you'd need 2 SSD's installed. In OMV-Extra's there's an option for installing Clonzilla. OMV allows a one time boot to Clonzilla. Clonzilla allows an SSH connection, useful for a headless server, where you could clone your boot SSD to the 2nd SSD for backup. (The choice of booting to one or the other could be made in BIOS.) This is a choice worth consideration.
    ______________________________________________


    RE Proxmox:
    I have Proxmox setup, but don't have an OMV VM running on it. (So I can't provide a performance comparison.) This is a pretty elaborate step to take, for the sake of being able to take snapshots of an operating system. Proxmox adds layers of complexity in configuring a VM guest (OMV) and in configuring the storage back end. If the only purpose for considering Proxmox is backing up OMV's OS, I'd go with either of the two above (Thumbdrives or Clonzilla with SSD's).



    Other forum users may have other opinions or ideas.


  • They're some really interesting ideas - thanks! Using USB to boot from seems like it's probably the way to go and I'm just trying to figure out whether to go for a 32GB pair of drives or 64GB since they're fairly cheap. I've read the PDF guide linked which is very useful and see that it mentions Win32diskimager which I'm familiar with. I do still think that it would be a hugely beneficial addition to have a GUI for backing up to images to a storage device from within OMV but we do have to work with what we have, maybe this will be considered for future releases?


    For the SSD idea - i'm wondering if it would work with one SSD partitioned down the middle? My SSD is 120GB so it would be two 60GB partitions. I guess in that case I'd need to be able to edit the GRUB menu? I'm not sure exactly how to do that or even if a partitioned SSD would work?


    I tried installing and setting up Promox and realised what you said in that it's overkill for what I'm trying to do - looked interesting though even if I stumbled at the hurdle of trying to figure out how to set up storage.

  • I run omv as a vm in proxmox. I don't use docker containers and only a few plugins. The omv uses very little system resources. It makes it easy to snapshot omv before an update and roll back if there is a problem. OMV makes sharing disks easy and proxmox makes it easy to increase the size of the virtual disks if needed or move them to a new disk.


    Try it you will like it.


    as for docker again I don't use that but proxmox does do lxd containers natively. They work fine.

    If you make it idiot proof, somebody will build a better idiot.

  • RE thumbdrives:
    It's just dirt simple to clone them and with one working clone, one backup, and one archive copy (the way I do it) it would be difficult to mess that up. You'd have a back out option. (I do label them, to prevent confusion.)
    For speed of cloning I'd go for the 32GB size. 64GB would last longer (wear leveling, also mentioned in the guide) but the cloning operation would take up to twice as long. If I used 64GB drives, I'd test Win32DM's option, "Read Only Allocated Partitions" to see how that works. (I have yet to try that option, but if the source and destination pass verification, that's what matters.)
    BTW: The image file itself is backup. With a date in the file name, if it's saved off to the side, you'd have a history of images to recover to. On occasion, I'll run an MD5 on a file, then compress it with 7zip and save it as an archive. The file is another recovery option.


    On using SSD's to boot:
    I don't understand the purpose behind partitioning a boot SSD. If you're going to use an SSD to boot, if you want ease of recovery, you'd need two. Otherwise, if you partition an SDD into two partitions, if the device itself fails, you're dead in the water. (No backup.)

  • I run omv as a vm in proxmox. I don't use docker containers and only a few plugins. The omv uses very little system resources. It makes it easy to snapshot omv before an update and roll back if there is a problem. OMV makes sharing disks easy and proxmox makes it easy to increase the size of the virtual disks if needed or move them to a new disk.


    Try it you will like it.


    as for docker again I don't use that but proxmox does do lxd containers natively. They work fine.


    That's really interesting. I did get Proxmox installed but couldn't figure out how to set up storage properly from within the interface as I wasn't sure how LVM should be configured or if I should use ZFS and mirror my two 1TB HDD's. Also realised that Proxmox used my whole SSD for system but I'd have liked to have used my SSD to store my Docker containers. I suppose if Proxmox can be run from a USB drive that might solve that problem easily.Main issue I had is that since Proxmox is fairly obscure I found it difficult to find any direct answers/guides as to how to set up hard drives/storage within the interface for either VM's or containers (is there an OMV LXD/C container?). I've read the documentation but am looking for practical guides/tutorials. Is the LVM the way to go? (I'm not 100% sure what the LVM is or how it should ideally be used.


    Would be interested to know how you've got your storage set up in Proxmox and how is it configured in relation to OMV?


    I've just reinstalled OMV on my SSD and am looking into getting a clonezilla backup of the system partition. Your idea about the USB pendrives sounds great and I'm going to order some to try that approach too as it does simplify things a lot. Good point about the SSD and if it dies the "backup" goes with it. I think I need more sleep instead of tinkering for hours at a time with this stuff! Thanks for your suggestions they've been very helpful.

  • You can install proxmox on a minimal debian install. That way you can set the disks up the way you like. I think setting up omv in an lxd container would work but I wouldn't do it that way. If you try and it works fine let us know.


    I use zfs mirrors for important data. Remember mirrors are not backups.

    If you make it idiot proof, somebody will build a better idiot.

  • You can install proxmox on a minimal debian install. That way you can set the disks up the way you like. I think setting up omv in an lxd container would work but I wouldn't do it that way. If you try and it works fine let us know.


    I use zfs mirrors for important data. Remember mirrors are not backups.

    Thanks for the suggestions and info. I'll hopefully have time over the weekend to give it a go. If I manage it I'll update here :D

Participate now!

Don’t have an account yet? Register yourself now and be a part of our community!