Posts by auanasgheps

    Both my real system and this VM are MBR, but just by mistake.

    In the future I will redo everything in UEFI.


    So, here are the steps:

    • Mount the filesystem where backup is located
    • Restore partition table with the following command:

    dd if=/mnt/backup/omvbackup/backup-omv-11-Nov-2020_23-32-27.grubparts of=/dev/sde bs=512 count=1

    • Format the main partition to ext4
    • Mount it
    • cd to the new mount point
    • borg extract --stdout /mnt/backup/omvbackup/borgbackup::backup-omv-xxxxxxx
    • Checked files got restored
    • Reboot

    Does Grub actually get restored as well with .grubparts?
    Because at this point, nothing boots, just got a screen with a blinking command, but can't enter anything.



    Hi,


    I am testing borgbackup as backup method in openmediavault-backup plugin. I've been using fsarchiver but I wanted something more efficient.


    Using a VM, I deleted the OMV system disk and created a new one, then booted with systemrescue 7.0. I am simulating the scenario where my real Flash dies and I need to swap it for a new one.


    I think am doing wrong two things:

    - grub restore command

    - borg restore command


    I mount the backup filesystem

    I restore grub with the following command:

    dd if=/mnt/backup/omvbackup/backup-omv-11-Nov-2020_23-32-27.grubparts of=/dev/sde bs=512 count=1


    I can see partitions in GParted, but there's a warning for /dev/sde1 which is the main partition:



    Then I attempt to restore the system with borg, am I running the right command? Taken from here

    borg extract --stdout /mnt/backup/omvbackup/borgbackup::backup-omv-11-Nov-2020_23-32-27 | dd of=/dev/sde1 bs=1M


    However nothing boots. Am I missing something?

    Thanks for the input guys and sorry for the late reply. My family kept me busy.

    Think I will start with a rsync backup and when this is running well I will look into borgbackup or rsnapshot eventually.

    Just published a guide to Borg here, hope it helps.


    I've been testing it since two weeks and it's a wonderful software.

    INTRODUCTION

    Borgbackup - friends call him Borg - is an extremely powerful and versatile backup program.


    Borg can be used in two modes:

    • local backup
    • remote backup over SSH

    You’re probably here because you want to backup data to another server - either in your LAN or over the Internet using SSH. In this scenario Borg must be installed on the other end, which can be whatever OS supported by Borg.


    The goal of this guide is to describe a simple Borg setup in remote mode using the borgbackup plugin. The official Borg documentation will help if you need more information and is the source for lot of the content in this guide.


    WARNING

    Exposing your server’s SSH port over the Internet can be extremely dangerous if not done correctly. This guide will not cover this aspect, only proceed with this if you know what you are doing.

    You need to know how to setup SSH authentication and test it first to verify this part is correct.


    BORG TERMINOLOGY

    You should know some terms before working with Borg. Don’t worry, it’s only the very basics to go through this guide.

    • repository: a directory where your backups will go. By default Borg creates this folder under the user’s home. A server can host infinite repositories. They are independent from each other, you could use one for each user or server.
    • archive: is the result of a single backup. An archive stores a snapshot of the data of the files “inside” it.
    • client: the actual OMV server you want to backup.
    • server: your backup destination server. Just install Borg and set up SSH properly.
    • passphrase: the password used to encrypt your key that encrypts your data.


    PRE-REQUISITES

    • Borg installed on your server, Borg plugin installed on your OMV client
    • client-server connection with SSH key authentication


    Borg (client) needs to know what SSH private key should be used when connecting to the server. Locate your key and execute

    Code
    export BORG_RSH='ssh -i /etc/ssh/my_secure_server_key'

    NOTE: This environment variable only applies to Borg, but if you want to do the same for standard SSH connections, create this file nano ~/.ssh/config and enter IdentityFile /etc/ssh/my_secure_server_key


    You can try the Borg connection to your server by running a command that would check a repo

    Code
    borg info 'ssh://omv@acme.com:7290/./check'


    • omv is the user on the backup server
    • acme.com is the server address
    • 7290 is the SSH port - ALWAYS use a custom port!
    • /./ tells Borg it’s a relative path inside the user’s home folder.
    • check is the backup repository - it does not exist but it’s needed to validate the SSH connection.

    You should get a message like:

    Repository ssh://omv@acme.com:7290/check does not exist.


    If this is the case the configuration is ready for Borg, otherwise there’s something wrong with your SSH/key configuration.


    REPOSITORIES

    Add a new repository by browsing to Services > BorgBackup > Repos > Add


    What you need to know:

    • Name: the repo identifier in OMV. No spaces are allowed
    • Type: Local or Remote - the latter since we’re backing up on another server
    • Remote path: the same command you typed above
      • To create a repository under your user’s home folder, use ssh://omv@acme.com:7290/./mybackup. To create a repository somewhere else, use the absolute path like ssh://omv@acme.com:7290/volume1/backups/mybackup.
      • Either way, the destination folder must not exist, Borg will create it upon initialization
      • More information about repositories paths can be found here
    • Passphrase: it is used for encryption. Use a password generator for this one, it should be very strong and stored in a safe place.
    • Encryption: enable this, you’ll want to use it
    • Skip init: if you were to add an already initialized repository

    2020-11-11 19_27-openmediavault control panel - omv-test.local.jpg


    Click Save and Borg will initialize your first repository with the provided settings.


    NOTE: If the plugin shows an error there is an issue with the remote path or you haven't configured your SSH key correctly.


    The key for this repo will be stored in /root/.config/borg/keys/. Make a copy and store it safely! It's the only way to access your backup.


    Once initialization is completed, you will see the repo and a lot of options.


    Here’s a brief description:

    • Check: verifies the consistency of a repository, archives or the actual data
    • Export: exports the repository config so it can be used on another Borg client to mount/restore/continue a backup
    • Extract: restore a backup on OMV
    • List: lists all archives in a repository
    • Mount: mounts a backup in your local filesystem with minimal network usage, only using the metadata. Files will be transferred only when they are opened or copied.

    To know more about these options, the Borg official documentation can be found here.

    2020-11-11 21_02-openmediavault control panel - omv-test.local.jpg


    ARCHIVES (BACKUPS)

    It’s finally time to create an archive to actually tell Borg what to backup.

    Navigate to Archives tab and click Add:

    2020-11-11 19_44-openmediavault control panel - omv-test.local.jpg

    2020-11-12 12_21-openmediavault control panel - omv-test.local.jpg


    What you need to know:

    • Name/Prefix: the archive name. No spaces are allowed
    • Repo: the repository we configured a minute ago
    • Compression Type: lz4 is the default compression method, but you might consider zstd. More info about compression can be found here
    • Compression ratio: applies to the type of compression you have chosen, more info at the same link above
    • One Filesystem: stay in the same file system and do not store mount points of other file systems
    • No atime: don’t store atime (last access time)
    • Includes: paths of the folders you want to backup. In this example I’ve added my UnionFS path.
    • Excludes: paths of the file/folders you want to exclude
    • Hourly, Daily, Weekly, Monthly, Yearly: How many archives you want to keep. They depend on your needs, more info can be found here. It also doubles down as scheduling.
    • Rate Limit: in case you want to limit the upload speed.


    Click Save and your backup is ready to go. Click Run to start it immediatelly.


    NOTE Every archive/backup execution is scheduled as frequently as the first non-zero retention policy. In this example it will run every day because Hourly is set to 0.

    This setting is baked into the plugin and not in Borg, which is not scheduled by design.


    FINAL NOTES

    The first backup will take a long time: this is expected, but the following ones will be much quicker.


    If the SSH connection is being dropped during the initial backup, it is likely that your client is compressing a lot of data and the receiving server is closing the connection due to inactivity.

    You can increase the timeout on your receiving server. Edit /etc/ssh/sshd_config and add the following

    Code
    ClientAliveInterval 10
    ClientAliveCountMax 30

    This will increase the maximum timeout to 30 minutes. It should not be needed after the first backup.


    You have two options to restore files: restore essentially restores everything to a defined path, while mount is great to browse your backup and pick a particular folder or file.



    Thanks to ryecoaaron for making this plugin and answering my questions.


    Happy backups!

    Yes, because spindown worked for the data drive until I installed NC. App and DB are on the data drive :saint:

    To change this would if work to delete NC, modify the compose file (App and DB folder on SSD), redeploy and copy old app and dB folder to the new destination on SSD. Or do I have to do a "clean" install of NC?

    Yes, this breaks everything.


    Are you using Docker to install Nextcloud? If yes, you can simply move and redeploy.


    I guess you have only one partition on the SSD occupied by OMV. Use the following approach

    • Since OMV only requires little space (32GB is plenty) resize the current partition with a Gparted live distro and create a new one with the remaining space
    • Install docker binary on the new partition
    • Install any app, db or docker container on the SSD, while pointing to data drives when needed.


    This is the best approach for speed since all DBs will be on SSD and power consumption since HDDs will be allowed to spin down.


    I am a fan of HDDs spindown as well - I rewrote the hd-idle guide because was bad.


    That's ironic because right now I am troubleshooting a similar issue where drives spin up by themselves, and I am going trough my dockers.

    Are you sure Nextcloud prevents spindown of the data drive?

    If it's not doing anything it does not access the drive.


    EDIT: If you installed Nextcloud on the data drive it will never spin down, but if Nextcloud app and its db are running from the SSD and access HDDs only to browse data, it should sleep.

    Hi, you might take inspiration from my build in the signature.

    It's MiniITX, has space for up to 6 HDDs, you could use a cheaper CPU if you want.


    I recommend 8GB, memory is so cheap these days

    This seems like a clean solution that I am happy to use. Thank you.

    Maybe rename the variable to OMV_POSTFIX_MAIN_MESSAGE_SIZE_LIMIT?

    I definitely have to start using these variables more often.


    auanasgheps yes I would be interested to see an alternative Snapraid script.

    Variables are a clever solution to change advanced stuff that must be persisted and is not worth showing in GUI.
    You'll be surprised by the amount of variables available.


    To see all of them, run


    grep -r 'default:OMV_' /srv/salt/omv/* | cut -d ":" -f3 | cut -d')' -f1 | sort | uniq


    I'll PM you the script I've put together.

    On many BIOS´s there is an option to power on automatically when power returns.

    That's what I ended up using on my server. To be frank I think it's on all BIOS's, obviously with a different name.

    I've tested it because there were many power outages last month and the server turned on every time the power came back because the UPS does a power cycle.

    It's working beautifully and I'm so thankful I've bought a UPS just in time.

    Vedo però nella tua firma che, invece Tu, utilizzi una USBKey per l'OS e un NVMe per i Container...uhmm...non ci avevo pensato allo spazio per i containers X/

    Ho voluto ottimizzare il più possibile.

    L'NVMe per l'OS è sprecato, ma puoi partizionarlo NVMe (ti basano 20-30 GB) per OMV e per i container.


    In realtà devi prima installare OMV sul NVMe, che creerà una partizione grande quanto tutto il disco. Avvia una bootable di Gparted, riduci la partizione e creane un'altra che userai per i container.


    Il RAID è sempre consigliato farlo gestire a OMV, non usare quello HW della motherboard.

    Io uso SNAPraid che è un raid alternativo (sempre software).


    Con 4 hard disk meccanici + il resto non so se ce la fai con il pico PSU. Quanti W eroga quello che hai scelto? Ha abbastanza connettori SATA?

    La scelta del processore era proprio basata sui bassi consumi


    All'inizio di quest'anno ho costruito il mio server OMV con obiettivi simili: buone prestazioni e consumi bassi. La mia configurazione è in firma. La CPU è perfetta per fare tutto e anche se è a 65W non arriva mai a tanto perché la sfrutto relativamente poco.

    L'alimentatore è meglio prenderlo il più efficiente possibile ai bassi carichi e dimensionato alle tue esigenze. Un pico PSU può andare bene ma dipende quanti dischi meccanici vuoi alimentare.


    Per trovare i componenti compatibili usa https://pcpartpicker.com/

    Nessuno può darmi qualche dritta (non dritto nè :P ), almeno sul processore / MOBO? Grazie

    La CPU va benissimo, consumi bassi ed efficiente. La versione T è inutile e difficile da trovare, prendi quello normale che tanto non andrà mai a massimo carico.

    OMV non gestisce "direttamente" le telecamere, non ha nulla di integrato, ma ho un amico che gestisce 2/3 flussi video senza problemi con un container docker.

    Il fatto che "/dev/sda" ecc... cambi a ogni riavvio/accensione, non so se sia normale o meno (non ci ho mai fatto caso onestamente); di sicuro cambia se cambi l'hardware del server (io l'ho fatto di recente senza reinstallare nulla).


    Tenete presente che, se non ricordo male, OMV monta i dischi usando la "label" come identificativo, quindi se il disco chiamato "DATI" un giorno è /dev/sda e un altro è /dev/sdc, è irrilevante.


    Tant'è che la path di solito è /dev-disk-by-label/DATI e non /dev/sda/DATI.

    Giusto, per la path per fortuna viene usata la label, ma i nomi delle unità non dovrebbero cambiare, e ho paura che si utilizza il RAID tradizionali ciò potrebbe causare problemi.