New openmediavault-kvm plugin

ryecoaaron · 6. Juni 2022

Zitat von mamema

Would that help with my backup?

Nope other than use less space.

Zitat von mamema

what would be different if a thin file would be backed up by your solution?

I have never compared the source file to backup file like you are doing. While the data and size should be the same I would think, I don't know if cmp would say they are the same.

Zitat von mamema

i have qcow2 files, are those not Thin files?

Yes they are thin.

Zitat von mamema

They are also not 1:1 the same even with not running VMs after backup....

Assuming you don't have the compression checkbox checked, qemu-img convert must be re-converting them. So, I am adding a "same format" flag to the backup script and scheduled backup form that will make the plugin use cp instead of qemu-img convert. I still need to look at something to skip the backup if the VM is off. I won't use any of this. So, it takes a bit longer to test.

mamema · 6. Juni 2022

Zitat von ryecoaaron

So, I am adding a "same format" flag to the backup script and scheduled backup form that will make the plugin use cp instead of qemu-img convert. I still need to look at something to skip the backup if the VM is off. I won't use any of this. So, it takes a bit longer to test.

thank you, if you need somebody to test, i'm happy to help

ryecoaaron · 6. Juni 2022

Zitat von mamema

thank you, if you need somebody to test, i'm happy to help

6.1.17 is in the repo.

ALEXSANPEDRO The pool with quote issue should be fixed as well.

mamema · 7. Juni 2022

Zitat von ryecoaaron

6.1.17 is in the repo.

ALEXSANPEDRO The pool with quote issue should be fixed as well.

thanks, tested, but still not working as expected

used the -s switch (same as source)

The files hashes are:

source: 6706823168,bf32500a3b863f08c577f40b0a611e71 (filesize,md5 hash)

kvm plugin: 6706823168,364decab00a6cd443111e9ef65447570 (filesize, md5 hash)

copy cp command line: 6706823168,bf32500a3b863f08c577f40b0a611e71 (filesize, hash)

as you can see, the command line copy is identical, your copy isn't

my assumption is, as you are not only copying but also renaming the file(s). Which is not needed, as you are creating backup folders with the current date and time. You could leave the files untouchted because of that.

mamema · 7. Juni 2022

further tests lead to the conclusion that the -s switch and the then used cp command isn't used, even selected in the GUI....but is used via cmd line

hashes are equal! Filename change DOES NOT change hash.

so almost..... GUI isn't picking up the -s switch

....oh and could it be that even though -k 3 is used in scripting, that more than three backups remain?

ryecoaaron · 7. Juni 2022

Zitat von mamema

further tests lead to the conclusion that the -s switch and the then used cp command isn't used, even selected in the GUI....but is used via cmd line

hashes are equal! Filename change DOES NOT change hash.
so almost..... GUI isn't picking up the -s switch

This change is killing me..............

And it is working on my system

Same format unchecked in the web interface

Code

aaron@omv6dev:~$ sudo omv-showkey kvm | grep samefmt
<samefmt>0</samefmt>
aaron@omv6dev:~$ grep cpvm /etc/cron.d/omv-backup-vm
@weekly root omv-backup-vm -v cpvm -d /tmp/backup -k 1 >/dev/null 2>&1

Same format checked in the web interface

Code

aaron@omv6dev:~$ sudo omv-showkey kvm | grep samefmt
<samefmt>1</samefmt>
aaron@omv6dev:~$ grep cpvm /etc/cron.d/omv-backup-vm
@weekly root omv-backup-vm -v cpvm -d /tmp/backup -k 1 -s >/dev/null 2>&1
aaron@omv6dev:~$ sudo vi /usr/sbin/omv-backup

Running script with -s flag to make sure script sees it.

Code

aaron@omv6dev:~$ sudo omv-backup-vm -v cpvm -d /tmp/backup -k 1 -s
[2022-06-07 06:22:13-0500] [backup] Backup VM :: cpvm
[2022-06-07 06:22:13-0500] [backup] Backup dir :: /tmp/backup
[2022-06-07 06:22:13-0500] [backup] Keep :: 1
[2022-06-07 06:22:13-0500] [backup] Same format :: 1

Maybe you need to run to get your salt cache up to date.

sudo /usr/bin/salt-call --local saltutil.clear_cache

sudo omv-salt stage run prepare

Zitat von mamema

.oh and could it be that even though -k 3 is used in scripting, that more than three backups remain?

On my main system, I am backing up five VMs. Two are keeping 2 backups, one is keeping 1 backup, and two are keeping 3 backups. All of them are keeping the proper number of backups. Not sure what to tell you on this one.

mamema · 7. Juni 2022

your commands didn't help.

on my side is more than one thing odd. After the update to the latest plugin, also the email notifications are no longer working (always talking about gui)

Whats the best approach? Reinstall the plugin. I've already rebooted without help

ryecoaaron · 7. Juni 2022

Zitat von mamema

your commands didn't help.

Of course they didn't. grrr. What is the output of:

sudo omv-showkey kvm | grep samefmt

sudo omv-salt deploy run kvm

cat /etc/cron.d/omv-backup-vm

Zitat von mamema

on my side is more than one thing odd. After the update to the latest plugin, also the email notifications are no longer working (always talking about gui)

All the plugin does is create a cron job. I have no idea what it could have possibly broke. If you are referring to all email notifications from OMV itself, the plugin did not break them.

Zitat von mamema

Whats the best approach? Reinstall the plugin.

I don't see how reinstalling could help anything. If you purge the plugin, you will have to recreate your scheduled tasks. If you just remove it, the config won't be lost. But I still don't know what this would fix.

mamema · 7. Juni 2022

notifications work, just the settings from YOUR plugin in the web GUI don't. Working fine from command line.....

Code

sudo omv-showkey kvm | grep samefmt
          <samefmt>1</samefmt>

Code

debian:
----------
          ID: configure_kvm_scheduled_jobs
    Function: file.managed
        Name: /etc/cron.d/omv-backup-vm
      Result: True
     Comment: File /etc/cron.d/omv-backup-vm is in the correct state
     Started: 14:17:21.233402
    Duration: 51.575 ms
     Changes:   
----------
          ID: configure_omv_backup_vm_logrotate
    Function: file.managed
        Name: /etc/logrotate.d/omv-backup-vm
      Result: True
     Comment: File /etc/logrotate.d/omv-backup-vm is in the correct state
     Started: 14:17:21.285142
    Duration: 3.542 ms
     Changes:   
----------
          ID: configure_omv_virsh_command_logrotate
    Function: file.managed
        Name: /etc/logrotate.d/omv-virsh-command
      Result: True
     Comment: File /etc/logrotate.d/omv-virsh-command is in the correct state
     Started: 14:17:21.288815
    Duration: 2.773 ms
     Changes:   

Summary for debian
------------
Succeeded: 3
Failed:    0
------------
Total states run:     3
Total run time:  57.890 ms

Alles anzeigen

Code

# This file is auto-generated by openmediavault (https://www.openmediavault.org)
# WARNING: Do not edit this file, your changes will get lost.
SHELL=/bin/sh
PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin
# m h dom mon dow user    command
44 13 * * * root omv-backup-vm -v Metasploitable_3_Linux -d /srv/dev-disk-by-uuid-0c6254ff-7e8e-4056-ae70-1c79fbce3d47/VM_Export/Metasploitable_3_Linux/ -k 3 -s 2>&1 | mail -E -s "Cron" -a "From: Cron Daemon <root>" root >/dev/null 2>&1

..i've install anacron plugin yesterday, testing something, uninstalled..... perhaps.....

ryecoaaron · 7. Juni 2022

Zitat von mamema

just the settings from YOUR plugin in the web GUI don't

From the KVM plugin? Something must not be working with the mail command then? The kvm plugin doesn't configure anything that the mail command uses.

Zitat von mamema

-k 3 -s

I see the -s flag there. So, it does look like things are working.

mamema · 7. Juni 2022

i can modify the cp part in the script to prove, that it is never touched, when launched via GUI, even when the -s param is configured.

Does it matter, that i start the script via the GUI and not waiting for the triggered time?

ryecoaaron · 7. Juni 2022

Zitat von mamema

i can modify the cp part in the script to prove, that it is never touched, when launched via GUI, even when the -s param is configured.
Does it matter, that i start the script via the GUI and not waiting for the triggered time?

I misunderstood the problem you were having. I thought it wasn't adding the -s param.

And yes, it makes a difference that you are running manually. I missed adding the -s param when running a scheduled task from the schedule tab. I just fixed and will push out in a bit. I thought you were running the command directly from the cron job. https://github.com/OpenMediaVa…237b7ff52057f765483fd12b5

mamema · 7. Juni 2022

so, wonderful to help you (me) out then. :-). Is this behaviour then the same with my email problem? I guess so....

ryecoaaron · 7. Juni 2022

Zitat von mamema

Is this behaviour then the same with my email problem? I guess so....

Manual runs of scheduled tasks will never send an email.

ryecoaaron · 7. Juni 2022

6.1.18 is in the repo.

mamema · 9. Juni 2022

Zitat von ryecoaaron

6.1.18 is in the repo.

sorry for the late reply, but tested a bit longer to be sure everything is working as it should. Which it is. Thank you.

ryecoaaron · 9. Juni 2022

Zitat von mamema

sorry for the late reply, but tested a bit longer to be sure everything is working as it should. Which it is

Thanks for the update.

mamema · 17. Juni 2022

i had problems with one VM just daily backed up even though it shouldn't, according to my script.... what the heck! It took me three days to remember to look into the KVM plugin, because there is also a crontab menu.

Dooh!!

Perhaps a good idea, like with the usb backup plugin, to be able to create schedules within the kvm plugin, but put the crontab entries into the "master" crontab from omv.

Ginpachi98 · 19. Juni 2022

6.1.18; tried all buses and multiple volumes
it works when you create volume tho

ryecoaaron · 19. Juni 2022

Zitat von Ginpachi98

6.1.18; tried all buses and multiple volumes
it works when you create volume tho

What kind of pool?

Jetzt mitmachen!