How do I know if the rsynch copy is good and usable ...

If you run a second rsync operation and there is no files list or transferred data, the copy is complete. In that case, if no errors are reported and there's no changed files list, the drives are mirrored.

If you have doubts, you can switch a shared folder from the source drive to HDD2, using the Guide's instructions for recovery. You'll find that the backup share on HDD2 is identical to the source in every detail and is immediately available. After the test, simply switch the shared folder back to the primary drive HDD1 in the same manner.

If you'd be happy with a files comparison, install midnight commander per the Guide and check the contents of the drives in the left and right windows. But exercise some caution. MC doesn't have an undo button. Once confirmed, operations are immediately executed.

With

diff -qr source-folder/ target-folder/

you can check if there are different files on source and target.

To check if the files were copied correctly you can run rsync with the -c option

This can take a long time.

Zitat

-c, --checksum
This changes the way rsync checks if the files have been changed and are in need of a transfer. Without this option, rsync uses a lqquick checkrq that (by default) checks if each file's size and time of last modification match between the sender and receiver. This option changes this to compare a 128-bit checksum for each file that has a matching size. Generating the checksums means that both sides will expend a lot of disk I/O reading all the data in the files in the transfer (and this is prior to any reading that will be done to transfer changed files), so this can slow things down significantly.

Rsync is around for a loooooong time (since 1996) and very widely used. So I would trust it.

Log files are in /var/log/

e.g. /var/log/rsync

You can also see the content of the log file in the GUI of OMV. What is being logged depends on the log level set. With -v as used in the command in your first post the verbosity is increased.

Did you run exactly the same command? A type or a missing or added / can make a big difference.

In the GUI you find log files in

Diagnostics >> System Logs >> Log

use the drop down field at the top to select the log file you want to see.

What is the output of

ls -al /var/log/rsync*

on my system it looks like this:

-rw-r----- 1 root adm    0 Nov 13 00:00 /var/log/rsync.log
-rw-r----- 1 root adm 1019 Nov  8 20:57 /var/log/rsync.log.1.gz
-rw-r----- 1 root adm  266 Nov  1 20:57 /var/log/rsync.log.2.gz
-rw-r----- 1 root adm  213 Oct 25 20:57 /var/log/rsync.log.3.gz
-rw-r----- 1 root adm  216 Oct 18 20:57 /var/log/rsync.log.4.gz

rsync.log is currently empty (0 byte). The log has been rotated to the compressed log files with .gz at the end

When you run the rsync job with the GUI of OMV rsync creates the logs. Might be that you have to specify a log-file when running it from the CLI.

Try to add --log-file=/var/log/rsync.log to the command you are running.

Just tested it. It is working like this. Did not know that rsync does not write a log file by default.

Zitat von Taipan

Given that there is an option in OMV to view the rsync logs, I am surprised that there is no mention in the document "Getting_Started-OMV5" that it is necessary to include --log-file=/var/log/rsync.log in the command line to produce the log files ...

crashtest might be worth a consideration to add this hint to the guide (though I would prefer to use the rsync capability from the GUI of OMV even if shared folders have to be created for this)

Zitat von macom

crashtest might be worth a consideration to add this hint to the guide

The idea was to help users create disk-to-disk backup, that's "dirt simple to restore", that could be setup and recovered from inside the GUI. (No command line required.) Of the thousands that have read the user guide, this is the first time I've heard a user doubt the outcome of an rsync operation and seek confirmation that it actually took place.

In the interests of trying to keep the guide simple and a reasonable length, I'm a bit leery about adding more detail. The User Guide is a start for learning OMV and, to a much lesser degree, Linux. It's not all inclusive and it's already over 80 pages long. There's much to be learned from tutorials and manpages on rsync.

Zitat von Taipan

I am not sure what you mean by that?

I have only ever been using the GUI of OMV and the "command line" that I refer to is the one in the Scheduled Job window:

macom is referring to rsync as it's found under Services, rsync. This implementation of rsync synchronizes two shared folders. It's useful but there are limitations, particularly when trying to describe how to mirror an entire data disk in a single operation.
((There are ways to do it, using Services - rsync, like sharing a root folder of a data drive with all SMB network shares added as sub-folders, but these are very specific setups that are not new user friendly.))
_______________________________________________

As an example of rsync under Services, in the GUI:

This is a old backup server that I setup a long time ago and didn't change. It uses the remote mount plugin, to make a remote share appear to be local, and rsync to "pull" sync remote shares. This is a relatively simple method of replicating server to server, one SMB share at a time, thereby creating a fully functional backup server. It's something I've thought about adding to the User Guide.

Perhaps it should be written as an addendum.

Zitat von crashtest

((There are ways to do it, using Services - rsync, like sharing a root folder of a data drive with all SMB network shares added as sub-folders, but these are very specific setups that are not new user friendly.))

It is possible to create a shared folder that is pointing at the complete drive, e.g.

Zitat von macom

It is possible to create a shared folder that is pointing at the complete drive, e.g.

The other option is, just create a path that is the top directory, and everything I want backed up by rsync, goes in there...

ken@openmediavault:~$ cd /srv/dev-disk-by-label-D1/Disk_1/
ken@openmediavault:/srv/dev-disk-by-label-D1/Disk_1$ ls
Data  Media
ken@openmediavault:/srv/dev-disk-by-label-D1/Disk_1$ cd /srv/dev-disk-by-id-ata-ST4000DM004-2CV104_ZFN1R4FX-part1/Disk_2/
ken@openmediavault:/srv/dev-disk-by-id-ata-ST4000DM004-2CV104_ZFN1R4FX-part1/Disk_2$ ls
Data  Media
ken@openmediavault:/srv/dev-disk-by-id-ata-ST4000DM004-2CV104_ZFN1R4FX-part1/Disk_2$ 

in the example above, Disk_1, and Disk_2, are top directories. Only two directories I don't put under there, is AppData and Containers (Home is under Media). What i've found is if I add Containers and or AppData to the job, if a container is logging, it will constantly loop and cause a bit of a problem (this was a long time ago w/ omv 3.. .so I just stuck with it). My Disk_1 to Disk_2 sync is automatic at 0900 every morning. AppData and Containers, I have manual jobs set up.. usually I log in, stop docker, run those two manually, then start docker again when it's done

Since my actual docker configs and containers do not change all that much, this works pretty well

For the docker stuff you could use rsnapshot-plugin for automation. The plugin allows you to run a command before and after the rsnapshot job runs. So sequence would be

1. stop docker
2. run rsnapshot
3. start docker

Zitat von macom

For the docker stuff you could use rsnapshot-plugin for automation. The plugin allows you to run a command before and after the rsnapshot job runs. So sequence would be

1. stop docker
2. run rsnapshot
3. start docker

I considered it... but honestly I didn't see any reason to run something else when rsync seemed to work just as well. I usually only sync my containers maybe once a month... it's just not necessary for me to do it more frequently than that. Other people's work flow may require it more frequently, but how I've got it works fine.