Rsync from one Hdd to another HDD doesn't work

  • Hello, I have the strangest problem. I have a NAS with 2 harddrives. One is used for nextcloud, the other one is a backup disk. I have placed a rsync job, to copy local, the entire content of the folder nextclouddata to a folder on the second hdd named weekly-backup. No matter what i do, when the time comes to execute this job than i see hdd's spinning up and than they stop and the rsync doesn't do any job except for making some folder names and not even that is done completely. The second harddrive is basically empty as this would be my first backup, so it is indeed about 100 Gb of data that needs to be backuped. Anyone who has an idea why this is not working at all. It seems like the most basic job of openmediavault, to take backups from data at certain times. thx

  • I think there are two main problems possible:


    1. Access rights - Make sure the job is run by/as a user with sufficient access. When you test the job using manual commands or scripts, use the exact same user as is used in scheduled jobs.


    2. Shell environment - I have noticed that some shell environments seem to be poorly configured by default in OMV. (OMV4 at least.) (In Debian?) Since I started using only the standard bash shell in OMV4 my scheduled jobs works fine. Again, when you test the job using manual commands or scripts, use the exact same shell environment as is used in scheduled jobs.


    I don't use the OMV GUI for scheduled jobs or rsync jobs. Instead I use an editor and entries in /etc/crontab to run bash scripts. Works fine. I assume that the OMV way works fine as well, but I used crontab and bash scripts long before I started using OMV.


    I typically run scripts from crontab by running bash with the script as a parameter.

    Be smart - be lazy. Clone your rootfs.
    OMV 5: 9 x Odroid HC2 + 1 x Odroid HC1 + 1 x Raspberry Pi 4

  • Well, this is more complicated than it seems.


    So i have unmounted the entire drive, formatted it and remounted it with clear read write permissions. used this command to check

    cat /proc/mounts


    Than i made a sharedfolder on this disk and checked again for permissions. it was still read/write


    Than i made a rsync job in the GUI, just a standard job. And than i clicked RUN the job.


    The result is that all folders are created and some files on this backupdisk but the whole disk is reverted to read only and know i can not write to it, nor is the rsync job doing anything, except giving the error of read only filesystem.


    How is this possible?

  • I don't use Nextcloud. It may be Nextcloud that mess things up?


    It seems that to backup a Nextcloud install you are supposed to follow a certain procedure. Using rsync is only one part of it. Especially you are supposed to set Nextcloud into maintenance mode before running rsync.


    Do you follow this procedure? Here it is for version 18:


    https://docs.nextcloud.com/ser…l/maintenance/backup.html


    If you are running Nextcloud in a docker I doubt you can use the OMV GUI to backup Nextcloud. I assume you need to access the docker container directly and then use the command line.


    I assume you tried to backup NC by running rsync on a running docker container? Then I assume very strange things may happen. As you noticed. You might get away with first shutting down everything docker and then rsync. Or perhaps clone the partition. IDK.


    This link may help (even if it uses tar rather than rsync):


    https://forums.docker.com/t/ho…llation-with-docker/85081

    Be smart - be lazy. Clone your rootfs.
    OMV 5: 9 x Odroid HC2 + 1 x Odroid HC1 + 1 x Raspberry Pi 4

    Edited once, last by Adoby ().

  • I dont want to backup the nextcloud installation. i just want to backup the files it has synced on my drive in OMV.


    I just dont understand why rsync is changing the read write permissions. and what are the settings for setting a rsync job without it altering the read write permissions on a disk and thus locking itself out of the disk

  • Perhaps you are not supposed to be able to access those files? I have a vague feeling that there is something important you are not telling us, and that is the cause of the problems.


    Do you have root access? What are the filesystems involved? Is there any form of compression or encryption involved?


    What happens if you copy one file?


    Typically...


    rsync -a source destination


    ... is used to copy files recursively with all attributes. Archive.


    I assume the archive mode is available in the OMV GUI.


    Or use the command line...

    Be smart - be lazy. Clone your rootfs.
    OMV 5: 9 x Odroid HC2 + 1 x Odroid HC1 + 1 x Raspberry Pi 4

  • Hi adoby,


    I am fairly new to Linux, so I have to research everything.


    I told myself, let stop trying to do this rsync running from omv and try it directly in the command line.


    So I ran indeed a few test files. Worked perfectly. Than I gave the rsync -av /shared folders/NextCloudData/ /shared folders/WeeklyBackup/


    I execute and the files start copying until it copied a file named "laser rar.rar" (this is a rar file I created. So nothing wrong with it) but it hangs on that file and than the whole disk and all folders become read only and there is no command I can give to overwrite this. Not even umount command works. It just says read only .

    Can a rar file cause this issue?


    My problem is to get the drive back I have to format it completely. Which is 2tb and takes about 30 min to accomplish. It is driving me nuts.


    Believe me I am telling you all I can here! :)

  • OK.


    There could be things wrong with the original filesystem. But then I would expect you to get errors during mount. Or there could be things wrong with that file. Or even with the filename itself. But you created it...


    It may be possible to intentionally mess up a filesystem or a filename so that it croaks a computer. Is it possible that someone has doctored the hdd or the filesystem. To poke fun at you?


    Where do the source hdd come from? A windows system? Is it fat32 or NTFS? And the destination, what filesystem do you format it into. Use ext4!


    Try to rename the offending file. Remove the space.


    I would check the source hdd. You can move the original hdd to a desktop Linux PC and chkdsk the hdd. Or boot the NAS from a thumbdrive. You can also run checks from gparted.


    Something is very strange with your hdd. It is not normal... Most likely that what trips rsync. You need to find out what is wrong with it and fix it.

    Be smart - be lazy. Clone your rootfs.
    OMV 5: 9 x Odroid HC2 + 1 x Odroid HC1 + 1 x Raspberry Pi 4

  • Hi,


    Both HDD's are brand new coming out of the box and have been used for about 48 hours. I bought them to make this NAS system.


    It's running on raspberry pi 3. The sd card is my bootdrive. So both HDD have nothing to do with the Linux or OMv installation. They are both used for storage only. Both set as ext4. The format is done by omv.

  • OK. Then a new suspect is the USB connections. And the power supply.


    Do you by any chance also use a powered USB hub? =O


    You might test with other cables. Move things around. Or have only one HDD connected to the OMV box. The other to a Linux PC and rsync over the network.


    And, as I said before, check the filesystem on the source hdd for errors!

    Be smart - be lazy. Clone your rootfs.
    OMV 5: 9 x Odroid HC2 + 1 x Odroid HC1 + 1 x Raspberry Pi 4

Participate now!

Don’t have an account yet? Register yourself now and be a part of our community!