PHP memory error when running rsync

Running a brand new VM install of openmediavault on a brand new system, and when I run rsync to pull data from another server in Trial Run mode, it starts sending the file list, showing a few thousand files, then it bombs at exactly the same file every time with this message in the log:

omv-engined[1155]: PHP Fatal error: Allowed memory size of 134217728 bytes exhausted (tried to allocate 33554440 bytes) in /usr/share/php/openmediavault/functions.inc on line 573

Line 573 is a function for UTF-8 encoding. I figured memory error, I'll throw 40gb of ram at it. Nope. Increase root drive to 85gb. Nope. I found a post from 2013 that mentioned editing a bunch of php.ini files to increase the memory limits from 128M to -1. The file names have changed, but I did that in /etc/php/7.3/*/php.ini and it still bombed.

Trying to back up 30tb of files, so the file list is pretty massive. I want to make sure trial run works before starting the multi-day copy process. A sample directory rsync'd over with no issues. Is there another ini file with a 128M limit that I can change? This is a frustrating error since it's the main thing I want to do. Liking OMV otherwise.

Yes, I'm running the command in the UI. Thanks for the help. Is there a way to find out what rsync command is being sent so I can test it in the command line? I'm not an rsync guru. That would confirm whether it's a gui issue or something else.

Changing /etc/openmediavault/php.ini had an effect, but the trial run transfer still stopped at the exact same point in the "Execute rsync job" window. Before, the error would only show up in the log, but with the omv/php.ini memory limit removed, it now says the sync completed successfully in the log. There was no success message, "done" or statistics in the Execute window. I watched it only go through less than 500gb of files (dry run).

I test in Quiet mode, and after a suspiciously short time, the Execute box says "done." But it happened too quickly, and no stats were shown (though I don't know rsync well enough to know if --stats and --progress are ignored with --quiet.)

I added the verbose mode option, and this caused it to stop at an earlier point on the filesystem. I copied the file list from the Execute window into gedit and it has 685,305 lines and 54,516,618 bytes. Verbose gave extra messages like:

[receiver] expand file_list pointer array to 524288 bytes, did move

[generator] expand file_list pointer array to 524288 bytes, did move 

.... lots of stuff ...

[receiver] expand file_list pointer array to 524288 bytes, did not move

Maybe a couple hundred lines later it just stopped.

Update! I think I found the source of the problem. I was using the firefox browser, and I think that was what was running out of memory and not displaying in the Execute window. I fired up Vivaldi, and, presto, got the 647,286 reg files in 30.41T success message. Confirmed that's the correct number of files. THANK YOU.

Changing the memory limit in the php.ini was necessary, though. Would be nice to have that as an advanced config option.

Zitat von KM0201

That's what I was thinking...

I've used rsync regularly since well before version 1.0 (probably .3 or .4) and I have never had this problem.

What's the biggest file system you've backed up? Not saying remove the limit, I'm saying change the limit as an advanced option. You might not need the change, so it stays the same for you. Anyone who needs it can check a box or fill in an amount. Potential alternative would be to truncate output to the Execute window. Show only the last 500,000 lines or something. That wouldn't make a difference to anyone who wouldn't be affected by running out of php memory.

Done.

The system backup is also underway, working smoothly. I'm monitoring with watch lsof /arrayname from console since my browser decided it didn't want to stay open for the whole transfer. OMV is exactly what I was looking for.