Beiträge von PangPinne

I have been trying to get rsync to work on my end (OMW) and my families end (QNAP) without any success (different locations).
Out networks are joined through a VPN tunnel. OMW can reach the QNAP shares and vice versa.

However, when trying to setup rsync to backup some of my data to QNAP and some of my families data to OMW, we are stuck.

When I run the job on OMW I get the following errors back:
Please wait, syncing </srv/dev-disk-by-label-Red/Backup/> to <rsync://username@qnap-remote:873/RM_Backup_Omw> ...

@ERROR: auth failed on module RM_Backup_Omw
rsync error: error starting client-server protocol (code 5) at main.c(1666) [sender=3.1.2]

And when I try testing the connection from QNAP it fails with a generic error of checking password, port etc (which we have checked are correct on both ends).
If I SSH into the QNAP and try a similar command:
rsync [options] '/share/MD1_DATA/Backup/' 'rsync://username@omv-remote:873/RM_Backup_Qnap'

I just get Segmentation fault error.

What I'm I doing wrong?

Zitat von tkaiser

You have them but since you don't read the replies to your questions it's useless to continue. Again: Home NAS build, FS info (last paragraph!)
Every time I see someone using RAID-1 in 2018 I feel sad. It's soooooo anachronistic and provides almost no protection.

Edit: apologies! Just realized that the last link in my old post was broken. Corrected it in the post and here: https://blog.a2o.si/2014/09/07…aid-10-instead-of-raid-1/

My knowledge about raid and linux is limited (still learning and want to learn).
But isn't RAID 10 with two disks (1 partition each) RAID 0? If one disk fails, all the data is lost? Or i'm I missing something here how RAID 10 works?
The Web GUI says 4 disk minimum when trying to create a RAID 10 device.

Zitat von Adoby

Yes, you are right: Write Cache on the HDDs.
From the manual:

Power Options
Pressing the edit button with a selected disk will give the following options available to set:

• Advanced power management (APM)
• Automatic accoustic management (Not all drives support this)
• Spindown time (ST)
• Write cache

Thank you, just activating Write Cache showed no change. But tweaking the values of vm.dirty_ratio and vm.dirty_background_ratio gave improvements!

Default values were:

• vm.dirty_ratio = 20
• vm.dirty_background_ratio = 1

I tried different values until I found these worked pretty well:

• vm.dirty_ratio = 10
• vm.dirty_background_ratio = 5

Left is new values and Right is default values

Will continue to experiment later, and reply with an update if more improvements are made.

Zitat von donh

What kind of raid controller are you using? If it is onboard you might try using mdadm.

No hardware controller, software controller (i think?).

I have the disks plugged in normally to my motherboard, then I created a RAID1 from within the OMV GUI.

Zitat von tkaiser

IMO a really bad idea (reasons). Anyway: you need to test your storage locally first to see whether storage performance is ok or not.
Drops when writing through the network can be due to the kernel's filesystem buffers filling up (now network transfer speeds only been limited by Gigabit Ethernet) and once they need to be flushed to disk the performance drops down to storage performance.

Personally I use iozone to test for storage performance (in the upper right corner is a search function). Others use dd for the same purpose.

What do you recommend instead of RAID1? I don't have the disks for RAID10 (If I understand it correctly, minimum is 4?) and is ZFS snapshots possible through OVM (I will have to read more about it too)?
Maybe something like @Adoby mentioned? Daily rsync to the other disk?
I tried iozone, but running the command sudo apt install iozone3 returns E: Unable to locate package iozone3. What have I missed?

Zitat von Adoby

It seems that your disk writes are slower than the network throughput. And pdflush throttle/locks other I/O while it flushes the full write cache. That is what you see as a dip/halt in throughput.

Raid 1 also typically slows down writes compared to single disks. If write performance is very important you might consider using one of the disks for automated backups instead of raid? Possibly using rsync snaphots.

Less drastic is to make sure you have write back activated on your disks. That might help throughput, possibly at the expense of corrupt disks in case of power failures.

You might also be able to tweak the write caches by changing the settings for:

vm.dirty_background_ratio=10

vm.dirty_ratio=20

See pages 109-110 in the book "Linux Performance and Tuning Guidelines" or google for other write-ups. Not for newbies...

pdflush starts writing dirty data in the background when dirty pages/RAM exceed dirty_background_ratio. Typically at 10%.
pdflush throttle/lock other I/O when dirty pages/RAM exceeds dirty_ratio. Typically at 20%.

(Dirty pages are also flushed when they get old...)

Perhaps something like...

vm.dirty_background_ratio=5

vm.dirty_ratio=40

...might help increase/smooth throughput.

However tweaking the cache behavior will most likely not help a lot if you are writing very large files, significantly larger than free RAM for the write cache pages. But it could help a lot with smaller random writes over a fast network to slow disks. Especially if you have plenty of unused RAM and an UPS...

At least it might be interesting to experiment with...

Alles anzeigen

"write back activated on your disks" - do you mean Write Cache?
Where can I change these values? Do I need to restart the machine each time or are the changes detected and used immediately?

Zitat von jollyrogr

I didn't notice that it was a recording. I see what you're referring to now. Do you have another computer to try transferring from to possibly eliminate your sending machine as the cause?

Yeah, same problem occurs on my other machines, a desktop and laptop (wired). All running windows 10.

Zitat von jollyrogr

From the screenshot it doesn't look like it's dropping that low to me? Try FTP to compare.

It's a screen recording, after a few seconds the speed drops to 0 (but only a small decent in the graph).

I just tried FTP, there it freezes for a second, then resumes with a slight lower speed and it repeats.
https://streamable.com/wiy5r

Hi,

I recently installed OMV (4.1.9) on an older gaming PC to try it out and experiment.

I have configured two 8TB WD RED disks (WD80EFZX-68U) as a raid 1. But I've noticed that the write speed drops to a very low speed and resumes again with large files. I am currently testing with very large files, 40GB
https://streamable.com/0man6

What is causing the drop? Read speed is unaffected from what I can see.

Hardware
CPU: Intel i7-5930K
RAM: 16GB