Bad writing performance and many errors with ZFS Mirror

  • Hello,


    I just started off with my first omv nas.

    After some playing around with it and being happy with the performance on a single ext4 drive, I tried to start off.

    I added to newly shucked 8TB WD80EDAZ to the system, installed the openmediavault-zfs plugin and set those two drives up as a ZFS mirror. I set compression to on (switched it to lz4 in the meantime) and activated dedupe. (mounted the pool at /zfs-pool/ )


    Next of, I tried to move the files of my external hard drive (2,2TB of pictures and videos) onto the mirror. I plugged it in via usb3, mounted it and started of with rsync (rsync -r -v -a --progress /mnt/MyBook/ /zfs-pool/nasdata/MyBook/). The rsync ran for around 5 hours yesterday, but only managed to copy 200gb.

    Underwhelmed by the performance I went for a quick search and read about slow performance of rsync vs cp. So I stopped the rsync and switched to cp. (cp -u -v -r /mnt/MyBook/ /zfs-pool/nasdata/)

    It then ran through the night and the work day. 20 hours later it managed to copy 800gb, so roughly the same performance.

    My cpu seldom hits 100% usage, but while copying there's always around 50% usage wait i/o. (see screenshots attached)


    When I tested to copy some files from my main pc over smb to the nas, it held a steady 60mb/s. So next up I thought, the bad performance may be because of the usb connection.

    So next up I stopped the copy again, unmounted the external drive, rebooted the nas, plugged the external drive in my main pc and started a copy over a smb.

    The speed is swinging from 60mb/s to 0 and back up (it gets worse with 4gb video files). Every other minute I get a copy error. I can click try again and it warns me of the file already existing (most of the time the file is smaller because I got interrupted) (see screenshot attached for a example)

    My system log brings up a lot "monit[18483]: '\openmediavault.local' loadavg (5min) of 5.0 matches resource limit [loadavg (5min) > 4.0]".


    This makes me think that something is wrong with the zfs pool.

    Can anyone point me in the right direction? Is my rather old system not beefy enough, is zfs buggy or should I reformat the new hdds as ext4 and try transfering again, to eliminate the possibility of a harddrive failure? Could it be because of the old sata ports on the mobo (SATA 3Gb/s ports, should be fast enough for hdd's I assume)?


    NAS-System Specs:

    intel i5-2400

    P8H61-M LE/USB3 Mainboard

    8gb ram

    120gb intel ssd (SSDSC2CT120A3) as system drive

    2x 8TB WD80EDAZ as zfs pool

    1x old 1,5TB random HDD ext4 for testing, will be removed as soon as the new drives work

    OMV 5.5.22-1 (Usul)

    Kernel Linux 5.9


    Thank you a lot in advance for taking your time!


    Steve

  • The speed is swinging from 60mb/s to 0 and back up (it gets worse with 4gb video files). Every other minute I get a copy error.

    Could be caused by one of the following:

    - SATA cable defect

    - SATA port defect on mobo

    - sector size 512e versus 4k native (there have been reports in the forum)

    - HDD defect (incl. firmware)


    I'd change setup in above sequence and would search in specialized forums like https://forum.hddguru.com/view…082ff9ad784bfea0e#p281949

    omv 5.6.3-1 (usul) on RPi4/4GB with Kernel 5.10.x and WittyPi 3 V2 RTC HAT

    2x 6TB HDD formatted with ext4 in Icy Box IB-RD3662-C31 / hardware supported RAID1

    For Read/Write performance of SMB shares hosted on this hardware see forum here


  • Thank you a lot already! I will test those parts and the specialized forum and report the results.


    Looking at those options I'd assume it's most likely a hardware failure and indepent from the omv/zfs setup?

  • most likely a hardware failure and indepent from the omv/zfs setup?

    How would I be able to tell?

    Successful troubleshooting starts always with the basic and simple things

    omv 5.6.3-1 (usul) on RPi4/4GB with Kernel 5.10.x and WittyPi 3 V2 RTC HAT

    2x 6TB HDD formatted with ext4 in Icy Box IB-RD3662-C31 / hardware supported RAID1

    For Read/Write performance of SMB shares hosted on this hardware see forum here

  • Ok, thank you again. I feel quite dumb now.


    I checked the sata cables and changed the ports on the mobo - no change.

    I postponed the change of the sector size, as I was not quite sure how to do it with ZFS.

    Then I deleted the zfs pool and checked the drives as single drive individually. The both performed normaly so I assumed the drives are fine.

    After this I set up the mirror pool again. While activating compression and dedupe again, I thought maybe they were the culprits.

    Disabling compression - no change, the drops in speed while copying were there again.

    But after disabling dedupe it turned stable instantly. I first checked with a 74gb image, then with a mixed video/picture folder of 100gb.

    They were stable, so I started the 2,2tb transfer via usb again. 5,5 hours later it was done.


    After a bit of additional research it seems like that dedupe is rather heavy on the hardware. As my nas system is rather old and low on ram, it was just to much for the system.

Participate now!

Don’t have an account yet? Register yourself now and be a part of our community!