Just built a new machine specifically for OMV - super slow writes

  • Hi all,


    I just finished building a brand new machine for my OMV server. It's pretty beefy, with the intent (someday) of using ZFS/BRTRFS once it's 'baked-into' OMV.


    Anyway, doing some very basic tests, I'm seeing write speeds that just don't make sense. I'm used to a single drive being able to do (conservatively) about 50MB/sec or so on SATA. However, here's what I'm getting on OMV:


    Code
    omv ~ $ dd if=/dev/urandom of=./test.dd bs=1048576 count=2048
    2048+0 records in
    2048+0 records out
    2147483648 bytes (2.1 GB) copied, 155.324 s, 13.8 MB/s


    That's, IMHO, exceptionally slow for a 4x2TB WD Red RAID5 array when the CPU is not even close to pegged.


    Read speeds are in the range that I think is maybe reasonable:


    Code
    omv ~ $ dd if=./test.dd of=/dev/null bs=1048576
    2048+0 records in
    2048+0 records out
    2147483648 bytes (2.1 GB) copied, 23.5908 s, 91.0 MB/s


    Any ideas how the hell brand new hardware is getting a measly 13MB/sec write speed? What can I do to bring this up to something usable for running VMs/etc.? Are my expectations for software RAID in the wrong ballpark? I'd think that modern hardware wouldn't even break a sweat calculating parity at 5x that rate...


    Anyway, I'm not sure what might be wrong, so I'm unsure what to post - please request info if I don't provide it.


    Build:

    • ASrock E3C224D2I server mobo
    • Intel E3-1231v3 Xeon CPU
    • Crucial 16GB ECC (2x8GB) CT2KIT102472BD160B
    • 4x2TB WD Red drives (RAID5 in OMV) connected to the four SATA3 ports
    • 1x320GB Hitachi 2.5" drive for OMV boot/install drive
    • OMV 1.9 fresh install as of last night


    Thanks!
    Mike

    • Offizieller Beitrag

    That system should be able to hit 200 MB/s+ from that array. Try installing the 3.16 backports kernel. There is a button for it in omv-extras.

    omv 7.0.4-2 sandworm | 64 bit | 6.5 proxmox kernel

    plugins :: omvextrasorg 7.0 | kvm 7.0.10 | compose 7.1.2 | k8s 7.0-6 | cputemp 7.0 | mergerfs 7.0.3


    omv-extras.org plugins source code and issue tracker - github


    Please try ctrl-shift-R and read this before posting a question.

    Please put your OMV system details in your signature.
    Please don't PM for support... Too many PMs!

  • Installed backports from omvextras:


    Code
    login as: root
    root@omv's password:
    Linux omv 3.16.0-0.bpo.4-amd64 #1 SMP Debian 3.16.7-ckt2-1~bpo70+1 (2014-12-08) x86_64


    But, performance remains similar:


    Code
    omv ~ $ dd if=/dev/urandom of=./test.dd bs=1048576 count=2048
    2048+0 records in
    2048+0 records out
    2147483648 bytes (2.1 GB) copied, 161.29 s, 13.3 MB/s


    Thanks,
    Mike

    • Offizieller Beitrag

    Do you have ahci enabled in the bios? What is the output of:


    mdadm --detail /dev/md*

    omv 7.0.4-2 sandworm | 64 bit | 6.5 proxmox kernel

    plugins :: omvextrasorg 7.0 | kvm 7.0.10 | compose 7.1.2 | k8s 7.0-6 | cputemp 7.0 | mergerfs 7.0.3


    omv-extras.org plugins source code and issue tracker - github


    Please try ctrl-shift-R and read this before posting a question.

    Please put your OMV system details in your signature.
    Please don't PM for support... Too many PMs!

  • Yep, AHCI is on. Here are the BIOS settings:


    Here is the mdadm output:



    Oh, and I just disabled that aggressive link power management and re-tested with the same results @ ~14.1MB/sec.


    Thanks!
    Mike

    • Offizieller Beitrag

    @subzero79 just pointed out to me that you are running your speed tests on your OS drive not your raid array. You would need to change directory to /media/long_uuid to use the commands you are using. I also have never seen the sata rx value... Can't find info about it.

    omv 7.0.4-2 sandworm | 64 bit | 6.5 proxmox kernel

    plugins :: omvextrasorg 7.0 | kvm 7.0.10 | compose 7.1.2 | k8s 7.0-6 | cputemp 7.0 | mergerfs 7.0.3


    omv-extras.org plugins source code and issue tracker - github


    Please try ctrl-shift-R and read this before posting a question.

    Please put your OMV system details in your signature.
    Please don't PM for support... Too many PMs!

  • When rebooted to look in the BIOS, I had a minor panic attack and thought "Mike, are you running on the 2.5" laptop drive, not the volume??"


    Unfortunately, that is not the case :(


  • OHHHHH! Maybe it's the urandom... Look what I get with /dev/zero instead:



    MUCH BETTER :D


    Thanks for the help, ryecoaaron!!!!


    Now, next project is to investigate why CIFS reads only about 30MB/sec most times, but writing is around 80MB/sec - both of which still seem slow, but at least I know it's not the disk itself!


    Mike

    • Offizieller Beitrag

    I always use /dev/zero but I didn't realize random would be that slow. Glad it is showing much better speeds :)

    omv 7.0.4-2 sandworm | 64 bit | 6.5 proxmox kernel

    plugins :: omvextrasorg 7.0 | kvm 7.0.10 | compose 7.1.2 | k8s 7.0-6 | cputemp 7.0 | mergerfs 7.0.3


    omv-extras.org plugins source code and issue tracker - github


    Please try ctrl-shift-R and read this before posting a question.

    Please put your OMV system details in your signature.
    Please don't PM for support... Too many PMs!

Jetzt mitmachen!

Sie haben noch kein Benutzerkonto auf unserer Seite? Registrieren Sie sich kostenlos und nehmen Sie an unserer Community teil!