Slow local transfer speeds

    I currently have OMV setup as a VM in proxmox. All drives are setup as VirtIO SCSI Single in proxmox.

    I get absolutely abysmal transfer speeds when transferring files locally from share to share (export folder to export folder). It will usually start at around 40mb/s and then gradually slow to 6-8mb/s and stay there. This obviously makes moving large amounts of data a real drag. This also happens when transferring between two shares that are on the same physical HDD.


    All drives previously worked fine on windows. They are all healthy according to SMART.

    The only thing I can see is that I get massive wait-io spikes in the OMV monitor.


    I would love some help with this because I have no idea why it is happening.


    Thanks

  • crashtest

    Approved the thread.

    There are potentially a few variables that could be contributing to this. Are you using CEPH? What overall system resources have you allocated to the VM? What other VMs are present on the host? Please provide more information on your setup.

    Thanks :)


    Sorry for the lack of info.


    No CEPH.

    It currently has 2 cores and 4GB RAM allocated (writing this now I realise this may be the cause of my issue)

    All the drives are formatted at EXT4 in proxmox.

    The only other VM is running 4 cores and 8GB Ram running Ubuntu server with docker containers setup for an arr stack, navidrome, bookorbit, seerr, etc.

    I've left 2 cores and 2GB for Proxmox.

    The resources allocated in the OMV VM aren't amazing, but not necessarily the root cause for slow transfer speeds. After re-reading your post I want to make sure that I fully understand what exactly you're attempting. You say that you're transferring from "export folder to export folder". Is this from one OMV folder to another OMV folder? This will likely cut the speeds in half right off the bat simply because the network interface is both sending and receiving files. What transfer protocol are you using?


    If you're moving the assets from one folder to another on OMV itself, the traditional Linux move command will be faster than network transfers.

    Code
    mv path/to/dir1 path/to/dir2


    Have you tried transferring files on the Ubuntu VM with similar slow speeds, or is this issue limited to the OMV VM?

    Yeah I'm talking about moving files within OMV itself through the terminal. I have been using Midnight Commander in the terminal and it will start with a burst of 50+ mb/s before dropping sometimes to 6-7. Weirdly, the latest batch of files I have transferred hovered between 20-30 the whole time without me changing anything.


    Also, weirdly, I transferred two isos from OMV to my windows PC last night over SMB and it sat on 100mb/s the whole time. So that makes me think it's an issue with reading and writing on the same disk at the same time.


    Across NFS to the Ubuntu VM, it seems to be roughly the same. From two shares on the same physical drive, it gets between 12-20mb/s, between drives it seems to max out at 12. This is all over 1gbps Ethernet.

    Quote from crmpf

    I have been using Midnight Commander in the terminal and it will start with a burst of 50+ mb/s before dropping sometimes to 6-7.

    That's horrifying. I'm using ancient JMB58x and ASM1166 SATA controllers and notoriously super slow Samsung SSD 870 QVO sata drives. They can still saturate the (2.5) ethernet connection at ~2.8Gib/s for a TB data transfer.


    This has happened to me, so not patronizing: Have you made sure your wiring doesn't have kinks and the connectors are undamaged? I had (invisible) corrosion on connectors before, un/replugging the connectors a couple of times fixed it for me. Power off for that, of course, if you manipulate the hard drive plugs.

    Quote from burnbasket

    That's horrifying. I'm using ancient JMB58x and ASM1166 SATA controllers and notoriously super slow Samsung SSD 870 QVO sata drives. They can still saturate the (2.5) ethernet connection at ~2.8Gib/s for a TB data transfer.


    This has happened to me, so not patronizing: Have you made sure your wiring doesn't have kinks and the connectors are undamaged? I had (invisible) corrosion on connectors before, un/replugging the connectors a couple of times fixed it for me. Power off for that, of course, if you manipulate the hard drive plugs.

    That's what confusing me so much. If it was busted cables or corrosion, it should be slow everywhere. But I'm currently copying using MC on omv from two folders on different HDDs and it is going at 50-70. Plus copying over SMB was hitting 100 stable. I'll post down and check all my connections to see, maybe I'll just get new data cables for all my drives, just to eliminate that as an issue.


    It seems like there's either an issue with that drive moving files within itself, some omv weirdness when copying files between shares on the same drive, or some proxmox setting that's give weird.

    If there is one individual disk that is slow: Make a backup immediately.

    This disk might be failing or already partially failed.

    When the backup is done, try to ascertain it is still in perfect working order ( assuming the best but expecting the worst, right :) )

    See if the disk already has recorded imperfections:
    sudo smartctl -a /dev/<diskName>

    Run a full check (after the backup - it might fail completely during backup - plus the check puts significant strain on the disk and it might fail there)

    sudo smartctl -t long /dev/<diskName>

    See the results, again with

    sudo smartctl -a /dev/<diskName>

    This won't (and can't) fix anything that has already gone bad - but you'll be aware of it.

    For reference of the SMART errors that are indicative of unfixable hardware problems, see

    Post
    RE: Problem with a single HDD.
    SMART (prefailure) stat's to look at:

    SMART 5 – Reallocated_Sector_Count.
    SMART 187 – Reported_Uncorrectable_Errors.
    SMART 188 – Command_Timeout.
    SMART 197 – Current_Pending_Sector_Count.
    SMART 198 – Offline_Uncorrectable.

    If any of the above raw counts are greater than "0", it's time to take note. If they're incrementing...
    ___________________________________

    The following may be cable related but it can be an issue with the drive controller or a mobo port.
    SMART 199 - UltraDMA CRC errors
    crashtest

Participate now!

Don’t have an account yet? Register yourself now and be a part of our community!

Register Yourself Login