slow transfer between shares.

    This site uses cookies. By continuing to browse this site, you are agreeing to our Cookie Policy.

    • slow transfer between shares.

      I have a new omv box that I've built. It has an A10-5800K APU and 8 Gigs of RAM. I have 3 seagate 4Tb NAS hard drives, and one WD red 4Tb NAS drive in a raid 5 array.

      When I download a file using transmission it puts it into the downloads folder on my raid array. Then I transfer it into the correct media folder on the same array, using either a windows 7 machine or one of my ubuntu 18.04 machines. The problem is that my transfer speeds while doing this start out at 50MB/s but quickly drop down to 20MB/s (within about 30 seconds). I have even tried transferring the files with midnight commander through SSH with the same result. Does anyone have any idea why this might be happening? Could it by caused by my raid settings? or by my SMB settings? Lots of fruitless googling has been done the last few days and I am stumped. Any suggestions would be appreciated.

      edit: I have thought of a few more details. On all disks: I have my disk properties set at 127-intermediate power usage with standby. and write cache is enabled.

      This also appears to only happen when transferring between different shares. If I transfer a file inside the same share it does it almost instantly.
    • blu64 wrote:

      The problem is that my transfer speeds while doing this start out at 50MB/s but quickly drop down to 20MB/s (within about 30 seconds)
      This is an indication that your network setup has a problem (just 50 MB/s) as well as your storage (20 MB/s once filesystem cache is full -- but with 8 GB RAM this should happen way later than 'quickly').

      You need to test for network and storage performance individually to identify the bottleneck(s). I would use iperf/iperf3 and iozone for this but most probably these tools are not installed by default on OMV on x86 and if you're a Linux novice it might be challenging. Hope others come up with different ideas.

      blu64 wrote:

      If I transfer a file inside the same share it does it almost instantly.

      This is just a move inside the share. If you copy between partitions it's always 'copy and delete afterwards'. Something entirely different.
    • blu64 wrote:

      The problem is that my transfer speeds while doing this start out at 50MB/s but quickly drop down to 20MB/s (within about 30 seconds).
      I have the same problem, but my hardware is a lot older than yours, if I transfer from W10 to the server it's fine, if I move a file from 1 share to another it's slow...and if it's a large file it's like watching paint dry :)

      There are a number of reasons this could be happening, like you I have searched and found nothing definitive, in other words one solution solves all.

      You're using Seagate and WD so am I, the Seagate's spin speed is 7200 the WD will mostly likely be 5400, according to what I have found you can mix drives but there is a performance loss and the array works to the slowest drive, something to do with I/O.

      Raid 5, which I use, also not the best solution, Raid 10 however would improve write speed but you lose on space this site is useful

      Onboard raid controller may be another problem, 'sometimes' independent controllers can improve how the raid performs, I know our last school server used an adaptec raid card to bypass the onboard raid controller.

      TBH I am now considering moving away from Raid for home use and looking at using mergefs and snapraid and setting up 2 backups using either rsync or rsnapshot, I already have an rsync running on Pi, performance wise it's crap, but it does a specific job and nothing else and I can set it to run overnight.

      I have also been considering a HP G7 or G8 or a HC2 as an alternative to this beast that sits under my desk.
      Raid is not a backup! Would you go skydiving without a parachute?
    • tkaiser wrote:

      This is an indication that your network setup has a problem (just 50 MB/s) as well as your storage (20 MB/s once filesystem cache is full -- but with 8 GB RAM this should happen way later than 'quickly').
      I don't understand how it could be a network problem if I am transferring files between two shares on the same machine. Yesterday I hooked up a monitor and keyboard to the machine and used midnight commander to copy a file from one share to another and I had the same problem.
    • blu64 wrote:

      I don't understand how it could be a network problem if I am transferring files between two shares on the same machine. Yesterday I hooked up a monitor and keyboard to the machine and used midnight commander to copy a file from one share to another and I had the same problem.
      Try to do the same just directly in /srv/ without touching /sharedfolders
      I'm not an expert. I'm just a tourist here.

      - ODROID-HC1 (Samsung Exynos5422, 2GB LPDDR3 RAM, Gigabit Ethernet)
      - Sandisk Ultra 16GB A1 "SDSQUAR-016G-GN6" (btrfs)
      - Samsung SpinPoint M8 1TB (ext4)
      - OMV 4.1.11 / ARMBIAN 5.60 + Kernel 4.14.69
    • The problem with the mv file speed when dealing with a mount folder is nothing new.
      I'm not an expert. I'm just a tourist here.

      - ODROID-HC1 (Samsung Exynos5422, 2GB LPDDR3 RAM, Gigabit Ethernet)
      - Sandisk Ultra 16GB A1 "SDSQUAR-016G-GN6" (btrfs)
      - Samsung SpinPoint M8 1TB (ext4)
      - OMV 4.1.11 / ARMBIAN 5.60 + Kernel 4.14.69
    • You can possibly create one share and the appropriate symlinks in it. This is not the best solution but ... it works.
      I'm not an expert. I'm just a tourist here.

      - ODROID-HC1 (Samsung Exynos5422, 2GB LPDDR3 RAM, Gigabit Ethernet)
      - Sandisk Ultra 16GB A1 "SDSQUAR-016G-GN6" (btrfs)
      - Samsung SpinPoint M8 1TB (ext4)
      - OMV 4.1.11 / ARMBIAN 5.60 + Kernel 4.14.69
    • blu64 wrote:

      I don't understand how it could be a network problem if I am transferring files between two shares on the same machine.

      Well, then it's just your local storage that is way too slow. As already suggested: test individually. I would suggest iozone but unfortunately iozone3 package is not installed on the x86 OMV images for whatever reasons. The 'non-free' Debian repo would've to be added to sources.list since otherwise you won't be able to install the iozone3 package.

      On the OMV ARM images it looks like this:

      Source Code

      1. root@espressobin:~# cat /etc/apt/sources.list
      2. deb http://httpredir.debian.org/debian stretch main contrib non-free
      3. deb-src http://httpredir.debian.org/debian stretch main contrib non-free
      4. deb http://httpredir.debian.org/debian stretch-updates main contrib non-free
      5. deb-src http://httpredir.debian.org/debian stretch-updates main contrib non-free
      6. deb http://security.debian.org/ stretch/updates main contrib non-free
      7. deb-src http://security.debian.org/ stretch/updates main contrib non-free