Very slow file copying

    • OMV 4.x
    • Very slow file copying

      I had to rebuild my OMV NAS and while doing that - I re-initiated the storage as ZFS. I re-created all my old SMB shares and am in the process of copying everything back over from a Nas4Free box (I use 2 different installs for testing). I use Allway Sync to copy from the Nas4Free to OMV using mapped drive letters from a windows 10 box.

      The process is painfully slow. My wife wrote a book on wildflowers and has 363,623 pictures (mostly .jpg, some raw) and it takes about a minute to copy 3-4 pictures over. - at this rate it is going to take forever..

      I have both boxes connected to a 1-Gig switch

      Any thoughts as to why it is so slow (I did read the SMB thread about tuning - but it did not help)

      TIA

      -George-

      The post was edited 1 time, last by ggibeau ().

    • This is a common problem when copying many small files. Start/stop transfers of many files over a network connection may be painfully slow. Instead of copying a lot of small files, try copying fewer big files. And see if that helps. You can do this by creating file archives locally on the source computer and then copy over the archives. And then extract the files locally on the destination computer. If it is jpg or mp4 or mp3, then there is little point in compressing. Just create uncompressed archives, that is faster.

      Or simply temporarily move the HDDs involved to the same computer copy over SATA locally and avoid all of the network bottleneck. This is most likely the easiest and fastest method.

      Or use netcat or SSH for fast network transfer of many small files, without the painful start/stop overhead. An automatic variant of creating file archives and transferring fewer bigger files instead of many small.

      See: makezine.com/2008/11/14/linux-tip-superfast-network-fi/

      Also make sure not to use a "middle man" transfer. It seems that you do the copying from a windows computer that is connected to both NASes. That greatly increase the overhead and slows down the transfer. Instead copy using console commands from one NAS directly to the other.
      OMV 4, 5 x ODROID HC2, 2 x 12TB, 2 x 8 TB, 1 x 500 GB SSD, GbE, WiFi mesh

      The post was edited 1 time, last by Adoby ().