Very slow file copying

  • I had to rebuild my OMV NAS and while doing that - I re-initiated the storage as ZFS. I re-created all my old SMB shares and am in the process of copying everything back over from a Nas4Free box (I use 2 different installs for testing). I use Allway Sync to copy from the Nas4Free to OMV using mapped drive letters from a windows 10 box.


    The process is painfully slow. My wife wrote a book on wildflowers and has 363,623 pictures (mostly .jpg, some raw) and it takes about a minute to copy 3-4 pictures over. - at this rate it is going to take forever..


    I have both boxes connected to a 1-Gig switch


    Any thoughts as to why it is so slow (I did read the SMB thread about tuning - but it did not help)


    TIA


    -George-

    • Offizieller Beitrag

    This is a common problem when copying many small files. Start/stop transfers of many files over a network connection may be painfully slow. Instead of copying a lot of small files, try copying fewer big files. And see if that helps. You can do this by creating file archives locally on the source computer and then copy over the archives. And then extract the files locally on the destination computer. If it is jpg or mp4 or mp3, then there is little point in compressing. Just create uncompressed archives, that is faster.


    Or simply temporarily move the HDDs involved to the same computer copy over SATA locally and avoid all of the network bottleneck. This is most likely the easiest and fastest method.


    Or use netcat or SSH for fast network transfer of many small files, without the painful start/stop overhead. An automatic variant of creating file archives and transferring fewer bigger files instead of many small.


    See: https://makezine.com/2008/11/1…tip-superfast-network-fi/


    Also make sure not to use a "middle man" transfer. It seems that you do the copying from a windows computer that is connected to both NASes. That greatly increase the overhead and slows down the transfer. Instead copy using console commands from one NAS directly to the other.

    Be smart - be lazy. Clone your rootfs.
    OMV 5: 9 x Odroid HC2 + 1 x Odroid HC1 + 1 x Raspberry Pi 4

    Einmal editiert, zuletzt von Adoby ()

  • Thank you so much for the suggestion - had never used rsync.


    Set it up as a server on Nas4Free (with modules for each share)


    Set up as a remote pull on OMV


    It positively flies -


    -george-

Jetzt mitmachen!

Sie haben noch kein Benutzerkonto auf unserer Seite? Registrieren Sie sich kostenlos und nehmen Sie an unserer Community teil!