mv operation is taking ages (1 hdd)

    • OMV 4.x
    • Resolved
    • mv operation is taking ages (1 hdd)

      Hi,
      while performing mv operation on bunch of files located on one disk, iotop shows:
      Total DISK READ : 19.31 M/s | Total DISK WRITE : 19.46 M/s
      Actual DISK READ: 19.60 M/s | Actual DISK WRITE: 31.34 M/s

      so my guess is, the data is being phisically transferred from one block to another. First time I see this that mv operation is taking ages...

      How can I fix it?

      Thanks and best regards

      ARMBIAN 5.85 stable Debian GNU/Linux 9 (stretch) 4.19.38-sunxi
    • Source Code

      1. df -h
      2. Filesystem Size Used Avail Use% Mounted on
      3. udev 429M 0 429M 0% /dev
      4. tmpfs 100M 5.7M 94M 6% /run
      5. /dev/mmcblk0p2 7.4G 1.4G 5.5G 21% /
      6. tmpfs 496M 0 496M 0% /dev/shm
      7. tmpfs 5.0M 4.0K 5.0M 1% /run/lock
      8. tmpfs 496M 0 496M 0% /sys/fs/cgroup
      9. tmpfs 496M 4.0K 496M 1% /tmp
      10. /dev/mmcblk0p1 58M 29M 27M 52% /boot
      11. /dev/sda1 3.6T 548G 3.1T 15% /srv/dev-disk-by-label-WDRED4TB
      12. folder2ram 496M 5.6M 491M 2% /var/log
      13. armbian-ramlog 50M 0 50M 0% /var/folder2ram/var/log
      14. folder2ram 496M 0 496M 0% /var/tmp
      15. folder2ram 496M 400K 496M 1% /var/lib/openmediavault/rrd
      16. folder2ram 496M 20K 496M 1% /var/spool
      17. folder2ram 496M 13M 483M 3% /var/lib/rrdcached
      18. folder2ram 496M 8.0K 496M 1% /var/lib/monit
      19. folder2ram 496M 4.0K 496M 1% /var/lib/php
      20. folder2ram 496M 4.0K 496M 1% /var/lib/netatalk/CNID
      21. folder2ram 496M 420K 496M 1% /var/cache/samba
      22. shm 64M 0 64M 0% /var/lib/docker/containers/f60be15435c08260244c1370b2bc64c2b8668933224c15fba5703bc0e1e55957/mounts/shm
      23. shm 64M 0 64M 0% /var/lib/docker/containers/fae4c5a5cc07792ee26ea9a25af9714deeb2699988e9a39cfce2c2a964937e0c/mounts/shm
      24. shm 64M 0 64M 0% /var/lib/docker/containers/d7b145e32cb10f95e8e0f50c84c80c223e0fb1e561731d316615336264e26765/mounts/shm
      25. tmpfs 100M 0 100M 0% /run/user/0
      Display All

      Source Code

      1. mv /sharedfolders/downloads/overl* /sharedfolders/Overl/
      /dev/sda1 (/srv/dev-disk-by-label-WDRED4TB) is the external disk attached containing sharedfolders, so one mount only.
    • olo wrote:

      mv /sharedfolders/downloads/overl* /sharedfolders/Overl/
      So please check with grep sharedfolders /etc/mtab whether these are different mounts. Adjusting source and destination and using the paths below /srv/dev-disk-by-label-WDRED4TB might solve the problem.

      BTW: If these are different shares I would never move/copy data locally (but that's one of the many potential problems users will only be able to accept once they're hit by data corruption): Copy files internally via ssh allowed?
    • Source Code

      1. grep sharedfolders /etc/mtab
      2. /dev/sda1 /sharedfolders/Overl ext4 rw,noexec,relatime,jqfmt=vfsv0,usrjquota=aquota.user,grpjquota=aquota.group 0 0
      3. /dev/sda1 /sharedfolders/downloads ext4 rw,noexec,relatime,jqfmt=vfsv0,usrjquota=aquota.user,grpjquota=aquota.group 0 0
      What do you mean by adjusting the source and destination?

      If not cp or mv, than how move files from one share to another? What's your recommendation?
    • replacing with /srv/dev-disk-by-label-WDRED4TB fixed it indeed. Thank you :).
      Is it the way to go, or did I mess somehing during the configuration on the way? On OMV 2.x I remember I could use /media/jsdojjoifjdsiojfssomething/share/... without headache, super fast...

      Thanks for pointing out the issue while performing operations on files disregarding the filesharing deamons. Good to know.
      However, I don't know how it's possible to move data via filesharing deamon (NFS, SAMBA - too slow) nearly as fast as by doing it via ssh...
    • olo wrote:

      However, I don't know how it's possible to move data via filesharing deamon (NFS, SAMBA - too slow) nearly as fast as by doing it via ssh...
      Some filesharing daemons implement 'server side copy' (including moves) but it depends on exact version of Netatalk or Samba, settings and client requirements.

      Anyway: since I help companies from time to time to recover from such issues I would always do such operations from the client and if needed consolidate shares into one so a move is always super fast.
    • tkaiser wrote:

      Some filesharing daemons implement 'server side copy' (including moves) but it depends on exact version of Netatalk or Samba, settings and client requirements
      Wasn't aware of this. I have to dig deeper to find out more. Sounds great and wonders me if it's possible with my config...

      tkaiser wrote:

      always do such operations from the client and if needed consolidate shares into one so a move is always super fast
      And that's the clue. I think, I should print this and put on a wall in my office :)