HC2 internal file transfers going through LAN :-(

    • HC2 internal file transfers going through LAN :-(

      Hi fellow OMVers.

      I seem to be having problems moving files from one location on disk to another without going through a local machine.
      I'm primarily using a linux laptop and have tried SFTP (through Nemo) and FileZilla. I've also used the command line through bash, and Midnight Commander.

      Basically it seems that no Cut / paste type operations (moving a file from one location to another) are happening instantly like on my previous Linux Server. Using MC I am seeing move speeds of 18-24MB/s. This is not across a network, this is from one folder on the ntfs disk to another. And these tests have been on single files varying from 850Mb to 2Gb.

      Why is internal transferring files so slow? But more to the point, what can I do to increase the speed of internal transfers, and what is good software to handle it remotely (rather than SSHing in and running mc)? Bear in mind I'm on the Arm platform with the HC2.

      Thanks for any suggestions you might be able to provide...
    • Odroid HC2 - armbian - Seagate ST4000DM004 - OMV4.x
      Asrock Q1900DC-ITX - 16GB - 2x Seagate ST3000VN000 - Intenso SSD 120GB - OMV4.x
      :!: Backup - Solutions to common problems - OMV setup videos - OMV4 Documentation - user guide :!:
    • Thanks.

      Unfortunately the Cloud Commander referenced above doesn't work on the HC arm-based processor. I did find a Pi version which 'sort of' worked, but also would crash. So I quit using it and uninstalled it. If you know of a specific one for Arm / Odroid then please do let me know.

      Wetty looks like a replacement for Shellinabox. That might be handy since Shellinabox fails to install correctly on my HC2. But again I'd need to find an Arm version of it for the Docker.

      Using MC, what sort of speeds do you get on a file move? Are you running ntfs or another filesystem?

      Right now I'm doing a large transfer (155Gb) and the average read speed across the gigabit network is 54MB/s. I was hoping for higher. Are there some tweaks I can make to improve performance?
      My write speeds are an abysmal 21-24MB/s. After performing my edits I think I'll have to remove the drive from the HC2 and hook it up to a SATA on my desktop to transfer the files back, or leave it to run overnight (defeating the object of having a low power NAS solution!).

      Surely the HC2 is much more capable than that... I must be missing something.

      The post was edited 1 time, last by OMV_HC2 ().

    • You have to move files within the same filesystem in order to get "instant" transfers. It is not enough that it is partitions on the same drive or even that it is the same partition, unless you also use the partition from the same mount point.

      This is the main reason why I prefer to have one big shared folder per drive. And subfolders inside that. When I "reorganize" I get instant transfers using mc when moving in the same filesystem. No problems.

      I connect to the OMV NAS using SSH and start mc there. And it is really fast! Local transfers! Also when copying files. Connecting from my laptop with nfs I also get instant moves within the same share. But not at all as fast as locally when I copy files.

      It may be different with (s)ftp? Using ftp in mc you may indeed transfer files on the same filesystem twice over the network via the computer running the ftp software. Don't do that... Also using NTFS might cause problems, but most likely not. I use ext4, SSH and nfs everywhere. Also CIFS for android clients.

      In some cases if you move files with mc and the destination already exists, then mc may resort to copy/delete when overwriting existing files. Also if it is many files, mc usually enumerates the files before moving them. That can take a while if a lot of files are to be moved. Then a plain mv may be faster.

      When I do massive file transfers (multi TB) between different filesystems or different computers, I typically use SSH, nfs, rsync and screen. I login on the destination HC2 using SSH. Start screen and launch a rsync copy from a nfs-share. I typically saturate the GbE, I belive. Then I can close the console and the transfer continues. And should there be any problem rsync can resume without having to copy already copied files. I run backups between my HC2s in a similar fashion, but from cron jobs that automatically launch rsync snapshot scripts.
      OMV 4, 5 x ODROID HC2, 2 x 12TB, 2 x 8 TB, 1 x 500 GB SSD, GbE, WiFi mesh

      The post was edited 13 times, last by Adoby ().

    • Adoby wrote:

      You have to move files within the same filesystem in order to get "instant" transfers. It is not enough that it is partitions on the same drive or even that it is the same partition, unless you also use the partition from the same mount point.
      This is the way I have it, and why I can't understand it being slow. One 10Tb disk with 8 folders(some with a couple of sub-directories). In example below I am logged in via SSH and running MC.

      Drive :
      Downloads
      |-Sub folder 1
      |-Sub sub folder A
      |-Sub folder 2
      |-Sub folder 3
      |-Sub sub folder B
      Share 2
      Share 3
      Share 4
      Share 5

      Move File A from Downloads to Downloads/Sub folder 1 = instant
      Move File A from Sub folder 1 to Downloads = instant

      Move File A from Downloads/Sub folder 1 to Share 4 = slow (about 18MB/s)
      Move File A from Downloads to Share 3 = slow (about 18MB/s)

      So moving any file (or group of files) from the Downloads folder to the correct Shared Folder takes an age.

      On my previous Ubuntu server (different hardware) any transfer within a disk was instant, regardless of which Shared Folder or sub-folder it was located in.

      After reading your post, I tried a test using File B (3.7Gb).

      Move File B from Downloads/Sub folder 1 to Downloads/Sub folder 2 = instant
      Move File B from Downloads/Sub folder 2 to Downloads = instant
      Move File B from Downloads to Downloads/Sub folder 1/Sub sub folder A = instant
      Move File B from Downloads/Sub folder 3/Sub Sub folder B to Downloads/Sub folder 1/Sub sub folder A = instant

      Interestingly, connecting with Nemo via SFTP or SMB allows the same instantaneous transfers described above, so no specific need to log in via SSH and run MC. Happy days!

      Now I see why you have one Master Shared Folder and then sub folders within it. I think I am going to have to reorganise my drive in the same way. But first I'll have to experiment with some software that connects to said folders to ensure it can handle everything being in one place rather than separate shares.

      I will lose the ability to easily 'hide' certain sub-folders from network users if they all fall within the same shared folder. For example I only want one user to have read/write access to the photo backup folder, but all users get read access. Perhaps it is best just to keep these folders as separate shares? Some further experimentation to be performed.

      I wonder why OMV can't handle the instantaneous transfers across shares, when on the same partition of a disk? I assume this isn't just an ntfs thing since you mentioned you're running ext4.
    • The different shares are, as far as the remote computer is concerned, different filesystems. This is not something OMV-specific. This is how Linux filesystems works.

      The instantaneous "transfers" are achieved by renaming the filename or path of a file/folder. This only works if the source path and the destination path are on the same filesystem.

      Try to create a main share, only for you, that includes all sub folders. Use that share to move stuff around. And create other shares that only include individual sub folders, with limited rights, perhaps for specific clients/users. ( I haven't tried it and I'm not at home, but I assume that OMV allows this.)

      The best of two worlds.
      OMV 4, 5 x ODROID HC2, 2 x 12TB, 2 x 8 TB, 1 x 500 GB SSD, GbE, WiFi mesh
    • Adoby wrote:

      Try to create a main share, only for you, that includes all sub folders. Use that share to move stuff around. And create other shares that only include individual sub folders, with limited rights, perhaps for specific clients/users. ( I haven't tried it and I'm not at home, but I assume that OMV allows this.)
      OK, I'll give this a shot later. I didn't think I'd be able to limit sub folders to specific users within a shared folder. If I could, that would be great.

      Did you do any tweaks to increase network performance on your HC2s?
      I'm getting read speeds across the gigabit network of 50MB/s and write of only around 20MB/s to my shares. On occasions I've seen it go up to 80MB/s on the read side (eg after first install).

      I'm in the same room, connected to the same gigabit router and using Cat6 cables.
      I made a couple of config tweaks (but not the CPU governor one). I'm going to make a couple of changes to the setup later today since I've just added a new switch to see if that makes a difference.
    • I use nfs. And I have increased the nfs read and write buffers to 2^17. Otherwise no tweaks. Except a slight underclocking and the conservative frequency scaling on most of the HC2s, to reduce the heat up. Still full speed network transfers .

      I have only Linux and Android clients. No Windows. And I have 25 cm long CAT6 cables to my HC2s from the switch. I haven't really measured speeds, but I assume they are close to saturating the GbE. When I initially set the HC2s up, and set up shares and backups, that was about what I got both reading and writing between HC2s. Typically my HC2s run daily backups unattended in the early morning hours. Between 04:00 and 05:00. I don't use OMV for the backups. I just run simple rsync snapshot scripts using crontab entries reading from remote OMV nfs shares mounted using autofs.

      I access the media on the HC2s using only wifi. Using a 3 unit ASUS Lyra 5GHz mesh. And depending on what I do and what clients are involved where in my house, I get typical speeds over wifi between 60MiBps and 25MiBps. (Yes, bytes,per second not bits!) More than fast enough for me. More than fast enough to not need any transcoding at all. I just have Emby stream the original media and let the clients scale as needed. I backup stuff on my laptop and my PC to the NASes using rsync snapshots over nfs. That is very fast since only changes since last time have to be written over wifi.
      OMV 4, 5 x ODROID HC2, 2 x 12TB, 2 x 8 TB, 1 x 500 GB SSD, GbE, WiFi mesh

      The post was edited 1 time, last by Adoby: Changed wrong GiBps to correct MiBps. ().

    • Adoby wrote:

      I get typical speeds over wifi between 60GiBps and 25GiBps. (Yes, bytes,per second not bits!)
      How do you get 25-60 gigabytes per second over a 1 gigabit per second network link on the HC2? I assume you meant megabytes per second? :)
      omv 4.1.13 arrakis | 64 bit | 4.15 proxmox kernel | omvextrasorg 4.1.13
      omv-extras.org plugins source code and issue tracker - github

      Please read this before posting a question and this and this for docker questions.
      Please don't PM for support... Too many PMs!
    • Adoby wrote:

      So I screwed up elsewhere. Oh, well....
      Just checking. That is very good for wireless.
      omv 4.1.13 arrakis | 64 bit | 4.15 proxmox kernel | omvextrasorg 4.1.13
      omv-extras.org plugins source code and issue tracker - github

      Please read this before posting a question and this and this for docker questions.
      Please don't PM for support... Too many PMs!
    • ryecoaaron wrote:

      Adoby wrote:

      So I screwed up elsewhere. Oh, well....
      Just checking. That is very good for wireless.
      Yes, I'm well pleased by it. It seems the Lyra can use two 5GHz channels to transfer data in parallel between two mesh nodes. And I have my PC plugged in directly to one of the mesh nodes, in a room next to the one that is attached to my NAS switch. With only an open door between them. Gives me close enough to GbE speeds to the PC for me to skip cables between the NAS switch and the PC.
      OMV 4, 5 x ODROID HC2, 2 x 12TB, 2 x 8 TB, 1 x 500 GB SSD, GbE, WiFi mesh
    • OMV_HC2 wrote:

      Did you do any tweaks to increase network performance on your HC2s?
      I'm getting read speeds across the gigabit network of 50MB/s and write of only around 20MB/s to my shares. On occasions I've seen it go up to 80MB/s on the read side (eg after first install).
      Have just done some more performance testing and am seeing some erratic results.

      When reading from the drive, the transfer speed is going up and down like crazy. Sometimes as low as 11MB/s, sometimes as high as 87MB/s, It fluctuates a lot while copying a 3.7Gb ISO file.

      So I have looked at the SMART report and am seeing Raw Read Error Rate in Prefail (Raw value = 0) and Spin Up Time is also showing Prefail (Raw Value 433, av 444).

      Does this mean my issue is the disk itself, and not something to do with the HC2?
      In which case I guess I might need to contact Hitachi!

      OK - just remvoed the disk from the HC2 and plugged it into my USB3 enclosure.
      Read speed is coming in at 165MB/s for a 2.2GB file and 167 MB/s for my 3.7Gb ISO.
      Write speed sitting at 162MB/s.
      Manufacturers Disk utility passing drive, as is CrystalDiskInfo 7.

      Looks like it is not the drive having issues (phew!), unless it is not getting enough power from the HC2 power supply? Or I am back to the problem being with the OMV4 config...

      The post was edited 2 times, last by OMV_HC2 ().

    • I've updated the SATA on my HC2s. Worth a try? (I've also set spindown to 30 minutes.)

      wiki.odroid.com/odroid-xu4/software/jms578_fw_update

      Also I had bad problems with my HDD (dropped them and wouldn't mount) if I changed any settings for Physical Disk Properties. It seems my drives are incompatible with hdparm. Try to disable any HDD tweaks. (Disk->Edit)

      Or it might be power.
      OMV 4, 5 x ODROID HC2, 2 x 12TB, 2 x 8 TB, 1 x 500 GB SSD, GbE, WiFi mesh
    • Adoby wrote:

      I've updated the SATA on my HC2s. Worth a try? (I've also set spindown to 30 minutes.)

      wiki.odroid.com/odroid-xu4/software/jms578_fw_update

      Also I had bad problems with my HDD (dropped them and wouldn't mount) if I changed any settings for Physical Disk Properties. It seems my drives are incompatible with hdparm. Try to disable any HDD tweaks. (Disk->Edit)
      Hmm, I might need to do this. Always good to be running latest firmware anyway. Last night when I went to bed the HDD hadn't spun down, and this morning when I woke up I can still hear it chuntering away which is unusual. It was typically spinning down after 2 or 3 minutes. My guess is it hasn't all night, but I don't know why.

      The only thing I changed was the location of the folders so the main shares are now in 1 master folder for easy file manipulation.

      Ah - just got the drive to spin down : the issue was Deluge within Docker. It wasn't downloading anything, but I needed to reset the mount point of the downloads folder since I moved it last night. It was probably repeatedly searching for the directory. I'll see if it spins down again after running a couple of tests on the new directory.

      I've downloaded & extracted the FW update file linked above (thank you). However it looks like I'm already running the latest version.

      sudo ./JMS578FwUpdate -d /dev/sda -v reports
      Bridge Firmware Version: v173.1.0.1

      Had nothing set in Disk Management, but have now enabled and set a 10 minutes spin down time to see if that works.

      After changing directory structure last night, reads across the network were higher : 85 to 100 MB/s. Writes were still low (23 MB/s). Will test again later today to see if the speeds have remained high on the read side at least.
    • Yes, using ntfs at the moment. Unfortunately I don't have another 10TB drive, or I would give the ext4 a shot and see how it does. Having just moved Country I also don't have any spare drives around that I can reformat as ext4 and test speeds with, Just couldn't bring all the computer gear with me :(

      I'm sure people are generally getting higher than 23 MB/s write on ntfs with SMB on the HC2 though.

      Just tested my speeds again. Today read speed is fluctuating from only 23 MB/s up when Deluge was active, up to 44 MB/s with the Docker Container stopped. That was on a 850MB ISO. My 3.7Gb test file transferred at an average of 38 MB/s. With each transfer I could see the speed fluctuating quite a bit.

      Obviously there's something I'm missing here. I guess the next thing is to try a beefier PSU, but unfortunately don't have anything more than 2amp here.
    • OMV_HC2 wrote:

      I'm sure people are generally getting higher than 23 MB/s write on ntfs with SMB on the HC2 though
      I doubt that it is possible to get the same speed as with ext4 and nfs. And that is what you are experiencing.

      You should be able to all but saturate GbE with sustained sequential reads and writes to/from the HC2. Random reads/writes may be another story, depending on the HDD.

      You could perhaps shrink the partition on the drive to create some free space on the drive, and then add another ext4 partition. I would suggest you do that before you buy a new PSU. But perhaps not before you go out and buy a backup HDD. But then you'd have an extra HDD to experiment with...
      OMV 4, 5 x ODROID HC2, 2 x 12TB, 2 x 8 TB, 1 x 500 GB SSD, GbE, WiFi mesh

      The post was edited 1 time, last by Adoby ().

    • OMV_HC2 wrote:

      I'm sure people are generally getting higher than 23 MB/s write on ntfs with SMB on the HC2 though.
      What kind of speeds do you get when you:

      dd if=/dev/zero of=/path/to/ntfs/filesystem/somewhere/testfile.dd bs=1M count=5000 conv=fdatasync status=progress && sync
      omv 4.1.13 arrakis | 64 bit | 4.15 proxmox kernel | omvextrasorg 4.1.13
      omv-extras.org plugins source code and issue tracker - github

      Please read this before posting a question and this and this for docker questions.
      Please don't PM for support... Too many PMs!
    • ryecoaaron wrote:

      What kind of speeds do you get when you:
      dd if=/dev/zero of=/path/to/ntfs/filesystem/somewhere/testfile.dd bs=1M count=5000 conv=fdatasync status=progress && sync
      1563426816 bytes (1.6 GB, 1.5 GiB) copied, 66.0285 s, 23.7 MB/s
      1508+0 records in
      1508+0 records out
      1581252608 bytes (1.6 GB, 1.5 GiB) copied, 66.7813 s, 23.7 MB/s

      What is that telling me? The native write speed to the drive?

      The post was edited 1 time, last by OMV_HC2 ().