Rsync error on external usb disk

    • OMV 3.x
    • Rsync error on external usb disk

      Hi, I have a NAS with OMV 3.(Debian Stretch) with two HD configured as RAID1 and an external usb 3.0 HD for backup.
      Dmesg lists the following error when I try to rsync files on that disk

      [ 306.164222] xhci_hcd 0000:00:14.0: ERROR Transfer event for disabled endpoint or incorrect stream ring
      [ 306.164481] xhci_hcd 0000:00:14.0: @0000000175a67ab0 00000000 00000000 04000000 04038001


      and the rsync process hangs. I cannot understand the error, can anyone help me?
      Thank you!
      kuks
    • This is the ouput:

      Source Code

      1. root@nas:~# uname -a
      2. Linux nas 4.9.0-0.bpo.6-amd64 #1 SMP Debian 4.9.88-1+deb9u1~bpo8+1 (2018-05-13) x86_64 GNU/Linux
      3. root@nas:~# lsusb
      4. Bus 002 Device 003: ID 0bc2:2312 Seagate RSS LLC
      5. Bus 002 Device 002: ID 174c:3074 ASMedia Technology Inc. ASM1074 SuperSpeed hub
      6. Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
      7. Bus 001 Device 003: ID 05e3:0608 Genesys Logic, Inc. Hub
      8. Bus 001 Device 002: ID 174c:2074 ASMedia Technology Inc. ASM1074 High-Speed hub
      9. Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
      10. root@nas:~# lsusb -t
      11. /: Bus 02.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/6p, 5000M
      12. |__ Port 1: Dev 2, If 0, Class=Hub, Driver=hub/4p, 5000M
      13. |__ Port 3: Dev 3, If 0, Class=Mass Storage, Driver=uas, 5000M
      14. /: Bus 01.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/7p, 480M
      15. |__ Port 1: Dev 2, If 0, Class=Hub, Driver=hub/4p, 480M
      16. |__ Port 5: Dev 3, If 0, Class=Hub, Driver=hub/4p, 480M
      Display All
    • Source Code

      1. Bus 002 Device 003: ID 0bc2:2312 Seagate RSS LLC
      2. ...
      3. |__ Port 3: Dev 3, If 0, Class=Mass Storage, Driver=uas, 5000M

      As expected: crappy external Seagate USB3 disk with UAS. All those Seagate drives ship with a broken firmware so they run into trouble with UAS (USB Attached SCSI).

      Two options:

      • do some research to 'UAS blacklist' your specific disk on Debian (search hits for Ubuntu are appropriate as well -- simply do a web search)
      • upgrade to at least kernel 4.15 since with this version the following patch has been applied to the kernel that should fix the issue: patchwork.kernel.org/patch/10058075/

      The post was edited 1 time, last by tkaiser: It's kernel 4.15 and not 4.14: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1584557/comments/26 ().

    • tkaiser wrote:

      do some research to 'UAS blacklist' your specific disk on Debian (search hits for Ubuntu are appropriate as well -- simply do a web search)
      Without the uas module I cannot mount the disk. Even the device name (sdd in my case) has disappeared from /dev directory...
      Did I do any mistake??


      tkaiser wrote:

      upgrade to at least kernel 4.14 since IIRC with this version the following patch has been applied to the kernel that should fix the issue: patchwork.kernel.org/patch/10058075/
      Sorry for my question, but is it safe (and is it possible) to upgrade manually from a 4.9.0-0.bpo.6-amd64 kernel to a newer (bpo??) kernel? From a quick attempt apt-get refuses to upgrade to 4.17.0-0.bpo.3-amd64 (broken dependence with linux-compiler).
      Thank you.
    • Blacklisting as you said gives me this output (after rebooting)

      Source Code

      1. root@nas:# lsusb
      2. Bus 002 Device 004: ID 0bc2:2312 Seagate RSS LLC
      3. Bus 002 Device 002: ID 174c:3074 ASMedia Technology Inc. ASM1074 SuperSpeed hub
      4. Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
      5. Bus 001 Device 003: ID 05e3:0608 Genesys Logic, Inc. Hub
      6. Bus 001 Device 002: ID 174c:2074 ASMedia Technology Inc. ASM1074 High-Speed hub
      7. Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
      8. root@nas:# lsusb -t
      9. /: Bus 02.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/6p, 5000M
      10. |__ Port 1: Dev 2, If 0, Class=Hub, Driver=hub/4p, 5000M
      11. |__ Port 2: Dev 4, If 0, Class=Mass Storage, Driver=, 5000M
      12. /: Bus 01.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/7p, 480M
      13. |__ Port 1: Dev 2, If 0, Class=Hub, Driver=hub/4p, 480M
      14. |__ Port 5: Dev 3, If 0, Class=Hub, Driver=hub/4p, 480M
      Display All
      therefore it seems that NO driver is loaded for the Seagate external HD, isn't it? I obtain a similar output connecting the hd to an usb 2.0 port
    • kuks wrote:

      |__ Port 2: Dev 4, If 0, Class=Mass Storage, Driver=, 5000M
      Yep, this looks strange (should read Driver=usb-storage instead). You still need to blacklist the device and not the module.

      Sorry, too tired, so just a link: forum.armbian.com/topic/7126-n…usb-disk-rsync-io-errors/

      (forget about /boot/armbianEnv.txt since you use OMV on x86 -- on ARM all of these problems are already long solved but on x86 you need to find a solution yourself)
    • tkaiser wrote:

      You still need to blacklist the device and not the module.
      Now my blacklist file is as follows:

      Source Code

      1. root@nas:~# lsusb
      2. Bus 002 Device 003: ID 0bc2:2312 Seagate RSS LLC
      3. Bus 002 Device 002: ID 174c:3074 ASMedia Technology Inc. ASM1074 SuperSpeed hub
      4. Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
      5. Bus 001 Device 003: ID 05e3:0608 Genesys Logic, Inc. Hub
      6. Bus 001 Device 002: ID 174c:2074 ASMedia Technology Inc. ASM1074 High-Speed hub
      7. Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
      8. root@nas:~# cat /etc/modprobe.d/blacklist-uas.conf
      9. options usb-storage quirks=0bc2:2312
      The Seagate disk is now found (/dev/sdd exists) and can be mounted, but the driver in use is again uas...


      Source Code

      1. root@nas:~# lsusb -t
      2. /: Bus 02.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/6p, 5000M
      3. |__ Port 1: Dev 2, If 0, Class=Hub, Driver=hub/4p, 5000M
      4. |__ Port 2: Dev 3, If 0, Class=Mass Storage, Driver=uas, 5000M
      5. /: Bus 01.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/7p, 480M
      6. |__ Port 1: Dev 2, If 0, Class=Hub, Driver=hub/4p, 480M
      7. |__ Port 5: Dev 3, If 0, Class=Hub, Driver=hub/4p, 480M
    • Ok, then running out of ideas.

      I dealt with UAS blacklisting (for Seagate and WD drives and a broken Norelsys USB-to-SATA bridge) last year and obviously have forgotten most things.

      In Armbian (on ARM hardware) we do it via kernel cmdline. We have a simple boot config file and a startup service that automagically UAS blacklists every Seagate USB3 HDD when found to be attached to the host. Looks like this here (sorry, I only use Seagate 'internal' SATA/SAS disks but would never buy any of their USB3 products due to the Linux problems with them):

      Source Code

      1. root@nanopim4:~# cat /boot/armbianEnv.txt
      2. ...
      3. usbstoragequirks=0x2537:0x1066:u,0x2537:0x1068:u

      And then the so called bootscript adds this to kernel cmdline automagically.

      To achieve the same on x86 you need to fiddle around somehow with GRUB config to tell the kernel

      Source Code

      1. usb-storage.quirks=0x0bc2:0x2312:u
      (sorry no idea about details: I don't run OMV on any x86 box)
    • Ok, thank you very much for your help anyway.

      tkaiser wrote:

      I only use Seagate 'internal' SATA/SAS disks but would never buy any of their USB3 products due to the Linux problems with them
      Therefore I guess that the best solution is to change the external disk and to buy a not-Seagate one, isn't it?? Do you have any suggestion about that?
    • I did a google search for your initial error and found this it just might give you some insight, but another option might be to see if there is a firmware update for the specific hard drive.

      I can't test this for you as my OMV doesn't have USB 3.0 ports just 2.0 although I have 2 USB 3.0 drives one I know is WD (my son's) the other is a Buffalo and never checked the drive. I do know that I can rsync without any issues because either drive is connected to usb 2.0.
      Raid is not a backup! Would you go skydiving without a parachute?
    • kuks wrote:

      options usb-storage quirks=0bc2:2312
      There the ':u' is missing. It need to read

      Source Code

      1. options usb-storage quirks=0bc2:2312:u

      kuks wrote:

      I guess that the best solution is to change the external disk and to buy a not-Seagate one, isn't it??
      Nope, you just need to UAS blacklist this model (unfortunately all those Seagate USB3 disks ship with the same broken firmware but each of them uses a different product ID so you can't generically blacklist them).

      Alternative: starting with kernel 4.15 the issue should also be resolved as I explained above. But this has another downside that with kernel 4.14 and UAS active now SMART readouts won't work anymore. So UAS blacklisting Seagate USB HDDs is always the best solution (implemented on all the ARM OMV images since based on Armbian but missing with x86 so an awful lot of Linux users are condemned to run into this situation again and again).

      BTW: With HDD UAS or not makes no real difference so there's no downside caused by blacklisting devices. See this performance report from a nice mini ARM OMV thingy with external USB3 disk but no UAS: forum.armbian.com/topic/8097-n…findComment&comment=61898 (close to 100 MB/s is something I consider 'fast enough')

      The post was edited 1 time, last by tkaiser: It's kernel 4.15 and not 4.14: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1584557/comments/26 ().

    • geaves wrote:

      another option might be to see if there is a firmware update for the specific hard drive
      Seagate is well aware of the problem but since it only seems to affect Linux they don't care. They use an ASM1153 in their USB3 disk enclosures. This bridge chip works fine with standard ASMedia firmware but Seagate somehow managed to screw things up.

      UAS or not makes a huge difference with SSDs (higher random IO performance with UAS and always less CPU utilization) but with HDDs it doesn't matter that much. So simply UAS blacklisting those disks should be fine.