Rockpro64 hard drives are not seen by omv until reboot

    • Rockpro64 hard drives are not seen by omv until reboot

      Hello,

      I have played with omv vm and decided to go with rockpro64, nas case, pcie sata controller and 2x 3.5 wd red drives.
      Main problem is that when i connect power plug to rockpro64 it boots but not recognizes any drives, i get below errors.
      [ 80.509588] ata1: SATA link down (SStatus 1 SControl 310)
      [ 80.534423] ata1: irq_stat 0x00000040, connection status changed
      [ 80.540057] ata1: SError: { CommWake DevExch }
      [ 80.545230] ata1: limiting SATA link speed to 1.5 Gbps
      [ 80.550431] ata1: hard resetting link

      [ 212.084097] ata2: EH complete
      [ 212.097400] ata2: exception Emask 0x10 SAct 0x0 SErr 0x4040000 action 0xe frozen
      [ 212.106333] ata2: irq_stat 0x00000040, connection status changed
      [ 212.110266] ata2: SError: { CommWake DevExch }
      [ 212.114038] ata2: limiting SATA link speed to 1.5 Gbps

      After reboot dmesg below shows that everything is good.
      [ 2.807119] ata1: SATA max UDMA/133 abar m512@0xfa010000 port 0xfa010100 irq 239
      [ 3.269313] ata1: SATA link up 3.0 Gbps (SStatus 123 SControl 300)
      [ 3.275468] ata1.00: ATA-10: WDC WD40EFRX-68N32N0, 82.00A82, max UDMA/133
      [ 3.286399] ata1.00: 7814037168 sectors, multi 16: LBA48 NCQ (depth 31/32), AA
      [ 3.299618] ata1.00: configured for UDMA/133
      [ 10.768358] ata1.00: configured for UDMA/133
      [ 10.781388] ata1: EH complete

      [ 2.809115] ata2: SATA max UDMA/133 abar m512@0xfa010000 port 0xfa010180 irq 239
      [ 3.784369] ata2: SATA link up 3.0 Gbps (SStatus 123 SControl 300)
      [ 3.790628] ata2.00: ATA-10: WDC WD40EFRX-68N32N0, 82.00A82, max UDMA/133
      [ 3.796111] ata2.00: 7814037168 sectors, multi 16: LBA48 NCQ (depth 31/32), AA
      [ 3.802514] ata2.00: configured for UDMA/133
      [ 10.986541] ata2.00: configured for UDMA/133
      [ 10.993483] ata2: EH complete


      Reboot resolves them most of the time but sometimes needs second reboot. I thought adding delay to boot will give sufficient amount of time for drives to spinup and be recognized but it didn't. I modified /boot/extlinux/extlinux.conf and /boot/efi/extlinux/extlinux.conf timeout value to 100 (10 secconds) but it didn't help. I know that it did increase startup time when disks are recognized and when not as well.

      Sata controller card i have is ASMedia Technology Inc. ASM1062 Serial ATA Controller (rev 01).

      I tried changing sata cables, powering hard drives with atx power supply but i just can't find a consistent pattern as to where the problem is.
      I will try to boot armbian tomorrow after being of all night and not connected to electricity.

      Can anyone advise what logs to look at and how to troubleshoot this problem?

      Thanks in advance
    • vecnar wrote:

      I tried changing sata cables, powering hard drives with atx power supply
      Have you also replaced/checked the SATA power cables? Your symptoms sound like a voltage drop on the 5V rail (SATA power cable). Some drives are fine here with as low as 4.4V, some start to have problems already at 4.75V (see this example/list but please keep in mind that there it's about 2.5" drives not comparable to your WD RED -- I don't know how sensitive they are to voltage drops since we don't use WD)
      No more contributions to this project until 'alternative facts' (AKA ignorance/stupidity) are gone
    • tkaiser wrote:

      vecnar wrote:

      I tried changing sata cables, powering hard drives with atx power supply
      Have you also replaced/checked the SATA power cables? Your symptoms sound like a voltage drop on the 5V rail (SATA power cable). Some drives are fine here with as low as 4.4V, some start to have problems already at 4.75V (see this example/list but please keep in mind that there it's about 2.5" drives not comparable to your WD RED -- I don't know how sensitive they are to voltage drops since we don't use WD)
      Hello tkaiser,
      I used atx power supply sata power cables and not the ones that are attached to sbc. The only downside is that i can't power both power supplies at the same time and the issue most likely to happen when power supplies first get power from ac.
      I had it off since last night and had armbian sd card inserted. I plugged in power today and armbian recognized both drives please see screenshot attached. After that i powered it off, replaced sd card with omv powered it on, both hard drives were not recognized. See screenshot of dmesg and full dmesg output from omv. I issued reboot from cli but it didn't recognize hard drives, only after second reboot command from cli they came back.
      Could you advise the best troubleshooting plan in this situation? Every time i take hard drive cage out i move all the cables so there are too many variables in play.
      Thank you for your time
      Images
      • ArmbianAfterNightOff.jpg

        165.11 kB, 841×614, viewed 29 times
      • OMVAfterSuccesfullArmbian_StartOfFailures.jpg

        402.06 kB, 1,280×1,024, viewed 40 times
      Files

      The post was edited 1 time, last by vecnar ().

    • tkaiser wrote:

      vecnar wrote:

      I tried changing sata cables, powering hard drives with atx power supply
      Have you also replaced/checked the SATA power cables? Your symptoms sound like a voltage drop on the 5V rail (SATA power cable). Some drives are fine here with as low as 4.4V, some start to have problems already at 4.75V (see this example/list but please keep in mind that there it's about 2.5" drives not comparable to your WD RED -- I don't know how sensitive they are to voltage drops since we don't use WD)
      Hello tkaiser,

      I tried posting below with a few screenshots and a log file but it was waited to be approved by moderator so i will try to put screenshots in plain text and upload full log if you need it.

      I used atx power supply sata power cables and not the ones that are attached to sbc. The only downside is that i can't power both power supplies at the same time and the issue most likely to happen when power supplies first get power from ac.
      I had it off since last night and had armbian sd card inserted. I plugged in power today and armbian recognized both drives please see dmesg output below.

      Source Code

      1. ata1: SATA link up 6.0 Gbps (SStatus 133 SControl 300)
      2. ata1.00: ATA-10 WDC "serial number", 82.00A82, max UDMA/133
      3. ata1.00: 7814037168 sectors, multi 16: LBA48 NCQ (depth 31/32), AA
      4. ata1.00: configured for udma/133
      5. ata2: SATA link up 6.0 Gbps (SStatus 133 SControl 300)
      6. ata2.00: ATA-10 WDC "serial number", 82.00A82, max UDMA/133
      7. ata2.00: 7814037168 sectors, multi 16: LBA48 NCQ (depth 31/32), AA
      8. ata2.00: configured for udma/133
      After that i powered it off, replaced sd card with omv powered it on, both hard drives were not recognized. See below dmesg errors. I issued reboot from cli but it didn't recognize hard drives, only after second "reboot" command from cli they came back.


      Source Code

      1. [ 7.461809] ata2: SATA link down (SStatus 1 SControl 300)
      2. [ 7.476399] ata2: exception Emask 0x10 SAct 0x0 SErr 0x4040000 action 0xe frozen
      3. [ 7.483279] ata2: irq_stat 0x00000040, connection status changed
      4. [ 7.489419] ata2: SError: { CommWake DevExch }
      5. [ 7.495414] ata2: hard resetting link
      6. [ 7.501306] ata1: exception Emask 0x10 SAct 0x0 SErr 0x4040000 action 0xe frozen
      7. [ 7.507682] ata1: irq_stat 0x00000040, connection status changed
      8. [ 7.513909] ata1: SError: { CommWake DevExch }
      9. [ 7.520018] ata1: limiting SATA link speed to 1.5 Gbps
      10. [ 7.525319] ata1: hard resetting link
      Let me know if you would like to see full dmesg output from omv.

      Could you advise the best troubleshooting plan in this situation? Every time i take hard drive cage out i move all the cables so there are too many variables in play.


      Thank you for your time
    • vecnar wrote:

      The only downside is that i can't power both power supplies at the same time and the issue most likely to happen when power supplies first get power from ac
      No idea what this means but your issue is clearly hardware related (see the dmesg messages, the drive's controller is physically not available).
      No more contributions to this project until 'alternative facts' (AKA ignorance/stupidity) are gone
    • tkaiser wrote:

      vecnar wrote:

      The only downside is that i can't power both power supplies at the same time and the issue most likely to happen when power supplies first get power from ac
      No idea what this means but your issue is clearly hardware related (see the dmesg messages, the drive's controller is physically not available).
      Sorry if i was not clear about atx power supply testing. The issue arises when i plugin dc adapter in the socket with 240 volts and rockpro64 starts itself automatically, i do not need to press anything. I was using atx power supply to power hard drives only, but i wanted to simulate the same behavior as the other dc adapter. I am sure i am not troubleshooting it correctly that is why i am looking for some guidance steps to follow in order to findout where the actual problem is and how to resolve it.

      So does it mean sata controller is an issue that resolves itself after a few restarts under omv? It is strange that armbian doesn't have this problem and detects it. I am not familiar with arm and linux so not sure if they use the same bios like before boot process or armbian and omv use different approaches or versions.

      Do you know if anyone is using rockpro64 wih 2 x 3.5 inch drives and psu 12v 5A successfully and if yes what sata controller card?
    • I got new card and it was working well for a few days but then same problems. It seems to be not a hardware but software not giving enough time for hardware to be initialized. There are some kernel patches going around but i haven't installed any of them as i do not know how to nor do i know if they fix my problem as now it is harder to reproduce and i do not want to go with unstable release of something that should be a stable NAS.
      I will try to wait and get a rough reproducible steps and try latest release but this will only tell me one thing that i will have to wait until stable will be available with that version.
    • vecnar wrote:

      Do you know if anyone is using rockpro64 wih 2 x 3.5 inch drives and psu 12v 5A successfully and if yes what sata controller card?
      I am using this card Marvell 9235 https://www.amazon.com/gp/product/B00AZ9T41M/ref=ppx_yo_dt_b_search_asin_title?ie=UTF8&psc=1 with 2 X 3.5 X10tb and 2 X 2.5 each 2 and 5 TB with original power supply 12v 5A store.pine64.org/?product=rockpro64-12v-5a-us-power-supply and 2 of this cable witch give you 12 and 5 Volts connected together at the white plug for more than 6 months no issues store.pine64.org/?product=rock…able-for-dual-sata-drives.
      Everything fit inside this store.pine64.org/?product=rockpro64-12v-5a-us-power-supply.
      The first Marvell card died after 15 days, subsequently got it replaced by Amazon, going strong since then.

      The post was edited 3 times, last by Nefertiti ().

    • Nefertiti wrote:

      vecnar wrote:

      Do you know if anyone is using rockpro64 wih 2 x 3.5 inch drives and psu 12v 5A successfully and if yes what sata controller card?
      I am using this card Marvell 9235 https://www.amazon.com/gp/product/B00AZ9T41M/ref=ppx_yo_dt_b_search_asin_title?ie=UTF8&psc=1 with 2 X 3.5 X10tb and 2 X 2.5 each 2 and 5 TB with original power supply 12v 5A store.pine64.org/?product=rockpro64-12v-5a-us-power-supply and 2 of this cable witch give you 12 and 5 Volts connected together at the white plug for more than 6 months no issues store.pine64.org/?product=rock…able-for-dual-sata-drives.Everything fit inside this store.pine64.org/?product=rockpro64-12v-5a-us-power-supply.
      The first Marvell card died after 15 days, subsequently got it replaced by Amazon, going strong since then.
      Thank you for sharing the information and i am glad it works well for you.
      How do you power all 4 hard drives, I assume 2 with power cables that feed from the board but what about the other 2?
      What image do you use and what media do you boot from?
      What errors did you get when your sata card failed, same as per below?

      Source Code

      1. rockchip-pcie f8000000.pcie: PCIe link training gen1 imeout!
      2. rockchip-pcie: probe of f8000000.pcie failed with error -110


      I am booting from microsd and using github.com/ayufan-rock64/linux…4-0.7.9-1067-arm64.img.xz
      I currently have marvel 88se9215 4 port sata card but it is pcie 1x (1.0 version) with low throughput.
      Everything is in pine64 case wiki.pine64.org/index.php/NASCase with the fan. I had everything stock but gave up on trying to make fan spinning at the same speed but not too fast constantly with any software so i took an old zalman fan speed controller and fed it from power plug that hard drives feed from and glued speed adjuster at the back.
      I definitely think sata cables with 90 degree angle on one side should be supplied to be attached on the hard drives side as clearance between fan is very small. I think same applies for sata power connectors.
    • vecnar wrote:

      rockchip-pcie f8000000.pcie: PCIe link training gen1 imeout!
      This means that the PCIe connection between RK3399 and your PCIe card is already failing. ayufan reported recently he found an underpowering issue for PCIe which will be fixed with next release (0.8.0 is about to be released soon).
      No more contributions to this project until 'alternative facts' (AKA ignorance/stupidity) are gone
    • vecnar wrote:

      Nefertiti wrote:

      vecnar wrote:

      Do you know if anyone is using rockpro64 wih 2 x 3.5 inch drives and psu 12v 5A successfully and if yes what sata controller card?
      I am using this card Marvell 9235 https://www.amazon.com/gp/product/B00AZ9T41M/ref=ppx_yo_dt_b_search_asin_title?ie=UTF8&psc=1 with 2 X 3.5 X10tb and 2 X 2.5 each 2 and 5 TB with original power supply 12v 5A store.pine64.org/?product=rockpro64-12v-5a-us-power-supply and 2 of this cable witch give you 12 and 5 Volts connected together at the white plug for more than 6 months no issues store.pine64.org/?product=rock…able-for-dual-sata-drives.Everything fit inside this store.pine64.org/?product=rockpro64-12v-5a-us-power-supply.The first Marvell card died after 15 days, subsequently got it replaced by Amazon, going strong since then.
      Thank you for sharing the information and i am glad it works well for you.How do you power all 4 hard drives, I assume 2 with power cables that feed from the board but what about the other 2?
      What image do you use and what media do you boot from?
      What errors did you get when your sata card failed, same as per below?

      Source Code

      1. rockchip-pcie f8000000.pcie: PCIe link training gen1 imeout!
      2. rockchip-pcie: probe of f8000000.pcie failed with error -110

      I am booting from microsd and using github.com/ayufan-rock64/linux…4-0.7.9-1067-arm64.img.xz
      I currently have marvel 88se9215 4 port sata card but it is pcie 1x (1.0 version) with low throughput.
      Everything is in pine64 case wiki.pine64.org/index.php/NASCase with the fan. I had everything stock but gave up on trying to make fan spinning at the same speed but not too fast constantly with any software so i took an old zalman fan speed controller and fed it from power plug that hard drives feed from and glued speed adjuster at the back.
      I definitely think sata cables with 90 degree angle on one side should be supplied to be attached on the hard drives side as clearance between fan is very small. I think same applies for sata power connectors.
      Like I said I connected the two white female connectors into two male connectors soldered pin to pin together to a single female connector going to the board
      I am using a beta image from github.com/ayufan-rock64/linux-build/releases ( I know Tkaiser would disagree about using beta so it is up's to you!) micro SD SanDisk extreme A2.
      if I remember at the time of failing lspci -V gave me no reply case.
      Fan is very important I am using some fan controller from aliexpress.



      Edit
      I decided to be nice! So I opened the case and moved the card out of the way in order to take the pictures of the connectors for you. By the way at the time I was wondering to pick the the 5Volt supply directly from the board at the Gpio pins but was not sure about amp withdraw there, so I decided not to but maybe it is possible and this way you can avoid the second cable.
      Pin out there forum.frank-mankel.org/topic/292/rockpro64-rp64-gpio/2 German site I put it in google translate for English speaking
      translate.google.com/translate…2Frockpro64-rp64-gpio%2F2
      Images
      • 2019-06-20_195945.jpg

        267.39 kB, 1,188×788, viewed 27 times
      • 2019-06-20_195838.jpg

        176.39 kB, 1,022×747, viewed 26 times

      The post was edited 3 times, last by Nefertiti: ADD Pictures ().

    • Nefertiti wrote:

      I know Tkaiser would disagree about using beta so it is up's to you!
      Tkaiser pointed out directly above that the new and different error @vecnar now experienced will be fixed only on beta images and also that 0.8 will be released soon. 0.8.0rc16 will most probably be the next stable version. Test matrix can be found here docs.google.com/spreadsheets/d…ApAyBBfc_BGv2A/edit#gid=0 and if you want to contribute then Pine64 IRC is the place to be.

      @Nefertiti the problem with 'beta releases' is that stuff is supposed to break without any reason since ayufan generates those releases without testing. They are only there to collect feedback and this is something experienced users are needed for willing to give constructive feedback where it belongs to (not here in OMV forum). So when using beta images you are simply on your own if stuff doesn't work as expected

      @vecnar I hope you are aware that you're reporting different issues. If not even a PCIe link between SoC and your new SATA HBA can be established then this is something different than the SATA HBA not being able to establish SATA links with disks.
      No more contributions to this project until 'alternative facts' (AKA ignorance/stupidity) are gone
    • tkaiser wrote:

      Nefertiti wrote:

      I know Tkaiser would disagree about using beta so it is up's to you!
      Tkaiser pointed out directly above that the new and different error @vecnar now experienced will be fixed only on beta images and also that 0.8 will be released soon. 0.8.0rc16 will most probably be the next stable version. Test matrix can be found here docs.google.com/spreadsheets/d…ApAyBBfc_BGv2A/edit#gid=0 and if you want to contribute then Pine64 IRC is the place to be.
      @Nefertiti the problem with 'beta releases' is that stuff is supposed to break without any reason since ayufan generates those releases without testing. They are only there to collect feedback and this is something experienced users are needed for willing to give constructive feedback where it belongs to (not here in OMV forum). So when using beta images you are simply on your own if stuff doesn't work as expected

      @vecnar I hope you are aware that you're reporting different issues. If not even a PCIe link between SoC and your new SATA HBA can be established then this is something different than the SATA HBA not being able to establish SATA links with disks.
      I tried latest release 0.8.0rc16 and couldn't reproduce the problem so fingers crossed it will be stable soon. Now usb dvb-t tuner is not working in docker where it was working in stable release, but this is not for this thread or this forum.
      Thank you for pointing out where the problem is now.
      Looking at the release notes ayufan suggests using armhf for openmediavault. But i saw somewhere that openmediavault stable was pointing out to arm64 image, which one should i choose?

      @Nefertiti Thank you for taking the case off and making pictures, it is not a nice case to remove the top cover from, too many screws. Nice connector for sata power cables. I hope fan controller doesn't restrict airflow too much.
      I am not going to put more than 2 hdds for now but i measured load with socket power meter and 2 3.5 hard drives with board consume 30 watts on startup.
      I would like to have stable release but i will try current release for short time and document all the setup process.
      I attached photos of fan speed controller and space limit between hdds and fan.
      Images
      • IMG_20190622_112639 (Medium).jpg

        230.06 kB, 1,024×768, viewed 19 times
      • IMG_20190622_113035 (Medium).jpg

        174 kB, 1,024×768, viewed 20 times
      • IMG_20190622_113050 (Medium).jpg

        265.94 kB, 1,024×768, viewed 19 times
    • vecnar wrote:

      Thank you for letting me know, I will give it a go and hopefully all will be good.What image should I use armhf or arm64? Ayufan suggests armhf which one do you use?
      It is up to you personally
      and without opening a can of worms I am using the arm64 for a long time without problem beside some plugin not available, anyway somebody complains about the armf version with some PCIE issue forum.pine64.org/showthread.php?tid=7638.
    • As far as i understood the advantage to use armhf is that there are more software packages developed to run on it.
      I read that post on pine64, thank you for pointing it out. I didn't have the same issues with 0.7.9-1067-arm64, i had different problems and i do not think i tried armhf image. I believe the difference between that person setup and mine is boot device, emmc vs microsd and image version/type.
      I will give a try to both images to see if rtl2838 tv card works on it, i am not using it a lot but i would like to have it working.
    • vecnar wrote:

      As far as i understood the advantage to use armhf is that there are more software packages developed to run on it.
      Nope, it was about Plex Media Server which was available only in armhf architecture 2 years ago when ayufan wrote his recommendations. Choosing armhf has another real advantage: smaller memory footprint. An arm64 userland simply wastes more memory.
      No more contributions to this project until 'alternative facts' (AKA ignorance/stupidity) are gone
    • tkaiser wrote:

      vecnar wrote:

      As far as i understood the advantage to use armhf is that there are more software packages developed to run on it.
      Nope, it was about Plex Media Server which was available only in armhf architecture 2 years ago when ayufan wrote his recommendations. Choosing armhf has another real advantage: smaller memory footprint. An arm64 userland simply wastes more memory.
      I read the posts and I saw that arm64 uses 663Mb Vs 363 Mb armhf but I think nobody identified if performance is better in arm64 or armhf, I could have missed something.
      My board has 4 GB and I started configuring arm64, is it worth going back to armhf in my case?
      I will try to run quiet a few dockers, tvheadened, jackett, radarr, sonarr, deluge, some type of Kodi, letsencrypt, airsonic, possibly nextcloud.

      Edit: I just looked online and saw your benchmarks github.com/ThomasKaiser/sbc-bench/blob/master/Results.md
      And arm64 4.4 has slightly higher scores throughout than armhf 4.4. just not sure how much ram each docker consumes and is it worth going back to armhf.

      The post was edited 1 time, last by vecnar ().