ATA Error Count Increasing

zach86 · 19. Februar 2016

Hey Guys,

I've been running OMV for a couple weeks now, a mixture of 4 older 3TB WD Red drives, and 4 new 3TB WD Red drives, using SnapRaid and MergerFS to pool them.

But I've noticed weekly I get hard drive reports stating the ATA error count has increased,

Device: WDC_WD30EFRX-68EUZN0_WD-WMC4N1021445 [SAT], ATA error count increased from 5 to 6
Device: WDC_WD30EFRX-68EUZN0_WD-WCC4N4RR3U8L [SAT], ATA error count increased from 8 to 9
Device: WDC_WD30EFRX-68EUZN0_WD-WCC4N7TACJ0Z [SAT], ATA error count increased from 5 to 6
Device: WDC_WD30EFRX-68EUZN0_WD-WCC4N7TACULH [SAT], ATA error count increased from 3 to 4
Device: WDC_WD30EFRX-68EUZN0_WD-WCC4N0EP7C5R [SAT], ATA error count increased from 5 to 6
Device: WDC_WD30EFRX-68AX9N0_WD-WCC1T1153741 [SAT], ATA error count increased from 5 to 6
Device: WDC_WD30EFRX-68EUZN0_WD-WMC4N0942489 [SAT], ATA error count increased from 5 to 6
Device: WDC_WD30EFRX-68EUZN0_WD-WCC4N76L03FY [SAT], ATA error count increased from 9 to 10

I used to run a QNAP before OMV with 4 of the drives in Raid5, so these errors are definitely new.

I do understand a couple of my drives have a high Load_Cycle_Count. 2 out of 4 of my original WD Reds were from the batch that had the head park set to like 7 seconds, so they racked up quite a count.

As for my system, I'm running:
Intel Xeon E3-1220V3
SUPERMICRO MBD-X10SL7 (Flashed to IT mode for running the 8 x 3TB drives)
16GB Cruial DDR3 SDRAM ECC Unbuffered Ram (2 x 8GB Sticks)

Can anyone provide some insight or suggestions as to what might be happening, or if this is a concern? I don't see any Offline Uncorrectable errors or bad sectors, so that has to mean a good thing right? All my drives are within warrenty, including my 4 original ones (until the end of this year)

Any help or input is very much appreciated!

~~I'll post one of the smart results in a reply to this, as I'm running into Message Too Long to post.~~
Scratch that, my reply don't seem to be showing up. Please find each drive's full Smart Extended Attributes attached here, with each filename being the drives serial number.

zach86 · 19. Februar 2016

WMC4N1021445:

Code

=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED


SMART Attributes Data Structure revision number: 16
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME FLAGS VALUE WORST THRESH FAIL RAW_VALUE
1 Raw_Read_Error_Rate POSR-K 200 200 051 - 0
3 Spin_Up_Time POS--K 174 174 021 - 6266
4 Start_Stop_Count -O--CK 100 100 000 - 39
5 Reallocated_Sector_Ct PO--CK 200 200 140 - 0
7 Seek_Error_Rate -OSR-K 200 200 000 - 0
9 Power_On_Hours -O--CK 074 074 000 - 19034
10 Spin_Retry_Count -O--CK 100 253 000 - 0
11 Calibration_Retry_Count -O--CK 100 253 000 - 0
12 Power_Cycle_Count -O--CK 100 100 000 - 38
192 Power-Off_Retract_Count -O--CK 200 200 000 - 24
193 Load_Cycle_Count -O--CK 065 065 000 - 405723
194 Temperature_Celsius -O---K 130 104 000 - 20
196 Reallocated_Event_Count -O--CK 200 200 000 - 0
197 Current_Pending_Sector -O--CK 200 200 000 - 0
198 Offline_Uncorrectable ----CK 200 200 000 - 0
199 UDMA_CRC_Error_Count -O--CK 200 200 000 - 0
200 Multi_Zone_Error_Rate ---R-- 200 200 000 - 0
||||||_ K auto-keep
|||||__ C event count
||||___ R error rate
|||____ S speed/performance
||_____ O updated online
|______ P prefailure warning


SMART Extended Comprehensive Error Log Version: 1 (6 sectors)
Device Error Count: 7


Error 7 [6] occurred at disk power-on lifetime: 19034 hours (793 days + 2 hours)
When the command that caused the error occurred, the device was active or idle.


After command completion occurred, registers were:
ER -- ST COUNT LBA_48 LH LM LL DV DC
-- -- -- == -- == == == -- -- -- -- --
04 -- 51 00 01 00 00 00 00 00 00 00 00 Error: ABRT


Commands leading to the command that caused the error were:
CR FEATR COUNT LBA_48 LH LM LL DV DC Powered_Up_Time Command/Feature_Name
-- == -- == -- == == == -- -- -- -- -- --------------- --------------------
b0 00 d5 00 01 00 00 00 c2 4f e1 00 00 19d+06:28:02.285 SMART READ LOG
b0 00 d5 00 01 00 00 00 c2 4f e1 00 00 19d+06:28:02.285 SMART READ LOG
b0 00 d6 00 01 00 00 00 c2 4f e0 00 00 19d+06:28:02.284 SMART WRITE LOG
b0 00 d6 00 01 00 00 00 c2 4f e0 00 00 19d+06:28:02.283 SMART WRITE LOG
b0 00 d5 00 01 00 00 00 c2 4f e0 00 00 19d+06:28:02.283 SMART READ LOG


Error 6 [5] occurred at disk power-on lifetime: 19033 hours (793 days + 1 hours)
When the command that caused the error occurred, the device was active or idle.


After command completion occurred, registers were:
ER -- ST COUNT LBA_48 LH LM LL DV DC
-- -- -- == -- == == == -- -- -- -- --
04 -- 51 00 01 00 00 00 00 00 00 00 00 Error: ABRT


Commands leading to the command that caused the error were:
CR FEATR COUNT LBA_48 LH LM LL DV DC Powered_Up_Time Command/Feature_Name
-- == -- == -- == == == -- -- -- -- -- --------------- --------------------
b0 00 d5 00 01 00 00 00 c2 4f e1 00 00 19d+05:37:28.472 SMART READ LOG
b0 00 d5 00 01 00 00 00 c2 4f e1 00 00 19d+05:37:28.472 SMART READ LOG
b0 00 d6 00 01 00 00 00 c2 4f e0 00 00 19d+05:37:28.471 SMART WRITE LOG
b0 00 d6 00 01 00 00 00 c2 4f e0 00 00 19d+05:37:28.470 SMART WRITE LOG
b0 00 d5 00 01 00 00 00 c2 4f e0 00 00 19d+05:37:28.470 SMART READ LOG


Error 5 [4] occurred at disk power-on lifetime: 18573 hours (773 days + 21 hours)
When the command that caused the error occurred, the device was active or idle.


After command completion occurred, registers were:
ER -- ST COUNT LBA_48 LH LM LL DV DC
-- -- -- == -- == == == -- -- -- -- --
04 -- 51 00 01 00 00 00 00 00 00 00 00 Error: ABRT


Commands leading to the command that caused the error were:
CR FEATR COUNT LBA_48 LH LM LL DV DC Powered_Up_Time Command/Feature_Name
-- == -- == -- == == == -- -- -- -- -- --------------- --------------------
b0 00 d5 00 01 00 00 00 c2 4f e1 00 00 01:00:30.746 SMART READ LOG
b0 00 d5 00 01 00 00 00 c2 4f e1 00 00 01:00:30.746 SMART READ LOG
b0 00 d6 00 01 00 00 00 c2 4f e0 00 00 01:00:30.745 SMART WRITE LOG
b0 00 d6 00 01 00 00 00 c2 4f e0 00 00 01:00:30.744 SMART WRITE LOG
b0 00 d5 00 01 00 00 00 c2 4f e0 00 00 01:00:30.744 SMART READ LOG


Error 4 [3] occurred at disk power-on lifetime: 18559 hours (773 days + 7 hours)
When the command that caused the error occurred, the device was active or idle.


After command completion occurred, registers were:
ER -- ST COUNT LBA_48 LH LM LL DV DC
-- -- -- == -- == == == -- -- -- -- --
04 -- 51 00 01 00 00 00 00 00 00 00 00 Error: ABRT


Commands leading to the command that caused the error were:
CR FEATR COUNT LBA_48 LH LM LL DV DC Powered_Up_Time Command/Feature_Name
-- == -- == -- == == == -- -- -- -- -- --------------- --------------------
b0 00 d5 00 01 00 00 00 c2 4f e1 00 00 01:38:43.651 SMART READ LOG
b0 00 d5 00 01 00 00 00 c2 4f e1 00 00 01:38:43.651 SMART READ LOG
e5 00 00 00 00 00 00 00 00 00 00 00 00 01:38:43.651 CHECK POWER MODE
b0 00 d6 00 01 00 00 00 c2 4f e0 00 00 01:38:43.650 SMART WRITE LOG
b0 00 d6 00 01 00 00 00 c2 4f e0 00 00 01:38:43.649 SMART WRITE LOG


Error 3 [2] occurred at disk power-on lifetime: 18559 hours (773 days + 7 hours)
When the command that caused the error occurred, the device was active or idle.


After command completion occurred, registers were:
ER -- ST COUNT LBA_48 LH LM LL DV DC
-- -- -- == -- == == == -- -- -- -- --
04 -- 51 00 01 00 00 00 00 00 00 00 00 Error: ABRT


Commands leading to the command that caused the error were:
CR FEATR COUNT LBA_48 LH LM LL DV DC Powered_Up_Time Command/Feature_Name
-- == -- == -- == == == -- -- -- -- -- --------------- --------------------
b0 00 d5 00 01 00 00 00 c2 4f e1 00 00 01:35:11.163 SMART READ LOG
b0 00 d5 00 01 00 00 00 c2 4f e1 00 00 01:35:11.163 SMART READ LOG
b0 00 d6 00 01 00 00 00 c2 4f e0 00 00 01:35:11.162 SMART WRITE LOG
b0 00 d6 00 01 00 00 00 c2 4f e0 00 00 01:35:11.161 SMART WRITE LOG
b0 00 d5 00 01 00 00 00 c2 4f e0 00 00 01:35:11.161 SMART READ LOG


Error 2 [1] occurred at disk power-on lifetime: 18559 hours (773 days + 7 hours)
When the command that caused the error occurred, the device was active or idle.


After command completion occurred, registers were:
ER -- ST COUNT LBA_48 LH LM LL DV DC
-- -- -- == -- == == == -- -- -- -- --
04 -- 51 00 01 00 00 00 00 00 00 00 00 Error: ABRT


Commands leading to the command that caused the error were:
CR FEATR COUNT LBA_48 LH LM LL DV DC Powered_Up_Time Command/Feature_Name
-- == -- == -- == == == -- -- -- -- -- --------------- --------------------
b0 00 d5 00 01 00 00 00 c2 4f e1 00 00 01:29:43.782 SMART READ LOG
b0 00 d5 00 01 00 00 00 c2 4f e1 00 00 01:29:43.782 SMART READ LOG
b0 00 d6 00 01 00 00 00 c2 4f e0 00 00 01:29:43.781 SMART WRITE LOG
b0 00 d6 00 01 00 00 00 c2 4f e0 00 00 01:29:43.780 SMART WRITE LOG
b0 00 d5 00 01 00 00 00 c2 4f e0 00 00 01:29:43.780 SMART READ LOG


Error 1 [0] occurred at disk power-on lifetime: 18559 hours (773 days + 7 hours)
When the command that caused the error occurred, the device was active or idle.


After command completion occurred, registers were:
ER -- ST COUNT LBA_48 LH LM LL DV DC
-- -- -- == -- == == == -- -- -- -- --
04 -- 51 00 01 00 00 00 00 00 00 00 00 Error: ABRT


Commands leading to the command that caused the error were:
CR FEATR COUNT LBA_48 LH LM LL DV DC Powered_Up_Time Command/Feature_Name
-- == -- == -- == == == -- -- -- -- -- --------------- --------------------
b0 00 d5 00 01 00 00 00 c2 4f e1 00 00 01:28:08.211 SMART READ LOG
b0 00 d5 00 01 00 00 00 c2 4f e1 00 00 01:28:08.211 SMART READ LOG
b0 00 d6 00 01 00 00 00 c2 4f e0 00 00 01:28:08.210 SMART WRITE LOG
b0 00 d6 00 01 00 00 00 c2 4f e0 00 00 01:28:08.209 SMART WRITE LOG
b0 00 d5 00 01 00 00 00 c2 4f e0 00 00 01:28:08.209 SMART READ LOG


SCT Error Recovery Control:
Read: 70 (7.0 seconds)
Write: 70 (7.0 seconds)


SATA Phy Event Counters (GP Log 0x11)
ID Size Value Description
0x0001 2 0 Command failed due to ICRC error
0x0002 2 0 R_ERR response for data FIS
0x0003 2 0 R_ERR response for device-to-host data FIS
0x0004 2 0 R_ERR response for host-to-device data FIS
0x0005 2 0 R_ERR response for non-data FIS
0x0006 2 0 R_ERR response for device-to-host non-data FIS
0x0007 2 0 R_ERR response for host-to-device non-data FIS
0x0008 2 0 Device-to-host non-data FIS retries
0x0009 2 3 Transition from drive PhyRdy to drive PhyNRdy
0x000a 2 4 Device-to-host register FISes sent due to a COMRESET
0x000b 2 0 CRC errors within host-to-device FIS
0x000f 2 0 R_ERR response for host-to-device data FIS, CRC
0x0012 2 0 R_ERR response for host-to-device non-data FIS, CRC
0x8000 4 1664883 Vendor specific

Alles anzeigen

tinh_x7 · 19. Februar 2016

I'm using 3TB WD Red also, and they're tend to be having this issue.
Swap out your SATA cables/port, and see if the error goes away.
If not, do a RMA while it's still in warranty.
Avoid buying 3TB WD Red in the future.
Avoid using spindown the HDDs in the power setting to prolong the HDD life.

zach86 · 20. Februar 2016

I can't just RMA all 8 of my drives (4 of them being 2 years old, and 4 of them being under 2 months old) just because of an ATA Error. The other drive statistics themselves are just fine.

I could see if one or two of them were having some issues, but all 8 of them showing this error, I'm trying to find out a valid cause as to why all 8 of them are doing this with OMV.

I never spin down my drives, the head parking was part of the hard drive firmware (Similar to the WD Green drives), and came from a bad batch of them. I disabled the head parking altogether with the 4 oldest ones as soon as I actually realized last month. All 4 of the new ones have a time set of 300 seconds before parking the heads which is fine for me.

tinh_x7 · 20. Februar 2016

I"m not implying you need to return all eight drives at the same time.
I'm not an expert on hard drive, but update your BIOS firmware, see if that help.

davidh2k · 21. Februar 2016

Reading that all of your drives go up on the ATA Error count either your Sata Controller wen't sideways or something else changed dramatically. When did those errors start to occur, right from the start after switching from QNAP?

Greetings
David

zach86 · 23. Februar 2016

Hey David,

Yeah these errors started after I built my NAS and switched over from my QNAP. Bought 4 additional drives, played with FreeNAS for awhile, but ended up moving over to OMV completely due to easier interface and usability.

I built the system brand new a month ago though, and noticed the first errors started around Jan 29th, shortly after I started using OMV.

I know its unlikely, but could I have 8 faulty Sata cables causing issues? Maybe I need to move the power and sata cables to be further apart from one another, I didn't think that would cause issues.
It could also be the firmware of the onboard LSI 2308 controller on the Supermicro X10SL7-F motherboard.
I flashed it to IT mode with the latest version 20 of the firmware. I know a few places suggested version 16, and a few suggested 15, I really wasn't sure which would ultimately be the best. I have versions 15, 16.0.1, 19, and 20 downloaded.

Attached is a picture with the first 4 drives installed initially after setup, I haven't had a chance to go down and pull the side off to get a picture with all 8, but they are all in a neat line.

davidh2k · 23. Februar 2016

I really can't tell but I think it may be worth to try the other fw version if those are the more suggested ones.

Greetings
David

ATA Error Count Increasing

Jetzt mitmachen!

Tags