DegradedArray Event

johnyb · 4. August 2014

I've enabled messaging and recieved this but cannot make sense of it. Is this something I need to be concerned with, maybe a disk failing?

Code

This is an automatically generated mail message from mdadm
running on OMV


A DegradedArray event had been detected on md device /dev/md/OMVRaid5.


Faithfully yours, etc.


P.S. The /proc/mdstat file currently contains the following:


Personalities : [raid6] [raid5] [raid4]
md127 : active raid5 sda[0] sdd[3] sdb[1]
      8790795264 blocks super 1.2 level 5, 512k chunk, algorithm 2 [4/3] [UU_U]


unused devices: <none>

Alles anzeigen

Gutz-Pilz · 4. August 2014

seems like sdc failed.

post the output of:

Code

smartctl -s on -a /dev/sdc

and

Code

dmesg | grep error

johnyb · 5. August 2014

Thanks for the reply, I've not ignored you - I haven't port forwarded 22 on the router so cannot access the server via command line right now... Need to make a house visit

johnyb · 6. August 2014

Looks like the disk has died, so being shipped out for warranty replacement. Once I recieve the replacement, what steps do I need complete to add this drive back into the RAID? Thanks

WastlJ · 7. August 2014

Replace the drive, look what letter it got assigned (sda, sdb, sdc, etc...) and replace it with sdX with the code below: (when sda is your system drive!)

Code

mdadm --stop /dev/md127
mdadm --zero-superblock /dev/sdX
mdadm --assemble /dev/md127 /dev/sd[bcdefghijklm] --verbose --force

johnyb · 7. August 2014

Thanks for the response. Will give this a go once the drive is replaced. I suppose I naively thought this would be covered somewhere within the GUI.

mcloum · 8. August 2014

It is, goto the RAID tab and expand the array and choose the new drive when you install it.

Not 100% sure this removes the failed drive from the array, would hope it does otherwise its a pretty pointless operation

SerErris · 11. August 2014

after you have rebuilt your raid, please check /etc/md/md.conf if youre spare settings are correct (correct number for spare disks). Otherwise you will start getting messages from the system because of missing spares.

WastlJ · 11. August 2014

Some of us are used to do things with the CLI. Sometimes I even forget what is all possible via GUI.

johnyb · 18. August 2014

Within the gui, used recover and selected the new drive and it started the process great I thought. I'm now receiving the following messages.... I've only got access to the server via a mobile handset right now so no to access info.... have I done something wrong?

This is an automatically generated mail message from mdadm
running on OMV

A DegradedArray event had been detected on md device /dev/md/OMVRaid5.

Faithfully yours, etc.

P.S. The /proc/mdstat file currently contains the following:

Personalities : [raid6] [raid5] [raid4]
md127 : active raid5 sdd[4](S) sdb[0] sde[3] sdc[1](F)
8790795264 blocks super 1.2 level 5, 512k chunk, algorithm 2 [4/2] [U__U]

unused devices: <none>

Looking at the raid it shows....

Version : 1.2
Creation Time : Tue May 28 15:18:10 2013
Raid Level : raid5
Array Size : 8790795264 (8383.56 GiB 9001.77 GB)
Used Dev Size : 2930265088 (2794.52 GiB 3000.59 GB)
Raid Devices : 4
Total Devices : 4
Persistence : Superblock is persistent

Update Time : Mon Aug 18 13:20:37 2014
State : clean, FAILED
Active Devices : 2
Working Devices : 3
Failed Devices : 1
Spare Devices : 1

Layout : left-symmetric
Chunk Size : 512K

Name : N36L-OMV:OMVRaid5
UUID : 860d598b:36d96569:15b715f8:06ffde88
Events : 417048

Number Major Minor RaidDevice State
0 8 16 0 active sync /dev/sdb
1 0 0 1 removed
2 0 0 2 removed
3 8 64 3 active sync /dev/sde

1 8 32 - faulty spare /dev/sdc
4 8 48 - spare /dev/sdd

johnyb · 18. August 2014

Hers one of the other message I revived
This email was generated by the smartd daemon running on:

host name: OMV
DNS domain: WORKGROUP
NIS domain: (none)

The following warning/error was logged by the smartd daemon:

Device: /dev/disk/by-id/wwn-0x50014ee60338b828 [SAT], 5 Offline uncorrectable sectors

For details see host's SYSLOG (default: /var/log/syslog).

You can also use the smartctl utility for further investigation.
No additional email messages about this problem will be sent.

johnyb · 18. August 2014

And another while it was recovering the raid..
This is an automatically generated mail message from mdadm
running on OMV

A Fail event had been detected on md device /dev/md/OMVRaid5.

It could be related to component device /dev/sdc.

Faithfully yours, etc.

P.S. The /proc/mdstat file currently contains the following:

Personalities : [raid6] [raid5] [raid4]
md127 : active raid5 sdd[4] sdb[0] sde[3] sdc[1](F)
8790795264 blocks super 1.2 level 5, 512k chunk, algorithm 2 [4/2] [U__U]
[==================>..] recovery = 93.4% (2738177408/2930265088) finish=335.8min speed=9532K/sec

unused devices: <none>

WastlJ · 18. August 2014

would you please do a:

Code

smartctl -a /dev/sdc

johnyb · 19. August 2014

Thanks....but looks like it cannot find the command.. from what I can see (remotely) the shares are not accessible

root@OMV:~# smartctl -a /dev/sdc
-bash: smartctl: command not found
root@OMV:~#

davidh2k · 19. August 2014

Weird... whats the output of

'dpkg -l | grep smartmontools'

Greetings
David

johnyb · 19. August 2014

I just tried rebooting server to see if that helped, now cannot get access. Twat

WastlJ · 19. August 2014

where can´t you get access?

davidh2k · 19. August 2014

Check the local output of you NAS. Maybe it holds because of a missing drive.

Greetings
David

johnyb · 19. August 2014

Yeah unfortunately I'm no where near the server right now, so it'll have to wait until next Monday. It's a headless unit so cannot even talk anyone through looking at it. Strange since all I've done is replace the faulty drive, asked our to rebuild the raid 5 and now this. Surely it would be pretty bad luck to get another faulty drive
Thanks for your help

johnyb · 25. August 2014

So home now and have physical access to the server, results to comand as follows:

Code

root@OMV:~# smartctl -a /dev/sdc
smartctl 5.40 2010-07-12 r3124 [x86_64-unknown-linux-gnu] (local build)
Copyright (C) 2002-10 by Bruce Allen, [url]http://smartmontools.sourceforge.net[/url]


=== START OF INFORMATION SECTION ===
Device Model:     WDC WD30EFRX-68AX9N0
Serial Number:    WD-WMC1T2837415
Firmware Version: 80.00A80
User Capacity:    3,000,592,982,016 bytes
Device is:        Not in smartctl database [for details use: -P showall]
ATA Version is:   9
ATA Standard is:  Exact ATA specification draft version not indicated
Local Time is:    Mon Aug 25 16:00:56 2014 BST
SMART support is: Available - device has SMART capability.
SMART support is: Enabled


=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED


General SMART Values:
Offline data collection status:  (0x82) Offline data collection activity
                                        was completed without error.
                                        Auto Offline Data Collection: Enabled.
Self-test execution status:      (   0) The previous self-test routine completed
                                        without error or no self-test has ever
                                        been run.
Total time to complete Offline
data collection:                 (41040) seconds.
Offline data collection
capabilities:                    (0x7b) SMART execute Offline immediate.
                                        Auto Offline data collection on/off support.
                                        Suspend Offline collection upon new
                                        command.
                                        Offline surface scan supported.
                                        Self-test supported.
                                        Conveyance Self-test supported.
                                        Selective Self-test supported.
SMART capabilities:            (0x0003) Saves SMART data before entering
                                        power-saving mode.
                                        Supports SMART auto save timer.
Error logging capability:        (0x01) Error logging supported.
                                        General Purpose Logging supported.
Short self-test routine
recommended polling time:        (   2) minutes.
Extended self-test routine
recommended polling time:        ( 255) minutes.
Conveyance self-test routine
recommended polling time:        (   5) minutes.
SCT capabilities:              (0x70bd) SCT Status supported.
                                        SCT Error Recovery Control supported.
                                        SCT Feature Control supported.
                                        SCT Data Table supported.


SMART Attributes Data Structure revision number: 16
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME          FLAG     VALUE WORST THRESH TYPE      UPDATED  WHEN_FAILED RAW_VALUE
  1 Raw_Read_Error_Rate     0x002f   200   200   051    Pre-fail  Always       -       1871
  3 Spin_Up_Time            0x0027   172   172   021    Pre-fail  Always       -       6366
  4 Start_Stop_Count        0x0032   100   100   000    Old_age   Always       -       485
  5 Reallocated_Sector_Ct   0x0033   200   200   140    Pre-fail  Always       -       0
  7 Seek_Error_Rate         0x002e   200   200   000    Old_age   Always       -       0
  9 Power_On_Hours          0x0032   086   086   000    Old_age   Always       -       10881
 10 Spin_Retry_Count        0x0032   100   100   000    Old_age   Always       -       0
 11 Calibration_Retry_Count 0x0032   100   253   000    Old_age   Always       -       0
 12 Power_Cycle_Count       0x0032   100   100   000    Old_age   Always       -       21
192 Power-Off_Retract_Count 0x0032   200   200   000    Old_age   Always       -       10
193 Load_Cycle_Count        0x0032   200   200   000    Old_age   Always       -       474
194 Temperature_Celsius     0x0022   123   113   000    Old_age   Always       -       27
196 Reallocated_Event_Count 0x0032   200   200   000    Old_age   Always       -       0
197 Current_Pending_Sector  0x0032   200   200   000    Old_age   Always       -       19
198 Offline_Uncorrectable   0x0030   200   200   000    Old_age   Offline      -       5
199 UDMA_CRC_Error_Count    0x0032   200   200   000    Old_age   Always       -       0
200 Multi_Zone_Error_Rate   0x0008   200   200   000    Old_age   Offline      -       378


SMART Error Log Version: 1
No Errors Logged


SMART Self-test log structure revision number 1
No self-tests have been logged.  [To run self-tests, use: smartctl -t]


SMART Selective self-test log data structure revision number 1
 SPAN  MIN_LBA  MAX_LBA  CURRENT_TEST_STATUS
    1        0        0  Not_testing
    2        0        0  Not_testing
    3        0        0  Not_testing
    4        0        0  Not_testing
    5        0        0  Not_testing
Selective self-test flags (0x0):
  After scanning selected spans, do NOT read-scan remainder of disk.
If Selective self-test is pending on power-up, resume after 0 minute delay.

Alles anzeigen

Jetzt mitmachen!