Extremely slow HD speeds

    This site uses cookies. By continuing to browse this site, you are agreeing to our Cookie Policy.

    • Extremely slow HD speeds

      Hi, OMV and linux noob here, (but long IT nerdery experience). I put together an OMV box using the latest stable OMV 2 on an HP proliant server. I have two identical WD Red NAS hard drives mounted and used in a grey hole pool (Disk A is set up as the landing zone for the shares). Performance from disk B is perfectly adequate but read and write from disk A is extremely poor. I first noticed very slow writes to the samba shares but then noticed intermittent issues with playing movies on Plex. Some movies (which I later worked out are saved on disk A by greyhole) won't play more than a few seconds, others (stored on disk B) play fine.

      I have followed the diagnostics from the sticky post at the top of this forum and everything looks ok until I get to the performance tests:

      Source Code

      1. root@Beast:~# dd conv=fdatasync if=/dev/mapper/sda1-sda1 of=/tmp/test.img bs=1G count=3
      2. 3+0 records in
      3. 3+0 records out
      4. 3221225472 bytes (3.2 GB) copied, 825.229 s, 3.9 MB/s
      5. root@Beast:~# dd if=/dev/mapper/sda1-sda1 of=/dev/null bs=1G count=3
      6. 3+0 records in
      7. 3+0 records out
      8. 3221225472 bytes (3.2 GB) copied, 650.054 s, 5.0 MB/s


      The same tests on disk B look fine:

      Source Code

      1. root@Beast:~# dd conv=fdatasync if=/dev/mapper/sdb1-sdb1 of=/tmp/test.img bs=1G count=3
      2. 3+0 records in
      3. 3+0 records out
      4. 3221225472 bytes (3.2 GB) copied, 29.2664 s, 110 MB/s
      5. root@Beast:~# dd if=/dev/mapper/sdb1-sdb1 of=/dev/null bs=1G count=3
      6. 3+0 records in
      7. 3+0 records out
      8. 3221225472 bytes (3.2 GB) copied, 21.3882 s, 151 MB/s

      I also ran the following:

      Source Code

      1. root@Beast:~# sudo hdparm -Tt /dev/mapper/sda1-sda1
      2. /dev/mapper/sda1-sda1:
      3. Timing cached reads: 2 MB in 18.79 seconds = 108.99 kB/sec
      4. Timing buffered disk reads: 2 MB in 15.69 seconds = 130.50 kB/sec

      As you can see, not very good numbers. For the record, the identical drive B shows the following:

      Source Code

      1. root@Beast:~# sudo hdparm -Tt /dev/mapper/sdb1-sdb1
      2. /dev/mapper/sdb1-sdb1:
      3. Timing cached reads: 24812 MB in 2.00 seconds = 12421.07 MB/sec
      4. Timing buffered disk reads: 456 MB in 3.01 seconds = 151.69 MB/sec

      As I said, I'm a linux noob so don't really know where to start investigating further to sort this out, so any help would be very welcome. Thanks in advance.
    • Source Code

      1. smartctl 5.41 2011-06-09 r3365 [x86_64-linux-3.2.0-4-amd64] (local build)
      2. Copyright (C) 2002-11 by Bruce Allen, http://smartmontools.sourceforge.net
      3. === START OF INFORMATION SECTION ===
      4. Model Family: Western Digital Caviar Green
      5. Device Model: WDC WD10EACS-00ZJB0
      6. Serial Number: WD-WCASJ2000356
      7. LU WWN Device Id: 5 0014ee 2574fc825
      8. Firmware Version: 01.01B01
      9. User Capacity: 1,000,203,804,160 bytes [1.00 TB]
      10. Sector Size: 512 bytes logical/physical
      11. Device is: In smartctl database [for details use: -P show]
      12. ATA Version is: 8
      13. ATA Standard is: Exact ATA specification draft version not indicated
      14. Local Time is: Fri Apr 21 12:37:41 2017 BST
      15. SMART support is: Available - device has SMART capability.
      16. SMART support is: Enabled
      17. === START OF READ SMART DATA SECTION ===
      18. SMART overall-health self-assessment test result: PASSED
      19. General SMART Values:
      20. Offline data collection status: (0x84) Offline data collection activity
      21. was suspended by an interrupting command from host.
      22. Auto Offline Data Collection: Enabled.
      23. Self-test execution status: ( 0) The previous self-test routine completed
      24. without error or no self-test has ever
      25. been run.
      26. Total time to complete Offline
      27. data collection: (27180) seconds.
      28. Offline data collection
      29. capabilities: (0x7b) SMART execute Offline immediate.
      30. Auto Offline data collection on/off support.
      31. Suspend Offline collection upon new
      32. command.
      33. Offline surface scan supported.
      34. Self-test supported.
      35. Conveyance Self-test supported.
      36. Selective Self-test supported.
      37. SMART capabilities: (0x0003) Saves SMART data before entering
      38. power-saving mode.
      39. Supports SMART auto save timer.
      40. Error logging capability: (0x01) Error logging supported.
      41. General Purpose Logging supported.
      42. Short self-test routine
      43. recommended polling time: ( 2) minutes.
      44. Extended self-test routine
      45. recommended polling time: ( 255) minutes.
      46. Conveyance self-test routine
      47. recommended polling time: ( 5) minutes.
      48. SCT capabilities: (0x303f) SCT Status supported.
      49. SCT Error Recovery Control supported.
      50. SCT Feature Control supported.
      51. SCT Data Table supported.
      52. SMART Attributes Data Structure revision number: 16
      53. Vendor Specific SMART Attributes with Thresholds:
      54. ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED WHEN_FAILED RAW_VALUE
      55. 1 Raw_Read_Error_Rate 0x000f 200 200 051 Pre-fail Always - 0
      56. 3 Spin_Up_Time 0x0003 197 187 021 Pre-fail Always - 7133
      57. 4 Start_Stop_Count 0x0032 084 084 000 Old_age Always - 16597
      58. 5 Reallocated_Sector_Ct 0x0033 200 200 140 Pre-fail Always - 0
      59. 7 Seek_Error_Rate 0x000e 200 200 051 Old_age Always - 0
      60. 9 Power_On_Hours 0x0032 007 007 000 Old_age Always - 68342
      61. 10 Spin_Retry_Count 0x0012 100 100 051 Old_age Always - 0
      62. 11 Calibration_Retry_Count 0x0012 100 100 051 Old_age Always - 0
      63. 12 Power_Cycle_Count 0x0032 100 100 000 Old_age Always - 284
      64. 192 Power-Off_Retract_Count 0x0032 200 200 000 Old_age Always - 103
      65. 193 Load_Cycle_Count 0x0032 001 001 000 Old_age Always - 948458
      66. 194 Temperature_Celsius 0x0022 113 093 000 Old_age Always - 39
      67. 196 Reallocated_Event_Count 0x0032 200 200 000 Old_age Always - 0
      68. 197 Current_Pending_Sector 0x0012 200 200 000 Old_age Always - 0
      69. 198 Offline_Uncorrectable 0x0010 200 200 000 Old_age Offline - 0
      70. 199 UDMA_CRC_Error_Count 0x003e 200 200 000 Old_age Always - 0
      71. 200 Multi_Zone_Error_Rate 0x0008 200 200 051 Old_age Offline - 0
      72. SMART Error Log Version: 1
      73. No Errors Logged
      74. SMART Self-test log structure revision number 1
      75. No self-tests have been logged. [To run self-tests, use: smartctl -t]
      76. SMART Selective self-test log data structure revision number 1
      77. SPAN MIN_LBA MAX_LBA CURRENT_TEST_STATUS
      78. 1 0 0 Not_testing
      79. 2 0 0 Not_testing
      80. 3 0 0 Not_testing
      81. 4 0 0 Not_testing
      82. 5 0 0 Not_testing
      83. Selective self-test flags (0x0):
      84. After scanning selected spans, do NOT read-scan remainder of disk.
      85. If Selective self-test is pending on power-up, resume after 0 minute delay.
      Display All
      I'm guessing that's bad?
    • Source Code

      1. smartctl 5.41 2011-06-09 r3365 [x86_64-linux-3.2.0-4-amd64] (local build)
      2. Copyright (C) 2002-11 by Bruce Allen, http://smartmontools.sourceforge.net
      3. === START OF INFORMATION SECTION ===
      4. Model Family: Western Digital Red
      5. Device Model: WDC WD30EFRX-68EUZN0
      6. Serial Number: WD-WMC4N0312651
      7. LU WWN Device Id: 5 0014ee 6ae6a6d08
      8. Firmware Version: 80.00A80
      9. User Capacity: 3,000,592,982,016 bytes [3.00 TB]
      10. Sector Sizes: 512 bytes logical, 4096 bytes physical
      11. Device is: In smartctl database [for details use: -P show]
      12. ATA Version is: 9
      13. ATA Standard is: Exact ATA specification draft version not indicated
      14. Local Time is: Fri Apr 21 12:55:39 2017 BST
      15. SMART support is: Available - device has SMART capability.
      16. SMART support is: Enabled
      17. === START OF READ SMART DATA SECTION ===
      18. SMART overall-health self-assessment test result: PASSED
      19. General SMART Values:
      20. Offline data collection status: (0x00) Offline data collection activity
      21. was never started.
      22. Auto Offline Data Collection: Disabled.
      23. Self-test execution status: ( 0) The previous self-test routine completed
      24. without error or no self-test has ever
      25. been run.
      26. Total time to complete Offline
      27. data collection: (40560) seconds.
      28. Offline data collection
      29. capabilities: (0x7b) SMART execute Offline immediate.
      30. Auto Offline data collection on/off support.
      31. Suspend Offline collection upon new
      32. command.
      33. Offline surface scan supported.
      34. Self-test supported.
      35. Conveyance Self-test supported.
      36. Selective Self-test supported.
      37. SMART capabilities: (0x0003) Saves SMART data before entering
      38. power-saving mode.
      39. Supports SMART auto save timer.
      40. Error logging capability: (0x01) Error logging supported.
      41. General Purpose Logging supported.
      42. Short self-test routine
      43. recommended polling time: ( 2) minutes.
      44. Extended self-test routine
      45. recommended polling time: ( 255) minutes.
      46. Conveyance self-test routine
      47. recommended polling time: ( 5) minutes.
      48. SCT capabilities: (0x703d) SCT Status supported.
      49. SCT Error Recovery Control supported.
      50. SCT Feature Control supported.
      51. SCT Data Table supported.
      52. SMART Attributes Data Structure revision number: 16
      53. Vendor Specific SMART Attributes with Thresholds:
      54. ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED WHEN_FAILED RAW_VALUE
      55. 1 Raw_Read_Error_Rate 0x002f 187 187 051 Pre-fail Always - 34710
      56. 3 Spin_Up_Time 0x0027 187 180 021 Pre-fail Always - 5650
      57. 4 Start_Stop_Count 0x0032 100 100 000 Old_age Always - 63
      58. 5 Reallocated_Sector_Ct 0x0033 200 200 140 Pre-fail Always - 0
      59. 7 Seek_Error_Rate 0x002e 200 200 000 Old_age Always - 0
      60. 9 Power_On_Hours 0x0032 073 073 000 Old_age Always - 20422
      61. 10 Spin_Retry_Count 0x0032 100 253 000 Old_age Always - 0
      62. 11 Calibration_Retry_Count 0x0032 100 253 000 Old_age Always - 0
      63. 12 Power_Cycle_Count 0x0032 100 100 000 Old_age Always - 63
      64. 192 Power-Off_Retract_Count 0x0032 200 200 000 Old_age Always - 58
      65. 193 Load_Cycle_Count 0x0032 200 200 000 Old_age Always - 4
      66. 194 Temperature_Celsius 0x0022 110 105 000 Old_age Always - 40
      67. 196 Reallocated_Event_Count 0x0032 200 200 000 Old_age Always - 0
      68. 197 Current_Pending_Sector 0x0032 200 200 000 Old_age Always - 1
      69. 198 Offline_Uncorrectable 0x0030 100 253 000 Old_age Offline - 0
      70. 199 UDMA_CRC_Error_Count 0x0032 200 200 000 Old_age Always - 0
      71. 200 Multi_Zone_Error_Rate 0x0008 100 253 000 Old_age Offline - 0
      72. SMART Error Log Version: 1
      73. No Errors Logged
      74. SMART Self-test log structure revision number 1
      75. No self-tests have been logged. [To run self-tests, use: smartctl -t]
      76. SMART Selective self-test log data structure revision number 1
      77. SPAN MIN_LBA MAX_LBA CURRENT_TEST_STATUS
      78. 1 0 0 Not_testing
      79. 2 0 0 Not_testing
      80. 3 0 0 Not_testing
      81. 4 0 0 Not_testing
      82. 5 0 0 Not_testing
      83. Selective self-test flags (0x0):
      84. After scanning selected spans, do NOT read-scan remainder of disk.
      85. If Selective self-test is pending on power-up, resume after 0 minute delay.
      Display All
      doh, wrong disk. that'll teach me to just copy/paste.
    • Source Code

      1. iozone -e -I -a -s 100M -r 4k -r 1024k -i 0 -i 1 -i 2
      Please do a 'sudo apt install iozone3', then chdir to the disk in question, run the above call and post results.

      Well, here 199 looks good but IIRC those WD never increment the counter even if errors occur. Anyway: if performance is that bad I would check cabling first and then test again with the above call. This iozone call also tests random IO and here it gets interesting since based on numbers some conclusions are possible what's happening or where the bottleneck is (if it's cabling most transactions are dropped and retransmits happen due to CRC errors)
      'OMV problems' with XU4 and Cloudshell 2? Nope, please read this first.
    • Source Code

      1. IOZONE_INSTALL_PATH=$HOME
      2. IOZONE_VERSION=iozone3_429
      3. # Download and build iozone.
      4. if [ ! -f $IOZONE_INSTALL_PATH/$IOZONE_VERSION/src/current/iozone ]; then
      5. printf "Installing iozone...\n"
      6. curl "http://www.iozone.org/src/current/$IOZONE_VERSION.tar" | tar -x
      7. cd $IOZONE_VERSION/src/current
      8. make --quiet linux-arm
      9. printf "Install complete!\n\n"
      10. else
      11. cd $IOZONE_VERSION/src/current
      12. fi
      13. make install
      Display All
      Oops, I wasn't aware that OMV2 is based on such an old Debian version. Only idea left (besides checking cabling) is the above (why the hell does code always gets inserted there?). At least that's how this guy here doing a lot of useful SD card benchmarks is doing it according to raw.githubusercontent.com/geer…rks/microsd-benchmarks.sh
      'OMV problems' with XU4 and Cloudshell 2? Nope, please read this first.
    • Sorry, managed to get iozone installed by manually specifying the url to the correct stable version. Anyway, running your previous command gives the following:

      Source Code

      1. ignore this, I ran it on the wrong drive. Just waiting for the correct drive to finish (which is taking ages)
      Not really sure what I'm looking at to be honest! But your help so far has been appreciated.

      The post was edited 1 time, last by aphillippe ().

    • I've been struggling to get the test to complete. It seems to hang after the first reread test. I have left it for multiple times for > 1 hour. Is this normal?

      Here's what I get anyway:

      Source Code

      1. Iozone: Performance Test of File I/O
      2. Version $Revision: 3.429 $
      3. Compiled for 64 bit mode.
      4. Build: linux-AMD64
      5. Contributors:William Norcott, Don Capps, Isom Crawford, Kirby Collins
      6. Al Slater, Scott Rhine, Mike Wisner, Ken Goss
      7. Steve Landherr, Brad Smith, Mark Kelly, Dr. Alain CYR,
      8. Randy Dunlap, Mark Montague, Dan Million, Gavin Brebner,
      9. Jean-Marc Zucconi, Jeff Blomberg, Benny Halevy, Dave Boone,
      10. Erik Habbinga, Kris Strecker, Walter Wong, Joshua Root,
      11. Fabrice Bacchella, Zhenghua Xue, Qin Li, Darren Sawyer,
      12. Vangel Bojaxhi, Ben England, Vikentsi Lapa.
      13. Run began: Sat Apr 22 06:51:34 2017
      14. Include fsync in write timing
      15. O_DIRECT feature enabled
      16. Auto Mode
      17. File size set to 102400 kB
      18. Record Size 4 kB
      19. Record Size 1024 kB
      20. Command line used: iozone -e -I -a -s 100M -r 4k -r 1024k -i 0 -i 1 -i 2
      21. Output is in kBytes/sec
      22. Time Resolution = 0.000001 seconds.
      23. Processor cache size set to 1024 kBytes.
      24. Processor cache line size set to 32 bytes.
      25. File stride size set to 17 * record size.
      26. random random bkwd record stride
      27. kB reclen write rewrite read reread read write read rewrite read fwrite frewrite fread freread
      28. 102400 4 233 247 358 516
      Display All
    • aphillippe wrote:

      I've been struggling to get the test to complete. It seems to hang after the first reread test. I have left it for multiple times for > 1 hour. Is this normal?
      No, that's more or less an indication of something seriously wrong wrt head positioning. The random IO tests end up with the drive being busy moving the heads all the time between the various tracks and it seems there's something really wrong. At least it seems it's not a cabling issue since then sequential and random IO performance would've resulted in similar numbers.

      Those WD RED implement SMART selftests, you could try to execute 'smartctl -t short /dev/sda' (will tell you how it needs so you can check with 'smartctl -a' later) and if that works execute 'smartctl -t long /dev/sda' (might tage hours/ages, you can always check status in the meantime with 'smartctl -a /dev/sda')
      'OMV problems' with XU4 and Cloudshell 2? Nope, please read this first.
    • Ok, this is the results of running smartctl -a after five minutes. Doesn't look like the test has worked to me, but I'm obviously no expert!

      Source Code

      1. smartctl 5.41 2011-06-09 r3365 [x86_64-linux-3.2.0-4-amd64] (local build)
      2. Copyright (C) 2002-11 by Bruce Allen, http://smartmontools.sourceforge.net
      3. === START OF INFORMATION SECTION ===
      4. Model Family: Western Digital Red
      5. Device Model: WDC WD30EFRX-68EUZN0
      6. Serial Number: WD-WMC4N0312651
      7. LU WWN Device Id: 5 0014ee 6ae6a6d08
      8. Firmware Version: 80.00A80
      9. User Capacity: 3,000,592,982,016 bytes [3.00 TB]
      10. Sector Sizes: 512 bytes logical, 4096 bytes physical
      11. Device is: In smartctl database [for details use: -P show]
      12. ATA Version is: 9
      13. ATA Standard is: Exact ATA specification draft version not indicated
      14. Local Time is: Sat Apr 22 10:45:06 2017 BST
      15. SMART support is: Available - device has SMART capability.
      16. SMART support is: Enabled
      17. === START OF READ SMART DATA SECTION ===
      18. SMART overall-health self-assessment test result: PASSED
      19. General SMART Values:
      20. Offline data collection status: (0x00) Offline data collection activity
      21. was never started.
      22. Auto Offline Data Collection: Disabled.
      23. Self-test execution status: ( 241) Self-test routine in progress...
      24. 10% of test remaining.
      25. Total time to complete Offline
      26. data collection: (40560) seconds.
      27. Offline data collection
      28. capabilities: (0x7b) SMART execute Offline immediate.
      29. Auto Offline data collection on/off support.
      30. Suspend Offline collection upon new
      31. command.
      32. Offline surface scan supported.
      33. Self-test supported.
      34. Conveyance Self-test supported.
      35. Selective Self-test supported.
      36. SMART capabilities: (0x0003) Saves SMART data before entering
      37. power-saving mode.
      38. Supports SMART auto save timer.
      39. Error logging capability: (0x01) Error logging supported.
      40. General Purpose Logging supported.
      41. Short self-test routine
      42. recommended polling time: ( 2) minutes.
      43. Extended self-test routine
      44. recommended polling time: ( 255) minutes.
      45. Conveyance self-test routine
      46. recommended polling time: ( 5) minutes.
      47. SCT capabilities: (0x703d) SCT Status supported.
      48. SCT Error Recovery Control supported.
      49. SCT Feature Control supported.
      50. SCT Data Table supported.
      51. SMART Attributes Data Structure revision number: 16
      52. Vendor Specific SMART Attributes with Thresholds:
      53. ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED WHEN_FAILED RAW_VALUE
      54. 1 Raw_Read_Error_Rate 0x002f 179 179 051 Pre-fail Always - 56172
      55. 3 Spin_Up_Time 0x0027 187 180 021 Pre-fail Always - 5650
      56. 4 Start_Stop_Count 0x0032 100 100 000 Old_age Always - 63
      57. 5 Reallocated_Sector_Ct 0x0033 200 200 140 Pre-fail Always - 0
      58. 7 Seek_Error_Rate 0x002e 200 200 000 Old_age Always - 0
      59. 9 Power_On_Hours 0x0032 072 072 000 Old_age Always - 20444
      60. 10 Spin_Retry_Count 0x0032 100 253 000 Old_age Always - 0
      61. 11 Calibration_Retry_Count 0x0032 100 253 000 Old_age Always - 0
      62. 12 Power_Cycle_Count 0x0032 100 100 000 Old_age Always - 63
      63. 192 Power-Off_Retract_Count 0x0032 200 200 000 Old_age Always - 58
      64. 193 Load_Cycle_Count 0x0032 200 200 000 Old_age Always - 4
      65. 194 Temperature_Celsius 0x0022 120 105 000 Old_age Always - 30
      66. 196 Reallocated_Event_Count 0x0032 200 200 000 Old_age Always - 0
      67. 197 Current_Pending_Sector 0x0032 200 200 000 Old_age Always - 1
      68. 198 Offline_Uncorrectable 0x0030 100 253 000 Old_age Offline - 0
      69. 199 UDMA_CRC_Error_Count 0x0032 200 200 000 Old_age Always - 0
      70. 200 Multi_Zone_Error_Rate 0x0008 100 253 000 Old_age Offline - 0
      71. SMART Error Log Version: 1
      72. No Errors Logged
      73. SMART Self-test log structure revision number 1
      74. No self-tests have been logged. [To run self-tests, use: smartctl -t]
      75. SMART Selective self-test log data structure revision number 1
      76. SPAN MIN_LBA MAX_LBA CURRENT_TEST_STATUS
      77. 1 0 0 Not_testing
      78. 2 0 0 Not_testing
      79. 3 0 0 Not_testing
      80. 4 0 0 Not_testing
      81. 5 0 0 Not_testing
      82. Selective self-test flags (0x0):
      83. After scanning selected spans, do NOT read-scan remainder of disk.
      84. If Selective self-test is pending on power-up, resume after 0 minute delay.
      Display All
    • Users Online 5

      5 Guests