RAID 5 slow write speeds

    • OMV 3.x
    • RAID 5 slow write speeds

      Hi everybody,

      I'm currently experiencing some heavy problems with my RAID 5 BTRFS.

      Background:

      I started with a ext4 RAID 1 (2 disks, 3TB)
      One week ago I added one more disk and converted to RAID 5 (with some problems during conversion, but solved)

      After that, I converted the ext4 file system to BTRFS and removed conversion snapshot.


      Now the write speed of the RAID 5 is REALLY slow - even below 10 MB/s.
      CPU load seems to be ok (about 20%).

      dbench shows only ~0.01 MB/s:

      Shell-Script

      1. [bbsan@omvserver volume1]$ dbench -D /media/b3183f09-e671-4dbe-b21f-d84fc12e9b97 -t 20 2
      2. dbench version 4.00 - Copyright Andrew Tridgell 1999-2004
      3. Running for 20 seconds with load '/usr/share/dbench/client.txt' and minimum warmup 4 secs
      4. 0 of 2 processes prepared for launch 0 sec
      5. 2 of 2 processes prepared for launch 0 sec
      6. releasing clients
      7. 2 664 42.32 MB/sec warmup 1 sec latency 982.196 ms
      8. 2 664 21.16 MB/sec warmup 2 sec latency 1982.244 ms
      9. 2 664 14.11 MB/sec warmup 3 sec latency 2982.285 ms
      10. 2 664 0.00 MB/sec execute 1 sec latency 4982.337 ms
      11. 2 667 0.01 MB/sec execute 2 sec latency 5389.880 ms
      12. 2 667 0.00 MB/sec execute 3 sec latency 1592.540 ms
      13. 2 667 0.00 MB/sec execute 4 sec latency 2592.588 ms
      14. 2 667 0.00 MB/sec execute 5 sec latency 3592.632 ms
      15. 2 667 0.00 MB/sec execute 6 sec latency 4592.666 ms
      16. 2 667 0.00 MB/sec execute 7 sec latency 5592.727 ms
      17. 2 698 0.02 MB/sec execute 8 sec latency 5743.095 ms
      18. 2 698 0.02 MB/sec execute 9 sec latency 1404.055 ms
      19. 2 698 0.02 MB/sec execute 10 sec latency 2404.108 ms
      20. 2 698 0.01 MB/sec execute 11 sec latency 3404.164 ms
      21. 2 698 0.01 MB/sec execute 12 sec latency 4404.214 ms
      22. 2 723 0.02 MB/sec execute 13 sec latency 4996.759 ms
      23. 2 724 0.02 MB/sec execute 14 sec latency 1051.250 ms
      24. 2 724 0.02 MB/sec execute 15 sec latency 2051.286 ms
      25. 2 724 0.02 MB/sec execute 16 sec latency 3051.320 ms
      26. 2 724 0.02 MB/sec execute 17 sec latency 4051.372 ms
      27. 2 724 0.02 MB/sec execute 18 sec latency 5051.433 ms
      28. 2 724 0.02 MB/sec execute 19 sec latency 6051.478 ms
      29. 2 cleanup 20 sec
      30. 2 cleanup 21 sec
      31. 2 cleanup 22 sec
      32. 2 cleanup 23 sec
      33. 0 cleanup 24 sec
      34. Operation Count AvgLat MaxLat
      35. ----------------------------------------
      36. NTCreateX 20 1.447 28.737
      37. Close 20 0.002 0.008
      38. Rename 2 0.045 0.049
      39. Unlink 2 0.382 0.735
      40. Qpathinfo 8 0.003 0.006
      41. Qfileinfo 5 0.003 0.006
      42. Qfsinfo 6 0.113 0.168
      43. Sfileinfo 4 81.523 326.061
      44. Find 4 0.034 0.050
      45. WriteX 24 4.503 102.366
      46. ReadX 12 0.003 0.007
      47. Flush 18 3032.779 10277.908
      48. Throughput 0.015008 MB/sec 2 clients 2 procs max_latency=6051.478 ms
      Display All



      mdstat:


      Shell-Script

      1. [bbsan@omvserver volume1]$ cat /proc/mdstat
      2. Personalities : [raid6] [raid5] [raid4]
      3. md127 : active raid5 sdb[0] sda[3] sdc[2]
      4. 5860270080 blocks super 1.2 level 5, 512k chunk, algorithm 2 [3/3] [UUU]
      5. unused devices: <none>

      Any suggestions?
    • That wasn't clear from reading the first half of your post. What kind of hardware is this on? Are you using the backports kernel?
      omv 4.0.5 arrakis | 64 bit | 4.12 backports kernel | omvextrasorg 4.0.4
      omv-extras.org plugins source code and issue tracker - github.com/OpenMediaVault-Plugin-Developers

      Please don't PM for support... Too many PMs!