Sata Raid Marvell 4port 9235 rockpi4

stuartiannaylor · 4. Juli 2019

Ok got some disks 4x Integral SSD 120gb P5 as that should been the card is the bottleneck.
Couldn't afford anything more really as x4 gets a bit expensive but rockpi4b-2gb marvell 9235 m.2 4 port sata.

Staring with Debian-stretch-4.4-latest and mdadm going to try a raid10

Code

Command line used: iozone -e -I -a -s 100M -r 4k -r 16k -r 512k -r 1024k -r 16384k -i 0 -i 1 -i 2


        Output is in kBytes/sec


        Time Resolution = 0.000001 seconds.


        Processor cache size set to 1024 kBytes.


        Processor cache line size set to 32 bytes.


        File stride size set to 17 * record size.


                                                              random    random     bkwd    record    stride


              kB  reclen    write  rewrite    read    reread    read     write     read   rewrite      read   fwrite frewrite    fread  freread


          102400       4    20355    28533    43513    44568    22556    28612


          102400      16    60835    71891   111520   107540    66074    71640


          102400     512   149988   129385   253123   263113   211684   131649


          102400    1024   161360   164943   274007   275765   253893   165764


          102400   16384   181646   182851   338294   347395   342601   176768

Alles anzeigen

*********************************************************************************
RAID5

Code

rock@rockpi4:~$ sudo mdadm --create --verbose /dev/md0 --level=5 --raid-devices=4 /dev/sda /dev/sdb /dev/sdc /dev/sdd


mdadm: layout defaults to left-symmetric


mdadm: layout defaults to left-symmetric


mdadm: chunk size defaults to 512K


mdadm: /dev/sdc appears to be part of a raid array:


       level=raid0 devices=0 ctime=Thu Jan  1 00:00:00 1970


mdadm: partition table exists on /dev/sdc but will be lost or


       meaningless after creating array


mdadm: size set to 117154816K


mdadm: automatically enabling write-intent bitmap on large array


Continue creating array? y


mdadm: Defaulting to version 1.2 metadata


mdadm: array /dev/md0 started.


rock@rockpi4:~$ cat /proc/mdstat


Personalities : [raid10] [raid6] [raid5] [raid4]


md0 : active raid5 sdd[4] sdc[2] sdb[1] sda[0]


      351464448 blocks super 1.2 level 5, 512k chunk, algorithm 2 [4/3] [UUU_]


      [>....................]  recovery =  1.6% (1898560/117154816) finish=19.2min speed=99924K/sec


      bitmap: 0/1 pages [0KB], 65536KB chunk

Alles anzeigen

Code

Command line used: iozone -e -I -a -s 100M -r 4k -r 16k -r 512k -r 1024k -r 16384k -i 0 -i 1 -i 2


        Output is in kBytes/sec


        Time Resolution = 0.000001 seconds.


        Processor cache size set to 1024 kBytes.


        Processor cache line size set to 32 bytes.


        File stride size set to 17 * record size.


                                                              random    random     bkwd    record    stride


              kB  reclen    write  rewrite    read    reread    read     write     read   rewrite      read   fwrite frewrite    fread  freread


          102400       4     8159     8947    43789    42643    24543    10212


          102400      16    33078    40985    98244    98407    70763    41851


          102400     512    52870    53418   212184   202157   203772    50657


          102400    1024    66426    69555   250660   250200   249607    69539


          102400   16384   108537   112300   326090   324173   320777   106363

Alles anzeigen

RAID1

Code

rock@rockpi4:~$ sudo mdadm --create --verbose /dev/md0 --level=1 --raid-devices=2 /dev/sda /dev/sdb


mdadm: Note: this array has metadata at the start and


    may not be suitable as a boot device.  If you plan to


    store '/boot' on this device please ensure that


    your boot-loader understands md/v1.x metadata, or use


    --metadata=0.90


mdadm: size set to 117155264K


mdadm: automatically enabling write-intent bitmap on large array


Continue creating array? y


mdadm: Defaulting to version 1.2 metadata


mdadm: array /dev/md0 started.


rock@rockpi4:~$ cat /proc/mdstat


Personalities : [raid10] [raid6] [raid5] [raid4] [raid1]


md0 : active raid1 sdb[1] sda[0]


      117155264 blocks super 1.2 [2/2] [UU]


      [>....................]  resync =  2.3% (2801408/117155264) finish=8.8min speed=215492K/sec


      bitmap: 1/1 pages [4KB], 65536KB chunk






unused devices: <none>

Alles anzeigen

Code

Command line used: iozone -e -I -a -s 100M -r 4k -r 16k -r 512k -r 1024k -r 16384k -i 0 -i 1 -i 2


        Output is in kBytes/sec


        Time Resolution = 0.000001 seconds.


        Processor cache size set to 1024 kBytes.


        Processor cache line size set to 32 bytes.


        File stride size set to 17 * record size.


                                                              random    random     bkwd    record    stride


              kB  reclen    write  rewrite    read    reread    read     write     read   rewrite      read   fwrite frewrite    fread  freread


          102400       4    24759    31559    39765    41196    25476    30710


          102400      16    62662    73245   124756   125744    62209    72778


          102400     512   139397   160038   260433   261606   218154   147652


          102400    1024   165815   155189   258119   261744   232643   164702


          102400   16384   172905   186702   318211   322998   321997   170680

Alles anzeigen

RAID0

Code

rock@rockpi4:~$ sudo mdadm --create --verbose /dev/md0 --level=0 --raid-devices=4 /dev/sda /dev/sdb  /dev/sdc /dev/sdd


mdadm: chunk size defaults to 512K


mdadm: /dev/sdc appears to be part of a raid array:


       level=raid0 devices=0 ctime=Thu Jan  1 00:00:00 1970


mdadm: partition table exists on /dev/sdc but will be lost or


       meaningless after creating array


Continue creating array? y


mdadm: Defaulting to version 1.2 metadata


mdadm: array /dev/md0 started.


rock@rockpi4:~$ cat /proc/mdstat


Personalities : [raid10] [raid6] [raid5] [raid4] [raid1] [raid0]


md0 : active raid0 sdd[3] sdc[2] sdb[1] sda[0]


      468619264 blocks super 1.2 512k chunks


unused devices: <none>

Alles anzeigen

Code

Command line used: iozone -e -I -a -s 100M -r 4k -r 16k -r 512k -r 1024k -r 16384k -i 0 -i 1 -i 2


        Output is in kBytes/sec


        Time Resolution = 0.000001 seconds.


        Processor cache size set to 1024 kBytes.


        Processor cache line size set to 32 bytes.


        File stride size set to 17 * record size.


                                                              random    random     bkwd    record    stride


              kB  reclen    write  rewrite    read    reread    read     write     read   rewrite      read   fwrite frewrite    fread  freread


          102400       4    31874    42784    44859    48796    26191    42465


          102400      16    89104   112188   110570   114486    77652   111816


          102400     512   248787   259180   258800   270097   227197   229707


          102400    1024   309271   324243   293455   293122   268819   286143


          102400   16384   373574   382208   324869   326204   326070   380622

Alles anzeigen

stuartiannaylor · 4. Juli 2019

Concurrent single disks

Code

Command line used: iozone -l 4 -u 4 -r 16k -s 512M -F /home/rock/sda/tmp1 /home/rock/sdb/tmp2 /home/rock/sdc/tmp3 /home/rock/sdd/tmp4


        Output is in kBytes/sec


        Time Resolution = 0.000001 seconds.


        Processor cache size set to 1024 kBytes.


        Processor cache line size set to 32 bytes.


        File stride size set to 17 * record size.


        Min process = 4


        Max process = 4


        Throughput test with 4 processes


        Each process writes a 524288 kByte file in 16 kByte records






        Children see throughput for  4 initial writers  =  468982.85 kB/sec


        Parent sees throughput for  4 initial writers   =  391562.16 kB/sec


        Min throughput per process                      =  115979.48 kB/sec


        Max throughput per process                      =  118095.79 kB/sec


        Avg throughput per process                      =  117245.71 kB/sec


        Min xfer                                        =  513488.00 kB






        Children see throughput for  4 rewriters        =  448753.70 kB/sec


        Parent sees throughput for  4 rewriters         =  378103.46 kB/sec


        Min throughput per process                      =  108174.91 kB/sec


        Max throughput per process                      =  119841.15 kB/sec


        Avg throughput per process                      =  112188.42 kB/sec


        Min xfer                                        =  472992.00 kB






        Children see throughput for  4 readers          =  319857.60 kB/sec


        Parent sees throughput for  4 readers           =  319587.93 kB/sec


        Min throughput per process                      =   78386.40 kB/sec


        Max throughput per process                      =   81170.33 kB/sec


        Avg throughput per process                      =   79964.40 kB/sec


        Min xfer                                        =  506336.00 kB






        Children see throughput for 4 re-readers        =  331737.53 kB/sec


        Parent sees throughput for 4 re-readers         =  331539.26 kB/sec


        Min throughput per process                      =   74617.11 kB/sec


        Max throughput per process                      =   90278.13 kB/sec


        Avg throughput per process                      =   82934.38 kB/sec


        Min xfer                                        =  433360.00 kB






        Children see throughput for 4 reverse readers   =  769042.86 kB/sec


        Parent sees throughput for 4 reverse readers    =  768023.53 kB/sec


        Min throughput per process                      =   43320.77 kB/sec


        Max throughput per process                      =  262961.66 kB/sec


        Avg throughput per process                      =  192260.72 kB/sec


        Min xfer                                        =   86384.00 kB






        Children see throughput for 4 stride readers    = 1795856.09 kB/sec


        Parent sees throughput for 4 stride readers     = 1781767.61 kB/sec


        Min throughput per process                      =   65569.88 kB/sec


        Max throughput per process                      =  920383.50 kB/sec


        Avg throughput per process                      =  448964.02 kB/sec


        Min xfer                                        =   37360.00 kB






        Children see throughput for 4 random readers    = 1971409.70 kB/sec


        Parent sees throughput for 4 random readers     = 1958188.18 kB/sec


        Min throughput per process                      =   69869.92 kB/sec


        Max throughput per process                      =  861175.75 kB/sec


        Avg throughput per process                      =  492852.43 kB/sec


        Min xfer                                        =   41904.00 kB






        Children see throughput for 4 mixed workload    = 1176863.17 kB/sec


        Parent sees throughput for 4 mixed workload     =  275991.88 kB/sec


        Min throughput per process                      =   98414.23 kB/sec


        Max throughput per process                      =  606498.81 kB/sec


        Avg throughput per process                      =  294215.79 kB/sec


        Min xfer                                        =   84304.00 kB






        Children see throughput for 4 random writers    =  428459.84 kB/sec


        Parent sees throughput for 4 random writers     =  318774.34 kB/sec


        Min throughput per process                      =   96696.56 kB/sec


        Max throughput per process                      =  118440.29 kB/sec


        Avg throughput per process                      =  107114.96 kB/sec


        Min xfer                                        =  428352.00 kB






        Children see throughput for 4 pwrite writers    =  467800.79 kB/sec


        Parent sees throughput for 4 pwrite writers     =  381736.33 kB/sec


        Min throughput per process                      =  111798.68 kB/sec


        Max throughput per process                      =  120814.23 kB/sec


        Avg throughput per process                      =  116950.20 kB/sec


        Min xfer                                        =  485168.00 kB






        Children see throughput for 4 pread readers     =  309714.87 kB/sec


        Parent sees throughput for 4 pread readers      =  309501.91 kB/sec


        Min throughput per process                      =   76447.56 kB/sec


        Max throughput per process                      =   79120.13 kB/sec


        Avg throughput per process                      =   77428.72 kB/sec


        Min xfer                                        =  506592.00 kB






        Children see throughput for  4 fwriters         =  442763.85 kB/sec


        Parent sees throughput for  4 fwriters          =  373418.60 kB/sec


        Min throughput per process                      =  107828.45 kB/sec


        Max throughput per process                      =  114495.70 kB/sec


        Avg throughput per process                      =  110690.96 kB/sec


        Min xfer                                        =  524288.00 kB






        Children see throughput for  4 freaders         =  331765.48 kB/sec


        Parent sees throughput for  4 freaders          =  325459.39 kB/sec


        Min throughput per process                      =   81387.83 kB/sec


        Max throughput per process                      =   86099.32 kB/sec


        Avg throughput per process                      =   82941.37 kB/sec


        Min xfer                                        =  524288.00 kB

Alles anzeigen

single disk sda

Code

   Command line used: iozone -e -I -a -s 100M -r 4k -r 16k -r 512k -r 1024k -r 16384k -i 0 -i 1 -i 2


        Output is in kBytes/sec


        Time Resolution = 0.000001 seconds.


        Processor cache size set to 1024 kBytes.


        Processor cache line size set to 32 bytes.


        File stride size set to 17 * record size.


                                                              random    random     bkwd    record    stride


              kB  reclen    write  rewrite    read    reread    read     write     read   rewrite      read   fwrite frewrite    fread  freread


          102400       4    36038    45031    52457    52672    27342    44553


          102400      16    93224   115531   124822   114115    79868   115219


          102400     512   249415   223799   267595   273488   227651   258480


          102400    1024   259449   236700   268852   273148   242803   266988


          102400   16384   313281   317096   324922   325600   319687   267843

Alles anzeigen

`

stuartiannaylor · 5. Juli 2019

Really helps when you set PCIe2 speed and not run tests @ pcie1 speeds.

Also if you are a plonker and forget to edit `/boot/hw_intfc.conf` from `#intfc:dtoverlay=pcie-gen2` to `intfc:dtoverlay=pcie-gen2` you will be running on pcie-gen1

RAID 10

Code

Command line used: iozone -e -I -a -s 100M -r 4k -r 16k -r 512k -r 1024k -r 16384k -i 0 -i 1 -i 2


Output is in kBytes/sec


Time Resolution = 0.000001 seconds.


Processor cache size set to 1024 kBytes.


Processor cache line size set to 32 bytes.


File stride size set to 17 * record size.


random random bkwd record stride


kB reclen write rewrite read reread read write read rewrite read fwrite frewrite fread freread


102400 4 11719 15447 55220 53720 25421 12773


102400 16 39410 54840 139482 145128 81258 43792


102400 512 228002 220126 334104 339660 265930 225507


102400 1024 244376 243730 451377 462467 397566 258481


102400 16384 270088 304411 597462 610057 615669 297855

Alles anzeigen

RAID 5

Code

Command line used: iozone -e -I -a -s 100M -r 4k -r 16k -r 512k -r 1024k -r 16384k -i 0 -i 1 -i 2


Output is in kBytes/sec


Time Resolution = 0.000001 seconds.


Processor cache size set to 1024 kBytes.


Processor cache line size set to 32 bytes.


File stride size set to 17 * record size.


random random bkwd record stride


kB reclen write rewrite read reread read write read rewrite read fwrite frewrite fread freread


102400 4 6133 6251 47505 46013 25046 8190


102400 16 17103 17134 113272 133606 79753 20420


102400 512 61418 50852 241860 246467 244030 58031


102400 1024 79325 73325 363343 359830 361882 83655


102400 16384 127548 124702 625256 642094 650407 136680

Alles anzeigen

RAID 1

Code

Command line used: iozone -e -I -a -s 100M -r 4k -r 16k -r 512k -r 1024k -r 16384k -i 0 -i 1 -i 2


Output is in kBytes/sec


Time Resolution = 0.000001 seconds.


Processor cache size set to 1024 kBytes.


Processor cache line size set to 32 bytes.


File stride size set to 17 * record size.


random random bkwd record stride


kB reclen write rewrite read reread read write read rewrite read fwrite frewrite fread freread


102400 4 23713 29698 45608 45983 23657 30381


102400 16 79205 82546 138060 144557 82126 93921


102400 512 212859 221943 307613 304036 259783 179355


102400 1024 235985 243783 366101 369935 317354 198861


102400 16384 289036 290279 410520 398875 399868 295329

Alles anzeigen

RAID 0

Code

Command line used: iozone -e -I -a -s 100M -r 4k -r 16k -r 512k -r 1024k -r 16384k -i 0 -i 1 -i 2


Output is in kBytes/sec


Time Resolution = 0.000001 seconds.


Processor cache size set to 1024 kBytes.


Processor cache line size set to 32 bytes.


File stride size set to 17 * record size.


random random bkwd record stride


kB reclen write rewrite read reread read write read rewrite read fwrite frewrite fread freread


102400 4 33519 47927 52701 51023 26700 46382


102400 16 105763 132604 138080 155514 87026 135111


102400 512 276220 320320 311343 294629 267624 335363


102400 1024 493565 522038 463105 470833 398584 522560


102400 16384 687516 701200 625733 623531 555318 681535

Alles anzeigen

stuartiannaylor · 5. Juli 2019

4 individual disk concurrent

Code

Command line used: iozone -l 4 -u 4 -r 16k -s 512M -F /srv/dev-disk-by-label-sda/tmp1 /srv/dev-disk-by-label-sdb/tmp2 /srv/dev-disk-by-label-sdc/tmp3 /srv/dev-disk-by-label-sdd/tmp4


Output is in kBytes/sec


Time Resolution = 0.000001 seconds.


Processor cache size set to 1024 kBytes.


Processor cache line size set to 32 bytes.


File stride size set to 17 * record size.


Min process = 4


Max process = 4


Throughput test with 4 processes


Each process writes a 524288 kByte file in 16 kByte records






Children see throughput for 4 initial writers = 884590.91 kB/sec


Parent sees throughput for 4 initial writers = 701620.17 kB/sec


Min throughput per process = 195561.27 kB/sec


Max throughput per process = 234457.59 kB/sec


Avg throughput per process = 221147.73 kB/sec


Min xfer = 437344.00 kB






Children see throughput for 4 rewriters = 822771.77 kB/sec


Parent sees throughput for 4 rewriters = 701488.29 kB/sec


Min throughput per process = 180381.25 kB/sec


Max throughput per process = 232223.50 kB/sec


Avg throughput per process = 205692.94 kB/sec


Min xfer = 408720.00 kB






Children see throughput for 4 readers = 755252.30 kB/sec


Parent sees throughput for 4 readers = 753357.02 kB/sec


Min throughput per process = 169105.11 kB/sec


Max throughput per process = 198976.81 kB/sec


Avg throughput per process = 188813.07 kB/sec


Min xfer = 445664.00 kB






Children see throughput for 4 re-readers = 753492.39 kB/sec


Parent sees throughput for 4 re-readers = 750353.64 kB/sec


Min throughput per process = 160626.64 kB/sec


Max throughput per process = 201223.11 kB/sec


Avg throughput per process = 188373.10 kB/sec


Min xfer = 418528.00 kB






Children see throughput for 4 reverse readers = 780261.86 kB/sec


Parent sees throughput for 4 reverse readers = 778761.55 kB/sec


Min throughput per process = 58371.02 kB/sec


Max throughput per process = 254657.08 kB/sec


Avg throughput per process = 195065.47 kB/sec


Min xfer = 120192.00 kB






Children see throughput for 4 stride readers = 317923.62 kB/sec


Parent sees throughput for 4 stride readers = 316905.36 kB/sec


Min throughput per process = 63171.63 kB/sec


Max throughput per process = 98114.27 kB/sec


Avg throughput per process = 79480.91 kB/sec


Min xfer = 337600.00 kB






Children see throughput for 4 random readers = 798898.78 kB/sec


Parent sees throughput for 4 random readers = 794905.95 kB/sec


Min throughput per process = 57059.89 kB/sec


Max throughput per process = 391248.59 kB/sec


Avg throughput per process = 199724.70 kB/sec


Min xfer = 76480.00 kB






Children see throughput for 4 mixed workload = 647158.06 kB/sec


Parent sees throughput for 4 mixed workload = 491223.65 kB/sec


Min throughput per process = 28319.04 kB/sec


Max throughput per process = 305288.75 kB/sec


Avg throughput per process = 161789.51 kB/sec


Min xfer = 48720.00 kB






Children see throughput for 4 random writers = 734947.98 kB/sec


Parent sees throughput for 4 random writers = 544531.66 kB/sec


Min throughput per process = 167241.00 kB/sec


Max throughput per process = 207134.38 kB/sec


Avg throughput per process = 183737.00 kB/sec


Min xfer = 424704.00 kB






Children see throughput for 4 pwrite writers = 879712.72 kB/sec


Parent sees throughput for 4 pwrite writers = 686621.58 kB/sec


Min throughput per process = 186624.69 kB/sec


Max throughput per process = 236047.30 kB/sec


Avg throughput per process = 219928.18 kB/sec


Min xfer = 415856.00 kB






Children see throughput for 4 pread readers = 777243.34 kB/sec


Parent sees throughput for 4 pread readers = 773302.81 kB/sec


Min throughput per process = 184983.08 kB/sec


Max throughput per process = 203392.77 kB/sec


Avg throughput per process = 194310.84 kB/sec


Min xfer = 476896.00 kB






Children see throughput for 4 fwriters = 820877.50 kB/sec


Parent sees throughput for 4 fwriters = 693823.17 kB/sec


Min throughput per process = 194228.28 kB/sec


Max throughput per process = 217311.28 kB/sec


Avg throughput per process = 205219.38 kB/sec


Min xfer = 524288.00 kB






Children see throughput for 4 freaders = 1924029.62 kB/sec


Parent sees throughput for 4 freaders = 1071393.99 kB/sec


Min throughput per process = 268087.50 kB/sec


Max throughput per process = 970331.94 kB/sec


Avg throughput per process = 481007.41 kB/sec


Min xfer = 524288.00 kB

Alles anzeigen

Single disk sda reference

Code

Command line used: iozone -e -I -a -s 100M -r 4k -r 16k -r 512k -r 1024k -r 16384k -i 0 -i 1 -i 2


Output is in kBytes/sec


Time Resolution = 0.000001 seconds.


Processor cache size set to 1024 kBytes.


Processor cache line size set to 32 bytes.


File stride size set to 17 * record size.


random random bkwd record stride


kB reclen write rewrite read reread read write read rewrite read fwrite frewrite fread freread


102400 4 35191 45728 56689 53307 27889 48508


102400 16 104379 122405 154385 157484 88670 113964


102400 512 315788 347042 351932 348604 271399 288430


102400 1024 358399 366194 388893 379453 338470 369888


102400 16384 353154 443256 425396 422384 410580 444530

Alles anzeigen

Jetzt mitmachen!