Hello.
I just installed OMV and did a SMB transfer that resulted in speeds from 7MB/s to 9MB/s, which is much slower than acceptable. I unfortunately started out by troubleshooting network connections, replacing switches, etc. However, when that didn't change anything, I started examining the raid setup itself.
Here are my basic specs:
I have RAID5 with EXT4 with 4x2TB Red NAS drives and an SSD for the OS. I came over from FreeNAS, where I was getting 40MB/s (at least), but switched because I was using ZFS without ECC RAM and my processor was a little underpowered (AMD c-60 APU) and was maxing out when transferring via SMB. I have 16GB of RAM in the machine.
After searching the forums I ran a few basic speed tests on the RAID itself, with odd results:
root@OMV-Server:/media/4f85211f-0972-4eb4-8ce3-ac7664a3cd4f# dd if=/dev/zero of=tempfile bs=1MB count=1024
1024+0 records in
1024+0 records out
1024000000 bytes (1.0 GB) copied, 3.51127 s, 292 MB/s
root@OMV-Server:/media/4f85211f-0972-4eb4-8ce3-ac7664a3cd4f# dd if=/dev/zero of=tempfile bs=1MB count=2048
2048+0 records in
2048+0 records out
2048000000 bytes (2.0 GB) copied, 377.226 s, 5.4 MB/s
root@OMV-Server:/media/4f85211f-0972-4eb4-8ce3-ac7664a3cd4f# dd if=/dev/zero of=tempfile bs=1MB count=1024
1024+0 records in
1024+0 records out
1024000000 bytes (1.0 GB) copied, 185.531 s, 5.5 MB/s
Alles anzeigen
These tests were all ran directly after eachother, nothing changed in the configuration. It wasn't a fresh reboot of the server either. Going from ~290MB/S to ~5.5MB/S is a LARGE drop...
EDIT:
Just ran the hdparm command on the RAID and found this which is almost exactly the transfer speed I was getting over the network:
I delved a little further, finding a post by datadigger on tests to run:
lspci -v
00:11.0 SATA controller: Advanced Micro Devices [AMD] nee ATI SB7x0/SB8x0/SB9x0 SATA Controller [IDE mode] (rev 40) (prog-if 01 [AHCI 1.0])
Subsystem: ASUSTeK Computer Inc. Device 8496
Flags: bus master, 66MHz, medium devsel, latency 32, IRQ 19
I/O ports at f190 [size=8]
I/O ports at f180 [size=4]
I/O ports at f170 [size=8]
I/O ports at f160 [size=4]
I/O ports at f150 [size=16]
Memory at feb4b000 (32-bit, non-prefetchable) [size=1K]
Capabilities: [70] SATA HBA v1.0
Capabilities: [a4] PCI Advanced Features
Kernel driver in use: ahci
root@OMV-Server:~# hdparm -I /dev/sda | grep -i speed
* Gen1 signaling speed (1.5Gb/s)
* Gen2 signaling speed (3.0Gb/s)
* Gen3 signaling speed (6.0Gb/s)
root@OMV-Server:~# hdparm -I /dev/sdb | grep -i speed
* Gen1 signaling speed (1.5Gb/s)
* Gen2 signaling speed (3.0Gb/s)
* Gen3 signaling speed (6.0Gb/s)
root@OMV-Server:~# hdparm -I /dev/sdc | grep -i speed
* Gen1 signaling speed (1.5Gb/s)
* Gen2 signaling speed (3.0Gb/s)
* Gen3 signaling speed (6.0Gb/s)
root@OMV-Server:~# hdparm -I /dev/sdd | grep -i speed
* Gen1 signaling speed (1.5Gb/s)
* Gen2 signaling speed (3.0Gb/s)
* Gen3 signaling speed (6.0Gb/s)
root@OMV-Server:~# hdparm -I /dev/sde | grep -i speed
* Gen1 signaling speed (1.5Gb/s)
* Gen2 signaling speed (3.0Gb/s)
* Gen3 signaling speed (6.0Gb/s)
root@OMV-Server:~# egrep 'ata[0-9]\.|SATA link up' /var/log/dmesg
[ 3.124920] ata5.00: ATA-9: WDC WD20EFRX-68AX9N0, 80.00A80, max UDMA/133
[ 3.124929] ata5.00: 3907029168 sectors, multi 16: LBA48 NCQ (depth 0/32)
[ 3.125166] ata5.01: ATA-9: WDC WD20EFRX-68AX9N0, 80.00A80, max UDMA/133
[ 3.125175] ata5.01: 3907029168 sectors, multi 16: LBA48 NCQ (depth 0/32)
[ 3.125190] ata5.00: limited to UDMA/33 due to 40-wire cable
[ 3.125196] ata5.01: limited to UDMA/33 due to 40-wire cable
[ 3.132901] ata5.00: configured for UDMA/33
[ 3.140849] ata5.01: configured for UDMA/33
[ 3.416269] ata1: SATA link up 6.0 Gbps (SStatus 133 SControl 300)
[ 3.416324] ata4: SATA link up 6.0 Gbps (SStatus 133 SControl 300)
[ 3.416365] ata3: SATA link up 6.0 Gbps (SStatus 133 SControl 300)
[ 3.417163] ata4.00: ATA-9: WDC WD20EFRX-68AX9N0, 80.00A80, max UDMA/133
[ 3.417178] ata4.00: 3907029168 sectors, multi 16: LBA48 NCQ (depth 31/32), AA
[ 3.417330] ata1.00: ATA-9: SanDisk SDSSDHII240G, X31200RL, max UDMA/133
[ 3.417341] ata1.00: 468862128 sectors, multi 1: LBA48 NCQ (depth 31/32), AA
[ 3.417981] ata3.00: ATA-9: WDC WD20EFRX-68AX9N0, 80.00A80, max UDMA/133
[ 3.418008] ata3.00: 3907029168 sectors, multi 16: LBA48 NCQ (depth 31/32), AA
[ 3.418186] ata4.00: configured for UDMA/133
[ 3.418388] ata1.00: configured for UDMA/133
[ 3.418863] ata3.00: configured for UDMA/133
Alles anzeigen
Now, it seemed odd that two of my drives were limited to UDMA/33, but datadigger had the same output (I think he might have the same board) and he said the speed didn't differ on those disks from the others, so "came to the conclusion that the board maker has used a port multiplier with different capabilities as the other four SATA ports and they seem to run on PATA-mode (A 40-wire cable is an old style flat cable)."
Also "AA" isn't enabled on those two drives, which may be a problem?
Next, he ran speed tests on individual disks, where I once again had another interesting result.
[/coroot@OMV-Server:~# dd conv=fdatasync if=/dev/sdc of=/home/test.imf bs=1MB count=1024
1024+0 records in
1024+0 records out
1024000000 bytes (1.0 GB) copied, 10.3246 s, 99.2 MB/s
root@OMV-Server:~# dd conv=fdatasync if=/dev/sdb of=/home/test.imf bs=1MB count=1024
1024+0 records in
1024+0 records out
1024000000 bytes (1.0 GB) copied, 10.4696 s, 97.8 MB/s
root@OMV-Server:~# dd conv=fdatasync if=/dev/sdd of=/home/test.imf bs=1MB count=1024
1024+0 records in
1024+0 records out
1024000000 bytes (1.0 GB) copied, 510.71 s, 2.0 MB/s
root@OMV-Server:~# dd conv=fdatasync if=/dev/sde of=/home/test.imf bs=1MB count=1024
1024+0 records in
1024+0 records out
1024000000 bytes (1.0 GB) copied, 10.6054 s, 96.6 MB/s
root@OMV-Server:~# dd conv=fdatasync if=/dev/sdd of=/home/test.imf bs=1MB count=1024
1024+0 records in
1024+0 records out
1024000000 bytes (1.0 GB) copied, 8.03321 s, 127 MB/s
root@OMV-Server:~# dd conv=fdatasync if=/dev/sdc of=/home/test.imf bs=1MB count=1024
1024+0 records in
1024+0 records out
1024000000 bytes (1.0 GB) copied, 8.1168 s, 126 MB/s
root@OMV-Server:~# dd conv=fdatasync if=/dev/sde of=/home/test.imf bs=1MB count=1024
1024+0 records in
1024+0 records out
1024000000 bytes (1.0 GB) copied, 8.05182 s, 127 MB/s
root@OMV-Server:~# dd conv=fdatasync if=/dev/sdd of=/home/test.imf bs=1MB count=1024
1024+0 records in
1024+0 records out
1024000000 bytes (1.0 GB) copied, 8.01716 s, 128 MB/s
Alles anzeigen
Notice that ONE TIME sdd had a copy speed of 2.0MB/S. However, all the times after that it worked fine! Does anyone know what might be going on here??