Hi All,
This thread is continue of a topic i opened recently regrading Using 10g iscsi over xenserver Sr iscsi
last time i did tested iscsi with OMV , i got very poor performance , two day ago i reinstalled OMV and started to test again same scenario and the result got little bit better
as shown below , the volume i have exported is raiz1 based ZFS with following below settings, I also tweaked the network card of both sides xen host and omv to following below
Now my question is there iscsi specific setting to max out the 10G network?
Thanks please advice
ZFS OPTIONS
Code
NAME PROPERTY VALUE SOURCE
po01 type filesystem -
po01 creation Sun Nov 1 11:15 2015 -
po01 used 619G -
po01 available 1.50T -
po01 referenced 19K -
po01 compressratio 1.01x -
po01 mounted yes -
po01 quota none default
po01 reservation none default
po01 recordsize 128K default
po01 mountpoint /po01 default
po01 sharenfs off default
po01 checksum on default
po01 compression lz4 local
po01 atime off local
po01 devices on default
po01 exec on default
po01 setuid on default
po01 readonly off default
po01 zoned off default
po01 snapdir hidden default
po01 aclinherit restricted default
po01 canmount on default
po01 xattr on default
po01 copies 1 default
po01 version 5 -
po01 utf8only off -
po01 normalization none -
po01 casesensitivity sensitive -
po01 vscan off default
po01 nbmand off default
po01 sharesmb off default
po01 refquota none default
po01 refreservation none default
po01 primarycache all default
po01 secondarycache all default
po01 usedbysnapshots 0 -
po01 usedbydataset 19K -
po01 usedbychildren 619G -
po01 usedbyrefreservation 0 -
po01 logbias latency default
po01 dedup off default
po01 mlslabel none default
po01 sync standard default
po01 refcompressratio 1.00x -
po01 written 19K -
po01 logicalused 8.13G -
po01 logicalreferenced 9.50K -
po01 filesystem_limit none default
po01 snapshot_limit none default
po01 filesystem_count none default
po01 snapshot_count none default
po01 snapdev hidden default
po01 acltype off default
po01 context none default
po01 fscontext none default
po01 defcontext none default
po01 rootcontext none default
po01 relatime off default
po01 redundant_metadata all default
po01 overlay off default
po01/iscsivol type volume -
po01/iscsivol creation Mon Nov 2 3:40 2015 -
po01/iscsivol used 619G -
po01/iscsivol available 2.09T -
po01/iscsivol referenced 8.07G -
po01/iscsivol compressratio 1.01x -
po01/iscsivol reservation none default
po01/iscsivol volsize 600G local
po01/iscsivol volblocksize 8K -
po01/iscsivol checksum on default
po01/iscsivol compression lz4 inherited from po01
po01/iscsivol readonly off default
po01/iscsivol copies 1 default
po01/iscsivol refreservation 619G local
po01/iscsivol primarycache all default
po01/iscsivol secondarycache all default
po01/iscsivol usedbysnapshots 0 -
po01/iscsivol usedbydataset 8.07G -
po01/iscsivol usedbychildren 0 -
po01/iscsivol usedbyrefreservation 611G -
po01/iscsivol logbias latency default
po01/iscsivol dedup off default
po01/iscsivol mlslabel none default
po01/iscsivol sync standard default
po01/iscsivol refcompressratio 1.01x -
po01/iscsivol written 8.07G -
po01/iscsivol logicalused 8.12G -
po01/iscsivol logicalreferenced 8.12G -
po01/iscsivol snapshot_limit none default
po01/iscsivol snapshot_count none default
po01/iscsivol snapdev hidden default
po01/iscsivol context none default
po01/iscsivol fscontext none default
po01/iscsivol defcontext none default
po01/iscsivol rootcontext none default
po01/iscsivol redundant_metadata all default
po01/iscsivol shareiscsi off default
Alles anzeigen
last tested performance
Code
----------------------------------------------------------------------
CrystalDiskMark 5.0.2 x64 (C) 2007-2015 hiyohiyo
Crystal Dew World : http://crystalmark.info/
-----------------------------------------------------------------------
* MB/s = 1,000,000 bytes/s [SATA/600 = 600,000,000 bytes/s]
* KB = 1000 bytes, KiB = 1024 bytes
Sequential Read (Q= 32,T= 1) : 411.012 MB/s
Sequential Write (Q= 32,T= 1) : 351.445 MB/s
Random Read 4KiB (Q= 32,T= 1) : 122.886 MB/s [ 30001.5 IOPS]
Random Write 4KiB (Q= 32,T= 1) : 115.293 MB/s [ 28147.7 IOPS]
Sequential Read (T= 1) : 332.592 MB/s
Sequential Write (T= 1) : 225.649 MB/s
Random Read 4KiB (Q= 1,T= 1) : 9.992 MB/s [ 2439.5 IOPS]
Random Write 4KiB (Q= 1,T= 1) : 7.127 MB/s [ 1740.0 IOPS]
Test : 1024 MiB [E: 0.0% (0.2/500.0 GiB)] (x1) [Interval=5 sec]
Date : 2015/11/03 12:58:26
OS : Windows Server 2012 R2 Server Standard (full installation) [6.3 Build 9600] (x64)
Alles anzeigen
Code
# -- tuning -- #
# Increase system file descriptor limit
fs.file-max = 65535
# Increase system IP port range to allow for more concurrent connections
net.ipv4.ip_local_port_range = 1024 65000
# -- 10gbe tuning from Intel ixgb driver README -- #
# turn off selective ACK and timestamps
net.ipv4.tcp_sack = 0
net.ipv4.tcp_timestamps = 0
# memory allocation min/pressure/max.
# read buffer, write buffer, and buffer space
net.ipv4.tcp_rmem = 10000000 10000000 10000000
net.ipv4.tcp_wmem = 10000000 10000000 10000000
net.ipv4.tcp_mem = 10000000 10000000 10000000
net.core.rmem_max = 524287
net.core.wmem_max = 524287
net.core.rmem_default = 524287
net.core.wmem_default = 524287
net.core.optmem_max = 524287
net.core.netdev_max_backlog = 300000
Alles anzeigen