UPDATED:3/26/2014
I CHANGED MANY THINGS WITH THE LAST UPDATE OF THIS GUIDE
- I removed the openiscsi portion of the guide, I still have some of the packages installed. I have to try this on a clean system to be sure if they are needed or not.
- I added the additional steps you need to add SCST's iSCSI.
- Startup uses init.d instead of rc.local
SCST does everything openiscsi does and more. SCST can also be used to present iSCSI as well as FC. So it's a very handy package on a storage array. SCST is supposed to be faster as well. For more info see:
http://scst.sourceforge.net/comparison.html
Donh has been awesome in pointing things out to me. I set things up and never went back to try it again, so he has been my "beta tester". --==THANK YOU Donh!==-- I hope you like the new changes I've made!
For this I used a qLogic 2460 card. Some are branded DELL or HP, they should all be the same. This one is a single port card, but there are 2 and 4 port cards as well. With the multi-port cards, you will have multiple WWN's.
You will see some DD commands to make blank image files or add to them. You can make them whatever size you need, you dont have to use the examples.
Also, in ESXi I was able to get my FC to fail-over to iSCSI.
Update OMV + debian
Add non-free contrib to sources.
deb http://debian.uchicago.edu/debian/ squeeze main non-free contrib
deb-src http://debian.uchicago.edu/debian/ squeeze main non-free contrib
deb http://security.debian.org/ squeeze/updates main contrib non-free
deb-src http://security.debian.org/ squeeze/updates main contrib non-free
# squeeze-updates, previously known as 'volatile'
deb http://debian.uchicago.edu/debian/ squeeze-updates main contrib non-free
deb-src http://debian.uchicago.edu/debian/ squeeze-updates main contrib non-free
Alles anzeigen
** Configure your drives or RAID in the GUI now **
The following commands check rebuild speed setting, then increase raid rebuild speed setting. In a production system, set these to something that won't bog you down during a rebuild, but during the initial build this will help.
sysctl dev.raid.speed_limit_min
sysctl dev.raid.speed_limit_max
sysctl -w dev.raid.speed_limit_max=20000000
SSD, FS, TCP tweaks
For me, I used a small 40gb intel SSD for OS. Not the fastest, but perfect for OMV.
SSD fstab options: discard,noatime,nodiratime
discard enables TRIM.
noatime, nodirtime disables inode time updates to reduce writes. (optional)
Example:
# / was on /dev/sda1 during installation
UUID=xxxxxxx / ext4 discard,noatime,nodiratime errors=remount-ro 0 1
A handy disk tool:
Next install IOtop. "iotop –only" is your friend. This shows what is using your drives and how fast.
Add to bottom of sysctl.conf, TCP speed bump:
net.ipv4.tcp_window_scaling = 1
net.ipv4.tcp_syncookies = 1
net.core.rmem_max = 16777216
net.core.wmem_max = 16777216
net.ipv4.tcp_rmem = 4096 87380 16777216
net.ipv4.tcp_wmem = 4096 65536 16777216
net.ipv4.tcp_no_metrics_save = 1
net.ipv4.tcp_moderate_rcvbuf = 1
net.core.netdev_max_backlog = 2500
Apt-get install lotsofstuff
Prereqs for later on and a few tools
apt-get install fakeroot kernel-wedge build-essential makedumpfile kernel-package libncurses5 libncurses5-dev gcc libncurses5-dev linux-headers-$(uname -r) lsscsi patch subversion
apt-get build-dep --no-install-recommends linux-image-$(uname -r)
apt-get install iscsitarget-dkms iscsitarget-source iscsitarget iscsitarget-module-dkms lsscsi less openmediavault-iscsitarget dnsutils pciutils firmware-qlogic
Check for qlogic firmware
Sync to SCST trunk with subversion, this should make a folder called "trunk"
Unload orig-driver, then blacklist it
Start build
Install linux source code, extract it
Link extracted source to linux folder (for ease)
Scst patches to kernel:
patch -p1 < /root/trunk/scst/kernel/scst_exec_req_fifo-2.6.32.patch
patch -p1 < /root/trunk/iscsi-scst/kernel/patches/put_page_callback-2.6.32.patch
Get current config for kernel, copy it to working kernel source for build
Rename old driver
Link new driver to source build folder
export is to speed up build (use # of cores +1)
In make menuconfig, make sure both are enabled:
Device Drivers> SCSI device support> SCSI low level drivers> [*]Qlogic 2xxx target mode support
Networking support> Networking options> [*]TCP/IP zero-copy transfer completion notification
Compile and make .deb's. Then you can install these later in another system
fakeroot make-kpkg --initrd --append-to-version=-scst kernel-image kernel-headers
cd /usr/src/
ls
Install newly compiled kernel
dpkg -i linux-image-2.6.32-scst_2.6.32-scst-10.00.Custom_amd64.deb
dpkg -i linux-headers-2.6.32-scst_2.6.32-scst-10.00.Custom_amd64.deb
Update ram images
Select new Kernel on boot!
Change GRUB_DEFAULT=0 to the entry on the boot menu, I used GRUB_DEFAULT=2 (The third one down). Then update grub.
Reboot to make sure it works
Build SCST + Kernel Modules
This installs EVERYTHING for SCST. Not a big deal, but you can be more specific. Then we check that the modules are present for the current working kernel.
Setup SCST service. Ignore warning for now.
Run "sysv-rc-conf" and find "scst", put an X with the space bar under 2 3 4 5. Press "q" to quit.
SCST Config
Get FC WWN: (target, i.e. SAN)
Create SCST config
Example config, use the right UUID's, WWN's. The iSCSI target has a group assigned for security. You can do the same for fiber, but it's not needed for an A to B setup. You will also notice that I have disk2 for the "TARGET_DRIVER qla2x00t" and for "TARGET_DRIVER iscsi". When you use this with ESXi, you get failover if the fiber fails. There are also mechanisms to password protect iSCSI, see the docs for SCST
vdisk_fileio attributes
blocksize 512, specifies block size, must be power of 2 and >= 512 bytes. Default is 512. (dont change with a formatted lun)
zero_copy 1, if set, device uses zero copy access to the page cache
rotational 1, if set disk is mechanical, not ssd
nv_cache 1, enables write through cache
filename X, location of file to write
removable 1, if set device is removeable (cdrom)
read_only 1, if set would make LUN read only.
thin_provisioned 1, if set device is thin disk
HANDLER vdisk_fileio {
DEVICE disk1 {
filename /media/--==UUID==--/FC/disk1
nv_cache 1
}
DEVICE disk2 {
filename /media/--==UUID==--/FC/disk2
nv_cache 1
}
DEVICE disk3 {
filename /media/--==UUID==--/FC/disk3
nv_cache 1
}
}
TARGET_DRIVER qla2x00t {
enabled 1
TARGET 00:00:00:00:00:00:00:00 {
enabled 1
LUN 0 disk1
LUN 1 disk2
}
}
TARGET_DRIVER iscsi {
enabled 1
TARGET iqn.0000-00.lucid:iscsi.lun0 {
enabled 1
rel_tgt_id 1
GROUP ESXi {
LUN 0 disk2
LUN 1 disk3
INITIATOR iqn.0000-00.com.vmware:esxi-00000000
}
}
}
Alles anzeigen
Make Drive Image For Target
It is possible to use /dev/sda or /dev/md, and give the target the whole drive/array. See SCST manual. You could also make an image and mount that with a loopback and present it that way, just to give you a taste of what you can do. Read the SCST docs for more. I prefer working with images.
Make a place for the images
Example DD commands.
Make 512Mb
Make 50Gb
Add 1Gb to disk1 (with zeros)
Make 512mb Sparse (thin) disk. (not reccomended, for speed)
Increase size to 1Gb (sparse)
This should start scst
This should restart scst
If you add a disk or resize one, update with this:
Reboot and make sure it comes back up as intended
Done!!