Setting Up Fibre Channel

  • I already have a couple of Fibre Channel Cards I took out of a server, and I've been offered a HP StorageWorks AD542B Storage Array Shelf complete with drives, for what I consider a fair price.


    The Shelf comes with
    PSU
    FCAL A AD623C IO Board
    FCAL B AD624C IO Board
    AD625B Controller


    Obviously I will need some cables.


    I have never used fibre channel before, and it does seem to be abit of a black art. Does anybody know if this lost going to work together, what I need to check etc, before I waste any money.


    And is it going to be possible for me to connect this lot up to my OpenMediaVault box, and get them to play ball?

  • What fibre Channel Cards do you have?


    That is a Shelf of an EVA Storage Array and there are nearly no information available regarding those shelfs.


    Can you please take a picture from the interface cards (please focus on the connectors) to find out what it could be?


    Then I could tell you, what kind of cabling, you need.


    Ser

    Everything is possible, sometimes it requires Google to find out how.

  • The Emulex Cards needs drivers from emulex. You can download them from http://www.emulex.com and there are also drivers for debian squeeze (6.0.x). I am not sure if the LP10ks are still in support and you can get any drivers, you need to give it a try. Howevery those cards require PCI-X ports, which may or may not exist in your "Server".
    Get your search started here:
    http://www.emulex.com/products…lse-lp10000/overview.html


    The question of Interface did go into the direction of
    FCAL A AD623C IO Board
    FCAL B AD624C IO Board


    I have no clue and there is no information available, what they can do. They could be copper based interface cards or fibre based cards (don't get confused by the word fibre channel, it also exists for copper cabling inside of storage arrays).


    So before I can tell you what cable you actually need, I need to know the connections of the FCAL IO Boards.


    Definately I can say now - you only need one Emulex controller. No need to put in two and I am also not sure if you can use both. Normally you then need Multipathing capable storage hardware (which you do not have AFAIK) and also then need to deploy MP drivers, that are able to handle it (available in Linux kernel 3.x by default, but not likely by 2.x.32).


    /EDIT:
    Maybe you are in luck and do not need any additional drivers. lpfc IS part of the 2.6.32-05 kernel of debian and there is a module for it. Maybe you need to load the module manually (should be identified on boot during hardware probe).

    Everything is possible, sometimes it requires Google to find out how.

  • I found this image not sure if it's enough to help identify the connectors, I believe although I'm not certain as I haven't yet purchased the kit, that the A+B interfaces are fairly similar but the shelf is equipped with 2 so that it can be part of a loop?




    I think I'll go ahead and purchase the drives and shelves, as it not a ridiculous amount of money, and could be a huge saving it it all works together. I'm not 100% sure my current Server has PCI-X slots without looking at the manual or the board. However I do defintely have a Dual 32 Bit Xeon, or a Quad 32bit Xeon sat on the shelf here, that do have PCI-X, that I could use to setup the system for testing before I dismantle the existing OMV for a rebuild.

  • Okay, if your version looks the same (and I would wait for the real thing to get to you before getting cabling), then you would need a fibre cable multimode wiht LC-LC connectors (means LC connector on both ends).


    For a loop, you do not need both adapters. I am not sure if you can use both adapters. If you can use them, you could potentially use them active/active, which would create multipathing to each physical disk. And THAT would enable a complete additional layer of failover capability. You may not require a loop config, cause you can simply setup each HBA to one of the interfaces. That should do the redundancy. I am also not sure if you can loop everything up including the two HBAs in one single AL.


    However, it is a very professional grade piece of hardware (even a little bit aged) and you can get a very good system for small money. Just give it a try, should work with OMV

    Everything is possible, sometimes it requires Google to find out how.

  • Once again thanks for the great reply.


    I'll confirm I want the kit, and order the cables, when it gets here. (Or upload another photo if it looks like a different connector ;) )


    I also think there is so little info on these shelves, or any of the similar ones that are starting to appear second-hand with good prices, that should I getit working, it would be worth me putting together a HOW-TO for the OMV site, it could potentially be a good plus point for OMV site in general.

  • Well the arrays are on the way to me, should be here tomorrow, and I spoke to the previous owner, and he says it an SFP connection on the shelves, and he's supplied me a some GBICs and some LC-LC cables, which is what we both reckon sounds like the right combination. So hopefully all that lot should be here tomorrow.


    In the meantime I've got a HP G4 DL380 out of the garage (Yes I have one of those garages, it's amazing what you find), stuck a 36Gb SCSI drive in it, and booted and installed OMV ready for tomorrow. Then while I was watching it install, I was sat looking at the stack of 72Gb SCSI drives, so I stuck 5 them in it, for a few extra spindles when it all comes together :)


    I've then installed one of Emulex HBAs Reset it to default settings in its bios, then restarted OMV, and I'm happy to say Debian found it straight away, and looking in /sys/class/fc_host shows "host0" with the relevant settings. So looking good so far. Now I'm waiting for kit to arrive tomorrow.

  • Normally you also need something like lputil to configure the WWN masking etc. pp. But I guess in that small environment it is not really required.


    So looks pretty good, now lets wait for your feedback :)

    Everything is possible, sometimes it requires Google to find out how.

  • Well the initial results appear good.


    Boot up with everything a default, and debian sees the drives, but is finding sectors faults left right and centre on drive sdm. One drive (bay 1) is showing a orange warning light. So I swap the drive. Still orange light, Figure it is something more involved. Pull the drive and continue. Now drive 2 shows the same fault :(


    Right this has to be configuration issue. So in the Emulux Bios I notice that it defaulting to "Loop", so I chance this to "FC-AL" - (It says it on the back of the storage shelf), Debian now boots, and slowly comes up. Seriously it takes about 5 minutes to boot, but I have just thrown 14 disks at it :)


    Next I log into openmediavault and check physical disks. I can see 13 extra disks - disk SDA is missing, but could this be a false flag, because could sda actually be my boot disk, which OMV won't let me use?? Time to investigate....................

  • Well sda is in fact the first drive on the fibre channel as shown by lsscsi and it is showing in debian, so not sure why it is not showing in OMV


    I have tried doing a SCAN in physical disks but it is still not showing.


    Anybody any ideas??

  • And to add to the information, the drive that is not participating in the LED disco caused by building a test RAID is what I would call drive 9 or would expect to be sdi?

  • Faulty Disk was the answer, swapping the disk for one out of the second array, and boots up in under 1 minute, with 14 disks. So looking good so far.


    Next hurdle 2 drive shelves :)

    • Offizieller Beitrag
    Zitat von "drumltd"

    Next hurdle 2 drive shelves :)


    and pictures :)

    omv 7.0.4-2 sandworm | 64 bit | 6.5 proxmox kernel

    plugins :: omvextrasorg 7.0 | kvm 7.0.10 | compose 7.1.2 | k8s 7.0-6 | cputemp 7.0 | mergerfs 7.0.3


    omv-extras.org plugins source code and issue tracker - github


    Please try ctrl-shift-R and read this before posting a question.

    Please put your OMV system details in your signature.
    Please don't PM for support... Too many PMs!

  • Well HBA to Drive Array appears to work in any port. However I can't for the life of me figure out daisy chaining the shelves. Even tried 2 HBA but I still only see one shelf. I'm beginning to suspect faulty fibre cable.

  • Yes having spent the last hour trying different combinations of cable, I can confirm out of 3 cables only 1 works, Typically the only brand new bit of kit in the entire setup is the fibre cables.


    Which would explain why neither daisy-chaining or 2 HBA will work.


    So while I'm waiting for the next set of cables to arrive, and a couple of HDs to be replaced. What are peoples thoughts on the RAID setup. I should have at the end of this 28 500GB Drives to build into a software Raid and 5 73Gb Drives that can be either Software or Hardware Raid, These could potentially have a write cache, but at the moment the battery is flat)


    The system is going to store approx


    350Gb of Source Data (Large Database files of about 1-2Gb)
    1Gb PXE boot system, with a basic copy of ubuntu, which is going to be used to boot 10 or so Servers.
    50+ VirtualBox VMs booting XP all to be run at the same time, each one is 10Gb (This is what I think crippled the last system)
    1Tb for Target Data, this will be thousands of small files (typically 100kb) in thousands of directories. And grow upto this size as the system runs.


    As you can see size is not going to be an issue, with approx 14Tb of disk space available.


    All this data is held on another machine (currently Raid5) and I can pull the target data across on a regular basis. So bearing this in mind my priority is Speed. So I guess straight striping is going to be the fastest setup, but beyond that I'm open to suggestions.


    Am I best doing 4 different Raids each optimized for the data they are holding. Or Blitz the whole lot onto one big raid, giving more spindles to the problem areas?? Is it worth me adjusting stripe size etc. for better performance.

  • Hmm, maybe it is not really the cable. You can change the ordering of the two connections of the cable.


    lets look at the HBA end of the cable.


    One fibre of the two you have is for sending data and one is for receiving data. So the fibre that is in the sending port of the HBA needs to go in the receiving port of the array.


    The cables can be (for whatever reason) changed and they can now be wrong. You can simply check, which of the ports is emitting light (this is the send port). Then connect the cable to the HBA and look on the cable, out of which fibre the light comes out and check with the Shelf if that is the correct orientation for the receive port in the shelf. If not, change the plug on the shelf side of the cable.


    Maybe that helps.

    Everything is possible, sometimes it requires Google to find out how.

Jetzt mitmachen!

Sie haben noch kein Benutzerkonto auf unserer Seite? Registrieren Sie sich kostenlos und nehmen Sie an unserer Community teil!