Stable device names

bensisko · 31. Oktober 2019

So for some time now I've been happily using OMV inside a QNAP TS-451+, and it's working much better than the stuff qnap gives. However, there's only thing I'm missing from this arrangement. If a drive fails, I have to basically guess which bay is the one that houses it. Is there a good way to make hdd devices have stable device names? I'm thinking something on the order of a udev rule, so I could assign disks to paths, and when it fails, i can see that instead of /dev/sda failing it's something like /dev/bay0 or something of the sort

systemd already does something similar to ethernet ports.

If anyone already has something similar done, I'd love to hear about it

geaves · 31. Oktober 2019

Zitat von bensisko

Is there a good way to make hdd devices have stable device names?

Yeah create a simple excel spreadsheet, with the drive bay as the reference, this then has the drive make. model, serial no, and drive label I have assigned in omv, so if a drive failed I know from the information in omv what drive bay it's in.

ryecoaaron · 31. Oktober 2019

I haven't checked on my QNAP but it looks like you could tell by the path in:

ls -al /dev/disk/by-path

Spoiler anzeigen

Just guessing but they should always be:

Bay 1 -- pci-0000:01:00.0-ata-1
Bay 2 -- pci-0000:01:00.0-ata-2
Bay 3 -- pci-0000:02:00.0-ata-1
Bay 4 -- pci-0000:02:00.0-ata-2

bensisko · 31. Oktober 2019

Unfortunately not for me, in fact they're quite not-in-order

Code

0 [3:0:0:0] /dev/sdb ata-Samsung_SSD_860_EVO_250GB_S3YJNF0M717186E  
1 [2:0:0:0] /dev/sda ata-ST4000VN008-2DR166_ZDH60ERC 
2 [7:0:0:0] /dev/sdd ata-ST4000VN008-2DR166_ZDH5T2L1 
3 [6:0:0:0] /dev/sdc ata-ST4000VN008-2DR166_ZGY3DRBN 


lrwxrwxrwx 1 root root   9 Oct 31 18:24 pci-0000:01:00.0-ata-3 -> ../../sda
lrwxrwxrwx 1 root root   9 Oct 31 18:24 pci-0000:01:00.0-ata-4 -> ../../sdb
lrwxrwxrwx 1 root root   9 Oct 31 18:24 pci-0000:02:00.0-ata-3 -> ../../sdc
lrwxrwxrwx 1 root root   9 Oct 31 18:24 pci-0000:02:00.0-ata-4 -> ../../sdd

I'm thinking on how to get udev to do what I do.. The attributes that I want are in the parent device of the actual disk, not in the disk itself..

Code

vault# udevadm info -a /dev/sda  


Udevadm info starts with the device specified by the devpath and then
walks up the chain of parent devices. It prints for every device
found, all possible attributes in the udev rules key format.
A rule to match, can be composed by the attributes of the device
and the attributes from one single parent device.


  looking at device '/devices/pci0000:00/0000:00:1c.0/0000:01:00.0/ata3/host2/target2:0:0/2:0:0:0/block/sda':
    KERNEL=="sda"
    SUBSYSTEM=="block"
    DRIVER==""
    ATTR{range}=="16"
    ATTR{removable}=="1"
    ATTR{events_async}==""
    ATTR{inflight}=="       0        0"
    ATTR{ro}=="0"
    ATTR{alignment_offset}=="0"
    ATTR{stat}=="     821      292   327584     7238      152       61     1494     3130        0     3196     8768        0        0        0        0"
    ATTR{ext_range}=="256"
    ATTR{hidden}=="0"
    ATTR{events_poll_msecs}=="-1"
    ATTR{size}=="7814037168"
    ATTR{capability}=="51"
    ATTR{discard_alignment}=="0"
    ATTR{events}=="media_change"


  looking at parent device '/devices/pci0000:00/0000:00:1c.0/0000:01:00.0/ata3/host2/target2:0:0/2:0:0:0':
    KERNELS=="2:0:0:0"
    SUBSYSTEMS=="scsi"
    DRIVERS=="sd"
    ATTRS{type}=="0"
    ATTRS{queue_type}=="simple"
    ATTRS{vpd_pg83}==""
    ATTRS{evt_mode_parameter_change_reported}=="0"
    ATTRS{eh_timeout}=="10"
    ATTRS{model}=="ST4000VN008-2DR1"
    ATTRS{evt_media_change}=="0"
    ATTRS{evt_inquiry_change_reported}=="0"
    ATTRS{evt_capacity_change_reported}=="0"
    ATTRS{evt_soft_threshold_reached}=="0"
    ATTRS{dh_state}=="detached"
    ATTRS{unload_heads}=="0"
    ATTRS{blacklist}==""
    ATTRS{device_busy}=="0"
    ATTRS{iorequest_cnt}=="0xcb5"
    ATTRS{vpd_pg80}==""
    ATTRS{device_blocked}=="0"
    ATTRS{state}=="running"
    ATTRS{queue_depth}=="32"
    ATTRS{rev}=="SC60"
    ATTRS{evt_lun_change_reported}=="0"
    ATTRS{wwid}=="t10.ATA     ST4000VN008-2DR166                                  ZDH60ERC"
    ATTRS{ncq_prio_enable}=="0"
    ATTRS{queue_ramp_up_period}=="120000"
    ATTRS{ioerr_cnt}=="0x205"
    ATTRS{inquiry}==""
    ATTRS{iodone_cnt}=="0xc9c"
    ATTRS{timeout}=="30"
    ATTRS{iocounterbits}=="32"
    ATTRS{scsi_level}=="6"
    ATTRS{vendor}=="ATA     "


  looking at parent device '/devices/pci0000:00/0000:00:1c.0/0000:01:00.0/ata3/host2/target2:0:0':
    KERNELS=="target2:0:0"
    SUBSYSTEMS=="scsi"
    DRIVERS==""


  looking at parent device '/devices/pci0000:00/0000:00:1c.0/0000:01:00.0/ata3/host2':
    KERNELS=="host2"
    SUBSYSTEMS=="scsi"
    DRIVERS==""


  looking at parent device '/devices/pci0000:00/0000:00:1c.0/0000:01:00.0/ata3':
    KERNELS=="ata3"
    SUBSYSTEMS==""
    DRIVERS==""


  looking at parent device '/devices/pci0000:00/0000:00:1c.0/0000:01:00.0':
    KERNELS=="0000:01:00.0"
    SUBSYSTEMS=="pci"
    DRIVERS=="ahci"
    ATTRS{revision}=="0x11"
    ATTRS{broken_parity_status}=="0"
    ATTRS{resource3}==""
    ATTRS{subsystem_device}=="0x9215"
    ATTRS{current_link_speed}=="5 GT/s"
    ATTRS{max_link_speed}=="5 GT/s"
    ATTRS{msi_bus}=="1"
    ATTRS{enable}=="1"
    ATTRS{resource1}==""
    ATTRS{local_cpulist}=="0-3"
    ATTRS{local_cpus}=="f"
    ATTRS{max_link_width}=="1"
    ATTRS{consistent_dma_mask_bits}=="64"
    ATTRS{ari_enabled}=="0"
    ATTRS{current_link_width}=="1"
    ATTRS{numa_node}=="-1"
    ATTRS{d3cold_allowed}=="1"
    ATTRS{driver_override}=="(null)"
    ATTRS{subsystem_vendor}=="0x1b4b"
    ATTRS{device}=="0x9215"
    ATTRS{vendor}=="0x1b4b"
    ATTRS{class}=="0x010601"
    ATTRS{irq}=="97"
    ATTRS{dma_mask_bits}=="64"


  looking at parent device '/devices/pci0000:00/0000:00:1c.0':
    KERNELS=="0000:00:1c.0"
    SUBSYSTEMS=="pci"
    DRIVERS=="pcieport"
    ATTRS{local_cpulist}=="0-3"
    ATTRS{max_link_speed}=="5 GT/s"
    ATTRS{vendor}=="0x8086"
    ATTRS{subsystem_device}=="0x0f48"
    ATTRS{broken_parity_status}=="0"
    ATTRS{aer_rootport_total_err_nonfatal}=="0"
    ATTRS{revision}=="0x0e"
    ATTRS{dma_mask_bits}=="32"
    ATTRS{secondary_bus_number}=="1"
    ATTRS{current_link_speed}=="5 GT/s"
    ATTRS{irq}=="87"
    ATTRS{enable}=="1"
    ATTRS{max_link_width}=="1"
    ATTRS{msi_bus}=="1"
    ATTRS{subsystem_vendor}=="0x8086"
    ATTRS{aer_rootport_total_err_fatal}=="0"
    ATTRS{ari_enabled}=="0"
    ATTRS{local_cpus}=="f"
    ATTRS{subordinate_bus_number}=="1"
    ATTRS{d3cold_allowed}=="1"
    ATTRS{numa_node}=="-1"
    ATTRS{current_link_width}=="1"
    ATTRS{consistent_dma_mask_bits}=="32"
    ATTRS{device}=="0x0f48"
    ATTRS{class}=="0x060400"
    ATTRS{driver_override}=="(null)"
    ATTRS{aer_rootport_total_err_cor}=="0"


  looking at parent device '/devices/pci0000:00':
    KERNELS=="pci0000:00"
    SUBSYSTEMS==""
    DRIVERS==""

Alles anzeigen

ryecoaaron · 1. November 2019

Zitat von bensisko

Unfortunately not for me, in fact they're quite not-in-order

The first half of the listing is the scsi id. I knew that wouldn't be in order. That is why I didn't list mine. The second group is the path and it is in the same order. The IDs are increment but I am guessing you have drives plugged into usb?

bensisko · 1. November 2019

No, that first list IS in order, this is how it should be counting from left to right (hence the index numbers). I've showed the by-id paths because they seem to have no relation to the SCSI IDs.

And no, perish the thought, they're all connected to a scsi controller on pci-express.

So to reiterate, I'd like to rename these drives based on the scsi port they're connected to

ryecoaaron · 1. November 2019

Zitat von bensisko

No, that first list IS in order, this is how it should be counting from left to right (hence the index numbers). I've showed the by-id paths because they seem to have no relation to the SCSI IDs.

Well, I was wrong. I was just guessing based on my output. Using scsi IDs won't work either because those can change.

Zitat von bensisko

And no, perish the thought, they're all connected to a scsi controller on pci-express.

While linux treats them as scsi devices, they are just connected to sata ports through a back plane.

Zitat von bensisko

So to reiterate, I'd like to rename these drives based on the scsi port they're connected to

You would be better off to read the serial number from the drive and put a sticker on the drive bay. I don't think anything else other than the serial number (and maybe label) are going to be the same on every boot.

bensisko · 1. November 2019

Well, I kinda figured it out. Unfortunately udev is unable to rename devices in the /dev folder (with the exception of interface names), it can only do symlinks. However, it can do symlinks. And that's kinda enough.

/etc/udev/rules.d/65-bays.rules:

Code

KERNEL=="sd*", SUBSYSTEM=="block", SUBSYSTEMS=="scsi", KERNELS=="3:0:0:0", SYMLINK+="bay0"
KERNEL=="sd*", SUBSYSTEM=="block", SUBSYSTEMS=="scsi", KERNELS=="2:0:0:0", SYMLINK+="bay1"
KERNEL=="sd*", SUBSYSTEM=="block", SUBSYSTEMS=="scsi", KERNELS=="7:0:0:0", SYMLINK+="bay2"
KERNEL=="sd*", SUBSYSTEM=="block", SUBSYSTEMS=="scsi", KERNELS=="6:0:0:0", SYMLINK+="bay3"

Now with that, one could create some kind of panel in OMV. Maybe I'll actually write one as a plugin. I'd be cool to integrate LEDs into this as well, but the standard debian kernel doesn't provide the gpio driver for my controller (even though it is present in the linux kernel. Debian just doesn't ship it for some reason)

ryecoaaron · 2. November 2019

Your rules assume the scsi id will always be the same. My QNAP is using different IDs and if you change a drive, they will change.

Zitat von bensisko

Now with that, one could create some kind of panel in OMV. Maybe I'll actually write one as a plugin. I'd be cool to integrate LEDs into this as well, but the standard debian kernel doesn't provide the gpio driver for my controller (even though it is present in the linux kernel. Debian just doesn't ship it for some reason)

That would be the hardest plugin to write ever since it would have to "know" how every server with bays works.

I looked into the LEDs a long time ago but it is pain since the code isn't in the Linux driver and QNAP patches their kernel. You would have to run the same kernel the stock QNAP does.

bensisko · 2. November 2019

Zitat von ryecoaaron

Your rules assume the scsi id will always be the same. My QNAP is using different IDs and if you change a drive, they will change.

From what I see those actually stay the same, I tried to trip it up. I'd love to know the mechanism of HOW they get those IDs. And yeah, I know there doesn't seem to be a way to automagically know which bay is which scsi id. But I don't think I'd be too much of a pain to ask the user that.

As for the LEDs, I actually found a guy who wrote a small driver for his 451. We're already talking about how to extend it You can take a peek here https://github.com/vpelletier/debian-qnap-tsx51

The biggest problem is that QNAP decided NOT to fill out the DMI properly, so the driver has no way of identifying what it's running on..
But as of now I think it's realistic to have the following functionality
#1 Map a drive to a bay (by way of the user pointing of the relation between drive and bay)
#2 Map an LED to a bay (by way of user pointing an LED to a bay)
#3 Mark the LED red if a drive fails (if #2 is configured)

This is somewhat how openwrt does it as well

Edit: I am only taking scsi drives into account here. I don't care about nvme and/or USB

ryecoaaron · 2. November 2019

Zitat von bensisko

I'd love to know the mechanism of HOW they get those IDs.

This should help...

But this line tells you that those numbers can change:

Zitat

The command multipath -l shows the mapping to the non-persistent identifiers: Host:Channel:Target:LUN, /dev/sd name, and the major:minor number.

If you want something persistent, it says you should use:

Zitat

World Wide Identifier (WWID)

The World Wide Identifier (WWID) can be used in reliably identifying devices. It is a persistent, system-independent ID that the SCSI Standard requires from all SCSI devices. The WWID identifier is guaranteed to be unique for every storage device, and independent of the path that is used to access the device.

Either way, someone else is going to have to write the plugin if desired. I don't have time to maintain the plugins out there already.

bensisko · 2. November 2019

The WWID is an ID of a particular drive, not it's port. Even the description mentions - "independent of the path that is used". I am not quite sure how host controller order is established, but it seems that for my case they seem to be stable

And yeah, I intend to write this myself. I'll ask for inclusion into omv-extras if I deem it worth it to anyone else, and you'll be free to reject it if it'll be too much of a burden

ryecoaaron · 2. November 2019

Zitat von bensisko

And yeah, I intend to write this myself. I'll ask for inclusion into omv-extras if I deem it worth it to anyone else, and you'll be free to reject it if it'll be too much of a burden

If I don't have to write and/or maintain it, it definitely isn't too much of a burden.

Zitat von bensisko

The WWID is an ID of a particular drive, not it's port. Even the description mentions - "independent of the path that is used". I am not quite sure how host controller order is established, but it seems that for my case they seem to be stable

I know it is but that seems to be the only thing guaranteed. Controller order is determined by the bios and that can change based on usb devices and/or nvme devices that are plugged in. So, if you put your qnap in a corner like I do, the order won't change but it is also less important which bay is which.

Jetzt mitmachen!