I noticed a strange issue that I initially blamed it on a few power outages. I am using the latest available OMV 6 with the latest Proxmox kernel 6.2.16-11-bpo11-pve. The HDD in question is new, it was purchased last year with another one. I know that the issues shown in dmesg should mean that the HDD turned bad and today I booted into GParted Live and Seatools Bootable to check all HDDs. None of the tools are showing any errors with the HDD in question, I let it run the Short Generic Test. SMART information is clean and I cannot see any dmesg errors in GParted Live, Seatools Bootable or Ubuntu 22.04 Live. I even tried changing the SATA port. I don't know when the issue appeared, but I noticed sometime at the end of last year that my snapraid sync and scrub jobs were hung for a long time and that the snapraid.content on the affected drive is different than the ones on the other content drives.
Could this be related to the installation or the kernel? I have not tried booting into the mainline OMV kernel or any other older kernels. I also realized I didn't change the SATA cable with a new one to rule out a bad cable. What puzzles me the most is that booting into anything else but OMV shows no error on the affected drive. One strange behavior of the affected drive is that it appeared as not mounted at some point - mountpoint was completely empty. All data on it was visible after a reboot - that scared me the most and then I started doing some basic checks that led me to running Seatools Bootable and checking it in a Live CD environment. I can read and write on it now without any issues. I fear that if I were to send it for warranty I would just get it back as functional.
I am pasting the dmesg logs below.
QuoteDisplay More[ 7.910943] sd 6:0:0:0: [sdd] 23437770752 512-byte logical blocks: (12.0 TB/10.9 TiB)
[ 7.910945] sd 6:0:0:0: [sdd] 4096-byte physical blocks
[ 7.910954] sd 6:0:0:0: [sdd] Write Protect is off
[ 7.910956] sd 6:0:0:0: [sdd] Mode Sense: 00 3a 00 00
[ 7.910978] sd 6:0:0:0: [sdd] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
[ 7.910999] sd 6:0:0:0: [sdd] Preferred minimum I/O size 4096 bytes
[ 7.988961] sdd: sdd1
[ 7.989102] sd 6:0:0:0: [sdd] Attached SCSI disk
[ 9.688795] EXT4-fs (sdd1): mounted filesystem 6c5625b1-5261-4dd3-a63a-e6b878d2572e with ordered data mode. Quota mode: journalled.
[ 64.828391] sd 6:0:0:0: [sdd] tag#0 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_OK cmd_age=0s
[ 64.828397] sd 6:0:0:0: [sdd] tag#0 Sense Key : Illegal Request [current]
[ 64.828401] sd 6:0:0:0: [sdd] tag#0 Add. Sense: Unaligned write command
[ 64.828406] sd 6:0:0:0: [sdd] tag#0 CDB: Read(16) 88 00 00 00 00 02 95 00 08 38 00 00 00 08 00 00
[ 64.828408] I/O error, dev sdd, sector 11089741880 op 0x0:(READ) flags 0x83700 phys_seg 1 prio class 2
[ 64.828433] sd 6:0:0:0: [sdd] tag#1 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_OK cmd_age=0s
[ 64.828437] sd 6:0:0:0: [sdd] tag#1 Sense Key : Illegal Request [current]
[ 64.828441] sd 6:0:0:0: [sdd] tag#1 Add. Sense: Unaligned write command
[ 64.828444] sd 6:0:0:0: [sdd] tag#1 CDB: Read(16) 88 00 00 00 00 02 95 40 08 00 00 00 00 08 00 00
[ 64.828446] I/O error, dev sdd, sector 11093936128 op 0x0:(READ) flags 0x83700 phys_seg 1 prio class 2
[ 64.828460] sd 6:0:0:0: [sdd] tag#3 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_OK cmd_age=0s
[ 64.828463] sd 6:0:0:0: [sdd] tag#3 Sense Key : Illegal Request [current]
[ 64.828467] sd 6:0:0:0: [sdd] tag#3 Add. Sense: Unaligned write command
[ 64.828470] sd 6:0:0:0: [sdd] tag#3 CDB: Read(16) 88 00 00 00 00 02 95 c0 08 00 00 00 00 08 00 00
[ 64.828472] I/O error, dev sdd, sector 11102324736 op 0x0:(READ) flags 0x83700 phys_seg 1 prio class 2
[ 64.828487] sd 6:0:0:0: [sdd] tag#20 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_OK cmd_age=0s
[ 64.828490] sd 6:0:0:0: [sdd] tag#20 Sense Key : Illegal Request [current]
[ 64.828493] sd 6:0:0:0: [sdd] tag#20 Add. Sense: Unaligned write command
[ 64.828497] sd 6:0:0:0: [sdd] tag#20 CDB: Read(16) 88 00 00 00 00 02 94 00 08 00 00 00 00 08 00 00
[ 64.828499] I/O error, dev sdd, sector 11072964608 op 0x0:(READ) flags 0x83700 phys_seg 1 prio class 2
[ 64.828510] sd 6:0:0:0: [sdd] tag#22 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_OK cmd_age=0s
[ 64.828513] sd 6:0:0:0: [sdd] tag#22 Sense Key : Illegal Request [current]
[ 64.828517] sd 6:0:0:0: [sdd] tag#22 Add. Sense: Unaligned write command
[ 64.828520] sd 6:0:0:0: [sdd] tag#22 CDB: Read(16) 88 00 00 00 00 02 94 80 08 00 00 00 00 08 00 00
[ 64.828522] I/O error, dev sdd, sector 11081353216 op 0x0:(READ) flags 0x83700 phys_seg 1 prio class 2
[ 64.828533] sd 6:0:0:0: [sdd] tag#23 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_OK cmd_age=0s
[ 64.828536] sd 6:0:0:0: [sdd] tag#23 Sense Key : Illegal Request [current]
[ 64.828539] sd 6:0:0:0: [sdd] tag#23 Add. Sense: Unaligned write command
[ 64.828543] sd 6:0:0:0: [sdd] tag#23 CDB: Read(16) 88 00 00 00 00 02 94 c0 08 00 00 00 00 08 00 00
[ 64.828545] I/O error, dev sdd, sector 11085547520 op 0x0:(READ) flags 0x83700 phys_seg 1 prio class 2
[ 64.828555] sd 6:0:0:0: [sdd] tag#30 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_OK cmd_age=0s
[ 64.828558] sd 6:0:0:0: [sdd] tag#30 Sense Key : Illegal Request [current]
[ 64.828562] sd 6:0:0:0: [sdd] tag#30 Add. Sense: Unaligned write command
[ 64.828565] sd 6:0:0:0: [sdd] tag#30 CDB: Read(16) 88 00 00 00 00 02 95 00 08 00 00 00 00 08 00 00
[ 64.828567] I/O error, dev sdd, sector 11089741824 op 0x0:(READ) flags 0x83700 phys_seg 1 prio class 2
Edit: I just booted with the non Proxmox kernel 6.1.0-0.deb11.13-amd64 and I cannot see any issue with any disks. I ran snapraid and I also copied a large file to the affected disk for testing. I need to be able to use OpenCL, that is why I want to use Proxmox kernel.