It's still working!! in fact in the current raid 5 setup....when I added it back to the then zfs raid 5 it just resilvered (after running dd) I decided the issue was with the metadata written to the drive rather than the physical drive itself.What you would expect when it shows as degraded is the drive failing to sync due to a failed servo...that has a distinctive noise....one of the best things I ever purchased was Spinrite, this has rescued so many drives not just for me but for friends, where a drive has potential bad sectors it can move the data making it recoverable.
Again, this doesn't instill confidence in ZFS. While the issue could have been an odd ball quark it's still an anomaly that occurs, how often? (That's the root question in all of it.)
I think I had spinrite at one time, or something similar. Since drive size is way outpacing the speed of the interface, how long did it take to run spinrite on drive you mentioned above?
Sounds promising, have you started coding it yet
In another life, I might have been a programmer but I tend to doubt it. (Politics tends to give me a rash and you'd be surprised at office politics in programming.)
Actually, I think the above would be fairly easy to implement as a mere extension of BTRFS. (Link 2 servers, with some sort of com channel, SSH?, and engage the same checking routine BTRFS uses for RAID1.) At our level, or possibly a small or medium sized business, such a solution would work very nicely. Data would remain free of corruption while providing for a full backup server system. (2 birds with one stone.) The reason why it probably won't happen is, there's no solid demand for it. The problem with the concept is, at our level most are willing to risk their data with no back up at all. Total data loss events are rare enough to where most don't worry about it, until it's too late. It's a bummer because I'd love to see something like it.