Suggestions on NAS hardware

    • Offizieller Beitrag

    The m'board Raid is hardware so the m'board would control the hard drives set up for Raid configuration.

    ____________________________________________________________________________________________________


    Another simple option is to use the drives as singles, set one for data and the other as rsync, effectively a mirror of the first, rsync could run after hours. The transfer to SnapRaid would be easier when you purchase a third drive

  • software RAID

    yes, see https://en.wikipedia.org/wiki/Intel_Rapid_Storage_Technology . I'd avoid using that proprietary Intel RST

    omv 6.9.6-2 (Shaitan) on RPi CM4/4GB with 64bit Kernel 6.1.21-v8+

    2x 6TB 3.5'' HDDs (CMR) formatted with ext4 via 2port PCIe SATA card with ASM1061R chipset providing hardware supported RAID1


    omv 6.9.3-1 (Shaitan) on RPi4/4GB with 32bit Kernel 5.10.63 and WittyPi 3 V2 RTC HAT

    2x 3TB 3.5'' HDDs (CMR) formatted with ext4 in Icy Box IB-RD3662-C31 / hardware supported RAID1

    For Read/Write performance of SMB shares hosted on this hardware see forum here

    2 Mal editiert, zuletzt von mi-hol ()

  • stop giving false information

    I'm happy to learn, my understanding is that Intel Rapid Storage Technology is:


    "Firmware- and driver-based

    Software-implemented RAID is not always compatible with the system's boot process, and it is generally impractical for desktop versions of Windows. However, hardware RAID controllers are expensive and proprietary. To fill this gap, inexpensive "RAID controllers" were introduced that do not contain a dedicated RAID controller chip, but simply a standard drive controller chip with proprietary firmware and drivers."

    omv 6.9.6-2 (Shaitan) on RPi CM4/4GB with 64bit Kernel 6.1.21-v8+

    2x 6TB 3.5'' HDDs (CMR) formatted with ext4 via 2port PCIe SATA card with ASM1061R chipset providing hardware supported RAID1


    omv 6.9.3-1 (Shaitan) on RPi4/4GB with 32bit Kernel 5.10.63 and WittyPi 3 V2 RTC HAT

    2x 3TB 3.5'' HDDs (CMR) formatted with ext4 in Icy Box IB-RD3662-C31 / hardware supported RAID1

    For Read/Write performance of SMB shares hosted on this hardware see forum here

    • Offizieller Beitrag

    I'm happy to learn, my understanding is that Intel Rapid Storage Technology is:

    As your gleaned information has pointed out "inexpensive Raid controllers" which primarily are built into the m'board bios, to initialise these controllers proprietary firmware and drivers are required, the driver makes it possible for the O/S to 'see' the Raid Controller. So if you use the onboard controller and set it to raid then the m'board presents the drives in a preset raid configuration, so in essence the m'board 'looks after the drives' similar to proprietary raid controllers.


    Software Raid is as it says, the first part is the clue, the drives are presented to the O/S as single drives, whereby in OMV's case mdadm can be used to configure the raid setups. In Windows they implement a form of drive pooling which does not require drives of the same size to be used.


    Your Icy Box uses a similar inexpensive raid controller, that is why those have a switch on the back that the user sets to configure how the drives are presented to the O/S


    As I tried to suggest there is a better way of doing this than using any type of raid configuration -> and by the power of Grayskull

  • Another simple option is to use the drives as singles, set one for data and the other as rsync, effectively a mirror of the first, rsync could run after hours. The transfer to SnapRaid would be easier when you purchase a third drive

    After reading through some comparisons between RAID 1 and rsync, I think a simple rsync job once a day is the best option for me. Thanks a lot for the recommendation.


    Just out of curiosity, if I were to use the motherboard RAID option or the OMV RAID option (which is based on mdadm, as far as I understand it) and one drive failed, would I be able to simply read the other drive, or would I always be dependent on the appropriate RAID option to read the data?

    • Offizieller Beitrag

    Just out of curiosity, if I were to use the motherboard RAID option or the OMV RAID option (which is based on mdadm, as far as I understand it) and one drive failed

    Technically yes in both instances, however, but, :) once a Raid has degraded it slows down in other words accessing the data is much much slower, this is compounded further when a new drive is added as the raid has to rebuild.


    You could run rsync every day, but assess how much data has been changed or added, + rsync jobs can be run in the early hours and set to send an email. The other + once it's set up it sends an incremental file list to check for changes, this takes seconds, if it finds no changes this is the output you get back;


    Code
    Please wait, syncing </trinity/Movies/> to </srv/dev-disk-by-label-RsyncBackUp/MoviesBack/> ...
    
    sending incremental file list
    
    sent 317,008 bytes  received 819 bytes  211,884.67 bytes/sec
    total size is 630,277,954,524  speedup is 1,983,084.99
    The synchronisation has completed successfully.
    Done ...
    • Offizieller Beitrag

    After reading through some comparisons between RAID 1 and rsync, I think a simple rsync job once a day is the best option for me. Thanks a lot for the recommendation.


    Just out of curiosity, if I were to use the motherboard RAID option or the OMV RAID option (which is based on mdadm, as far as I understand it) and one drive failed, would I be able to simply read the other drive, or would I always be dependent on the appropriate RAID option to read the data?

    I have literally used rsync as opposed to raid1 for about 8yrs... never a single problem other than a drive failure, and it was incredibly simple to recover from. I personally don't add "Drive 2" to anything other than rsync.. no services, no smb, etc. All services are ran off drive 1... Sync it every day at 0900, and it's all good for me.

    • Offizieller Beitrag

    I personally don't add "Drive 2" to anything other than rsync.. no services, no smb, etc. All services are ran off raid 1... Sync it every day at 0900, and it's all good for me.

    This is smart. No viruses from networked systems can get to it then. And if your "drive 1" fails, you can change the drive your sharedfolders are pointing to to drive 2 and everything is back up and running.

    • Offizieller Beitrag

    This is smart. No viruses from networked systems can get to it then. And if your "drive 1" fails, you can change the drive your sharedfolders are pointing to to drive 2 and everything is back up and running.

    Yep, and if you're smart, you'll leave the delete trigger off. A few times I've accidentally deleted a file, made a change I wanted to go back on, etc.. it's just a matter of using SSH to copy whatever I need from "drive 2" back to "drive 1".. as it will still be there even if a sync was done (you'll just have two copies). Once a month or so, I enable the delete trigger on the rsync job, run it manually, then disable it again. Takes no time, although I've considered automating this as well w/ a duplicate job that runs once or twice a month, but w/ the delete trigger enabled.


    It's brain dead simple, and just considerably better than raid1, IMO.

  • I personally don't add "Drive 2" to anything other than rsync.. no services, no smb, etc. All services are ran off raid 1... Sync it every day at 0900, and it's all good for me.

    That is exactly what I am planning to do. But what do you mean with "All services are ran off raid 1?"


    Yep, and if you're smart, you'll leave the delete trigger off.

    That makes sense as well. However, I would still set it up as an automatic job that it synchronizes everything again once a month.


    Thanks again for all your help!

    • Offizieller Beitrag

    That is exactly what I am planning to do. But what do you mean with "All services are ran off raid 1?"


    That makes sense as well. However, I would still set it up as an automatic job that it synchronizes everything again once a month.


    Thanks again for all your help!

    Sorry that was a typo on my part.. I meant drive 1, not raid 1. If it's not clear, I don't use raid at all.

    • Offizieller Beitrag

    Only reason I like having the delete job run manually, is when I run it, I know there is no concern I'm losing anything I need.


    If you need something and it has been deleted. You only have until X date (when the auto delete job runs) to catch it, or it's gone for good.

  • Regarding RAID/redundancy/backup, I have a question: My idea was to set up snapRAID for redundancy and back up my data to an external hard drive, plus keep the most important data on my Google Drive. With 2 hard drives snapRAID is not an option, so I would just mirror the drive and once I get a third hard drive delete one and set up snapRAID. What are your suggestions on this? Would you set up mirroring (which is basically the same as RAID 1, am I right?) in OMV or in BIOS? Are there any differences there?


    Also, what is the common practice for occasionally backing up data to an external hard drive in OMV? Can this be done semi-automatically, or do I have to manually plug in a hard drive and perform the backup every now and then?

    If you want to use classic RAID don't use hardware RAID, but enable it in OMV GUI.

    I would still recommend snapRAID for added flexibility. With RAID you won't be able to add a third drive without reformatting, with snapRAID yes, by rebuilding the parity.


    Regarding backup to an external drive, there a perfect plugin called openmediavault-usbbackup.

    When configured just connect the external drive, he'll do the job and eject it for you when completed.

    You'd better do it this way (manually plugging/unplugging the drive) because is more secure to store the external drive away from the NAS, even if it's the same home.

  • Sorry that was a typo on my part.. I meant drive 1, not raid 1. If it's not clear, I don't use raid at all.

    Ah okay, now it makes sense. I could have guessed that from the context ;)



    If you want to use classic RAID don't use hardware RAID, but enable it in OMV GUI.

    I would still recommend snapRAID for added flexibility. With RAID you won't be able to add a third drive without reformatting, with snapRAID yes, by rebuilding the parity.

    I will use rsync for now (with 2 drives). As soon as I get a third drive I will switch to snapRAID.


    Regarding backup to an external drive, there a perfect plugin called openmediavault-usbbackup.

    When configured just connect the external drive, he'll do the job and eject it for you when completed.

    You'd better do it this way (manually plugging/unplugging the drive) because is more secure to store the external drive away from the NAS, even if it's the same home.

    I'll have a look at that, thanks. Sounds reasonable.

  • Pfff... as a photographer, --delete is a nightmare. You want to keep all edits even if the directory or filename has been changed, but with --delete that is simply a no-go and makes --delete 100% useless. Can't tell you how many .xmp's I lost with 1 sync, but it was around 500.


    Even worse, if you use Adobe Lightroom it puts the old school file lock on there and *if* you leave Lightroom open rsync can hang (1 of the many reasons I stopped using Lightroom).

  • Pfff... as a photographer, --delete is a nightmare. You want to keep all edits even if the directory or filename has been changed, but with --delete that is simply a no-go and makes --delete 100% useless. Can't tell you how many .xmp's I lost with 1 sync, but it was around 500.


    Even worse, if you use Adobe Lightroom it puts the old school file lock on there and *if* you leave Lightroom open rsync can hang (1 of the many reasons I stopped using Lightroom).

    I don't exactly understand what you mean. Why does the --delete option delete all your .xmp files?

  • I don't exactly understand what you mean. Why does the --delete option delete all your .xmp files?

    On Windows, for a tmpfs substitute I use something called imdisk (http://www.ltr-data.se/opencode.html/) for working/scratch drives and use mklink to hide the fact that the source is not local. As long as the virtual drive mounts and assigns the same drive letter everything is peachy, but when it doesn't mklink will still work as it's part of the script routine, but rsync won't be looking at what I want it to. In this specific case, it worked at first but during some reboot the drive letters rearranged and rysnc was looking at a mounted CF. I didn't notice and carried on, so needless to say when the virtual drive went away that left me with no existing edits.


    Was this my fault? Sure, but since I removed --delete I've never had an error I can't recover from.


    Let me add, that the only time you delete photographs is on import and from that point on, you'll never delete anything again. So why have a delete option of any kind in your workflow anywhere?

Jetzt mitmachen!

Sie haben noch kein Benutzerkonto auf unserer Seite? Registrieren Sie sich kostenlos und nehmen Sie an unserer Community teil!