Newbie trying to build a little NAS for family photos

  • After operating two external drives on my Asus Chromebox for some years, I recently discovered that (at least) one of them is having bad sectors. Since I don't want to jeopardise my digital family memories, I want to switch to a more robust setup with OMV on the Chromebox and two 2 TB Ironwolfs in an external Icy Box case. Since I am new to the OMV family, I appreciate any hints or advice to my planned setup. OMV4 is already installed on the Chromebox and I have made the first steps with it (the HDDs were not yet delivered).


    My plan is to operate the two HDDs separately (i.e. no RAID), writing to disk A and rsync the files to disk B as a backup. I also plan on regularly making an off-site backup to an external volume. I would prefer to let rsync run without --delete flag to avoid any loss due to accidental deletion on drive A.


    In the beginning I was thinking about a RAID 1 setup, but apart from the photos I also automatically copy recordings from TV (from my cable receiver) to drive A , which are then directly uploaded to the cloud and deleted after upload from the drive. So these files do not need to be mirrored and mirroring them only increases writings without any purpose in my opinion. Future usage might change, but currently this is what is planned.


    As file system for the discs I thought about keeping things simple using ext4 on both drives, but I have no experience with the other formats (XFS, ZFS, BTRFS) and if they offer any advantages in my case.


    So if you have any comments/hints on my setup please let me know. If I posted in the wrong section, please move to a more appropriate area.

  • I think your plans sounds excellent. Go for it!


    Using two separate drives, one for shared data and one for rsync backups/snapshots, is a simple and very robust setup.


    If you want to improve the security further you could store the most important images in RAR-archives with the archive protection feature activated. Or use some utility to calculate a checksum for a subfolder and check for changes.


    Another (future?) option is the not only have two separate drives but even two separate NAS. At different locations.

    Be smart - be lazy. Clone your rootfs.
    OMV 5: 9 x Odroid HC2 + 1 x Odroid HC1 + 1 x Raspberry Pi 4

  • Thank you both, so be it :)
    I will have a look at rsnapshot. haven't heard of that yet. Sounds well suited for my case as it seems.


    One question about that Icy box encasing (IB-RD3621U3): if I understood correctly, this case also has a RAID controller on board, the settings RAID 0, 1, Single or JBOD/Big can be set with a jumper-linke setup (https://www.raidsonic.de/produ…x_de.php?we_objectID=3589). If I would want to use any of the RAID settings of the drive, how would that needed to be addressed in OMV? I wouldn't set a software RAID on top, would I?

  • If you setup the dual drive cabinet for RAID 1 it will appear in OMV as one single drive. And you won't have to do anything special in OMV.


    If you setup the dual drive cabinet for JBOD then you could either use each disk separately or configure a software RAID. However, since both drives use the same USB cable, performance would be lower with software RAID in OMV.


    But if you setup RAID in the cabinet, if something goes wrong with the cabinet itself, you might loose all data unless you can get a new identical cabinet.


    You could also (if you know how) set it up with btrfs RAID1. That would allow for automatic bitrot protection. But perhaps with low performance since both drives are on the same cable.


    However, OMV doesn't support software RAID over USB, so you would have to set it up yourself from the command line.


    One potential problem with the IB cabinet is that the specs states that it is only compatible with Windows and Mac OS X. That doesn't sound nice. It might still support Linux, but perhaps with very low performance? Or it might work perfectly fine with OMV and Linux, only it hasn't been tested? I don't know. I wouldn't buy it for use with OMV/Linux...


    I use a similar cabinet for one of my RPi4, but from StarTech. It is speced as "OS independent" and I know for a fact that it works fine with OMV on my RPi4. I have it setup for JBOD with two separate independent HDDs. I use it with two old 8TB Archival SMR HDDs for secondary "cold" backups. That is, an extra set of snapshot copies of my most important stuff. I update it only only if needed. Perhaps monthly. It is mostly shut down.


    About rsync and rsnapshot, and my rsync script...


    Snapshots using rsync lets you create backup copies that look just like a full copy of the original. But instead of copying every single file, rsync can identify unchanged files and then "hardlink" to a previous identical copy of the same file. This means that each snapshot consists of a bunch of folders with hardlinks and the occasional updated or new file. It is much faster to create a snapshot like this and it takes up very little storage.


    This means that you can have two similar sized HDDs and use one for shared data and one for backup snapshot. And each snapshot is (appears as) a time-stamped full copy of the original.


    The plugin rsnapshot supports this form of snapshots. it allows you to snapshot shared folders.


    But I prefer to do this outside OMV using cron and my own snapshot script. Typically I use a couple of scripts, on for each major subfolder on the original HDD. For instance one script for music, one for ebooks, one for tv and one for movies.


    https://github.com/WikiBox/snapshot.sh

    Be smart - be lazy. Clone your rootfs.
    OMV 5: 9 x Odroid HC2 + 1 x Odroid HC1 + 1 x Raspberry Pi 4

    Edited 2 times, last by Adoby ().

  • RAID increases the chance of loosing data (vs separate drive you backup to). Go with using the separate backup drive.


    QUESTION: Has anyone tried the Borg Backup OMV Plug-in? (OPENMEDIAVAULT-BORGBACKUP) I can't find any information or guide or example on how to use it anywhere.

  • Thanks Adoby for that long answer. The RAID setup was just for information. I will see if the case suits my purpose and if it doesn't work properly, I will send it back. It seems as I don't need the RAID function of the case anyway, but now it is ordered ;)


    How will the speed from one HDD to the other be in such a setup? I mean the files don't need to pass the USB bus twice, don't they? Since I don't copy to both HDDs simultaneously, speed won't be cut in half I hope.

  • Unfortunately I suspect that copying from one HDD to another in a dual USB HDD case will be slower than copying between two separate single USB HDD cases. I think the data has to pass twice through the cable... It might be possible to pass data full speed in two directions simultaneously over USB. Full duplex like a network card. I doubt it...


    But I haven't tested.


    However with regards to rsync snapshots this will only be a big problem for the first snapshot. That might take some extra time. Subsequent snapshots should be pretty fast since most files already are available for hardlinking. It should be fast enough, even at half speed, I think.


    I suspect that the speed of copying from one HDD to another in a dual HDD USB case will still be faster than copying from one NAS to another over GbE. So you should be fine.

    Be smart - be lazy. Clone your rootfs.
    OMV 5: 9 x Odroid HC2 + 1 x Odroid HC1 + 1 x Raspberry Pi 4

  • I happened to order a different external case (IB-RD3662U3S), but from the same brand. Should be here today, I didn't read any reports that these don't work with Linux, so my hopes are still high that I won't encounter any problems.
    In the mean time I already got used to OMV and so far like the capabilities of it.
    I also bought a small UPS, altouhght there are barely any power outages in my area ;)

  • I happened to order a different external case (IB-RD3662U3S), but from the same brand. Should be here today, I didn't read any reports that these don't work with Linux, so my hopes are still high that I won't encounter any problems.
    In the mean time I already got used to OMV and so far like the capabilities of it.
    I also bought a small UPS, altouhght there are barely any power outages in my area ;)

    Wow, that thing is expensive over here ($230 on some sites)...


    If you spent anywhere near that much... I'd ditch it completely, buy an RockPro64, sata card, case, etc. and just install the two drives in there and keep the Chromebook independent of all this.

    Air Conditioners are a lot like PC's... They work great until you open Windows.


    Edited once, last by KM0201 ().

  • First annoyance probably related to the case:
    The drives have a different ID after every reboot, since their serial number is displayed as "RANDOM_some characters", and that characters change with every reboot. Therefore smart monitoring doesn't work after reboot and has to be activated again every time.


    Can I put the /dev/by-label in the /etc/smartd.conf instead of the ID? Or is there any other possibility for solving this?

  • I assume you talk about the UUID? The Universal Unique ID that is assigned to a block device and stored in the masterblock of that device. Making each drive unique, even when moved.


    If you see that it change between reboots, then something is very seriously wrong with your system. The whole idea behind UUID is that it will uniquely and persistently identify a block device, even after reboots or if new devices are plugged in or the block devices are plugged in using different interfaces/cables/ports.


    What have you done to break this? Do you reformat drives during the boot process?


    I am very impressed by this and I am truly curious about how you did it.

    Be smart - be lazy. Clone your rootfs.
    OMV 5: 9 x Odroid HC2 + 1 x Odroid HC1 + 1 x Raspberry Pi 4

  • First annoyance probably related to the case:
    The drives have a different ID after every reboot, since their serial number is displayed as "RANDOM_some characters", and that characters change with every reboot. Therefore smart monitoring doesn't work after reboot and has to be activated again every time.


    Can I put the /dev/by-label in the /etc/smartd.conf instead of the ID? Or is there any other possibility for solving this?

    yeah, something is definitely not right

  • Sorry for being so unspecific, so here is a bit more info on the subject:


    What I am taking about is the ID as in /dev/disk/by-id/ :


    The part behind the RANDOM changes with every reboot. This is also, what is displayed in the drives section of OMV (s. screenshot attached).


    The UUIDs are different and probably do not change during reboot (haven't checked that). See the "ironwolf" entries.


    Code
    root@chromebox:~# blkid
    /dev/sda1: UUID="3a4bfe58-2d98-4b26-b97a-988100f310a2" TYPE="ext4" PARTUUID="0443930c-01"
    /dev/sda5: UUID="8e937b4d-72d8-4a67-bfc8-6812c4afc9da" TYPE="swap" PARTUUID="0443930c-05"
    /dev/sdb1: LABEL="ironwolf1" UUID="e8a0ab25-f031-4f04-8faa-f5a8a77950dd" TYPE="ext4" PARTUUID="0d621189-a97d-44ef-b945-0f1bb83ea4ff"
    /dev/sdc1: LABEL="ironwolf2" UUID="680ee841-1f68-4a0d-ab99-f287c6a18ac4" TYPE="ext4" PARTUUID="ea15b7be-7425-44b0-b2a3-92049a0aaca4"
    /dev/sdd1: LABEL="sd" UUID="da30eda9-c56d-4a67-b216-1933ebf21d74" TYPE="ext4" PARTUUID="328e759b-9aa5-4f88-ba8e-3fab2052b4f8"

    But as I said, the s.m.a.r.t monitoring is directed to the "by-id" devices and therefore fails after reboot, since the drive now has a different id:

  • (Had to seprate the posts due to too many characters)


    This is what the smart-log said yesterday when I rebooted. The upper four entries where from former boots with different drive IDs:


    Additionally, I am not sure whether smart status can be checked correctly. "not capable of SMART Health Status check" does not sound like everything is ok. Dunno if it makes sense to investigate further or if I just ditch that case (I can still send it back) and look for a different solution.

  • Had time to think about this over the weekend. I will probably switch to a proprietary solution (QNAP probably), since this USB case probably takes me nowhere.
    My family media is also precious enough to spend a euro or two more, if things run smooth and without hiccups.


    Thanks to everybody that helped to support me, I always appreciate the communities around these kind of projects.


    Cheers

  • I have a Helios4 that at this stage am planning to start off with 2 separate drives. One for shared data and the other for snapshot backups.
    I was planning on using rsnapshot to achieve this, but I'm intrigued by the method that you've developed @Adoby.


    I understand that both use rsync behind the scenes and you have mentioned that you like your process as it's more 'external' and you run it via a cron.
    What are the differences and/or benefits between using rsnapshot and your method?
    Thanks :)

Participate now!

Don’t have an account yet? Register yourself now and be a part of our community!