Posts by Adoby

    Many docker apps store frequently variable data and metadata. Perhaps in a database. This may lead to random writes. Examples are torrent download clients or media managers with frequently updated metadata.

    I have a spare Crucial MX100 2.5 500 Gig SSD.

    Should i keep as intended installing OMV onto 64GB Flash, and then have the 500Gb SSD where i install the docker and all Docker stacks?

    That is how I would do it. The rootfs on a USB thumb drive and the flash memory plugin installed. And everything docker on a SSD. Media files on spinning disks.

    Yes, you will be able to add and access the data drives after a reinstall.


    Just to be on the safe side, it might be a good good idea to disconnect the data drives before you reinstall. Then, when OMV is up an running fine, reconnect and add the drives. And you should be able to mount the filesystems and recreate shares and so on.


    Then, clone the rootfs...

    If your 4TB WD Reds are SMR, then you may want to avoid storing the dockers on them and perhaps use a small SSD for dockers instead.


    Dockers often use a lot of random writes. That may cause significant write magnification causing the SMR drives to slow down and even wear out prematurely. A good SSD will handle that much better. That said, some of the newer quad-bit SSDs have poor write performance. Worth checking before you buy.

    Should work fine!


    One complication is that the 4TB Red drives might be SMR. Shingled Magnetic Recording. I am not sure. SMR may lead to very poor performance when used in a RAID. There has been a lot of discussion about this, since it is not always obvious if a drive is SMR or not.


    WD has published a list of drives using SMR.


    https://blog.westerndigital.com/wd-red-nas-drives/


    But perhaps you should reconsider RAID5? It seems to be an unnecessary complication? Also you don't mention anything about backups. I would argue that good backups are much more important than RAID. A simple and robust setup is to use two internal drives, one for normal shared storage and one for backups of the first. In addition perhaps backup to an external USB3 drive and/or another NAS or the Cloud.


    Cheap SMR drives are normally fine for non-RAID general storage.


    The praxis is to install OMV and then get shares up. And then install dockers for special functionality, like surveillance, media streaming and downloading. It might be a good idea to use a small and fast extra disk volume dedicated to docker usage. Perhaps a small SSD, just for dockers? This is to avoid wearing out the USB thumbdrive and to improve performance, especially if you use SMR drives for data storage, to allow the data volumes to be dedicated to store data that is rarely accessed using random writes. SMR and random writes is a bad combination.


    I don't know anything about "bBittorrent", I assume it is a bittorrent client? If that is not easily available as a docker, there are many alternatives.

    Would you do it over SSH or use something else? I'm planning to do remote backup too and I still haven't sorted the security part.

    Also, for remote backup would you use rclone or stick with rsync? I don't know the exact difference between the two.

    I haven't had reason to consider it. But I would probably start with a VPN and rsync. And see what performance I would get. Possibly use a local backup server to buffer the backups and sync from that.

    ... and that is what I do locally with rsync. As I backup I keep multiple versions of my files. Typically I keep 7 daily, 4 weekly and 3 monthly, but I can change that as I wish. So I can go back and retrieve a file (or a folder of files) as it was last Thursday. Or last month. Or the month before that.


    I have posted it before, but here is my script for versioned rsync snapshot style backups.


    https://github.com/WikiBox/snapshot.sh


    Should be relatively straightforward to modify it for remote backups.

    I suspect that you need to be a little more systematic.


    Most likely you configure or install something and then the rootfs becomes ro. As you setup your NAS, backup/clone the rootfs after you made some changes and rebooted and verified it works OK. Carry on step by step. When you discover problems, you may be able to determine what the problem is. Just restore the previous working backup and try again. But in smaller steps, to pinpoint the problem.

    You can do versioned "snapshot" style backups using rsync. That is how I do my backups. Works fine. But I do it locally in my LAN. But it would work just as well remotely. Only new or modified files need to be synced. Unchanged files are hardlinked from a previous snapshot.


    For information about how to use rsync, there are tons of pages online. I suspect that rsync is one of the best documented file utilities there are. Find some tutorial and do some experimenting.

    One option:


    One drive for shared data. One drive for automatic frequently versioned backups.


    I wrote my own rsync script to do versioned daily snapshots from one drive to another. That way I can, if needed, "travel back in time" and retrieve a file as it was yesterday, a week ago or three months ago. Each snapshot looks like a full copy, but unchanged files are hard linked between snapshots, so each snapshot (almost) only takes up space needed to account for any new or updated files. All files are uncompressed and unencrypted.


    https://github.com/WikiBox/snapshot.sh


    rsnapshot is available for OMV, it does something similar.

    I have very good experience with 8 & 12 TB Ironwolf HDDs and 16TB Seagate Exos in HC2s. Just works. But most likely any good 3.5" SATA drives will work fine.


    Please note that using the physical disk properties in OMV will cause problems with some drives. Especially newer models. The reason for this seem to be that the drives are not fully compatible with hdparm, that OMV use to change drive settings. As long as you don't touch the physical drive settings in OMV you should be fine. The drive defaults are OK.


    The problems manifest as the drive not mounting and/or unmounting. And corrupt drive settings.


    Also note that you set HD spindown by flashing the sata/USB bridge firmware. Not in the OMV settings.

    A HC2 is still a great basis for a home NAS over GbE. Extremely stable and performant. But: Can only handle a single SATA drive and has only one USB2 port. Also ARM32, not ARM64. So it is a bit old now.


    The HC2 is a nice self-contained package.


    A RPi4 has finicky USBS3. You may find that some USB3 enclosures work fine, some don't. Also using USB to connect drives means less stability and in some cases even power problems and reboots. But the RPi4 is ARM64. 2GB would be fine for OMV, but 4GB better and the price difference is small...


    You may need to buy a case for the RPi4 and also use powered external drives. Messier.


    I have a RPi4 connected to a powered 4 bay USB3 enclosure with 4x16TB drives. And it is rock steady as long as I don't try to connect anything else. If I try to connect some other USB device, like a thumb drive or a un-powered SSD, the RPi4 crash after a few minutes, presumably because of power problems.


    If you just are after long term basic NAS functionality and stability, and ARM32 is OK, go with a HC2. If you need USB3 and ARM64, and don't mind extra cables, go with RPi4.


    Both provide very good basic NAS functionality for a home GbE network.

    The purpose of the flash memory plugin is to minimize writes to flash memory. You disabled the swap partition on flash memory. That seems to be reasonable if you want to minimize writes to flash.


    During boot OMV detects that swap is disabled. That is expected.


    However, you are using a SSD, not a SD card or a thumb drive. Using a SSD with better endurance than other simpler types of flash memory may make it less important to use the flash memory plugin at all. The SSD can be expected to last several years even without the flash plugin, also with swap to the SSD activated.


    <personal opinion>


    Personally, If I booted from SSD I would not use the flash memory plugin. But I would on the other hand most likely not setup a OMV NAS to boot from a SSD, unless I had several I had to use for that purpose. Instead I would setup OMV to boot from SD card or a USB thumbdrive, with the flash memory plugin, and use the whole SSD for dockers and databases and so on. I would not use any swap with OMV, expect possibly to a small amount of compressed RAM. Instead I would make sure that the OMV NAS had plenty of RAM. Enough to run all dockers and still have plenty of disk caches left.

    Without a SSD:

    1. Boot from SD card or a thumbdrive with flash memory plugin. Dockers on spinning drives. No swap or swap to compressed RAM.


    With one SSD:

    2. Boot from SD card or a thumbdrive with flash memory plugin. Dockers on SSD. No swap or swap to compressed RAM or SSD.


    With two SSD:

    3. Boot from SSD without flash memory plugin. Dockers on the other SSD. No swap or swap to compressed RAM or boot SSD.


    </personal opinion>

    You say that you want to replace the two 4TB drives with two 8TB drives.


    Then do that.


    Please note that my point 1 is critical. Please, please don't skip that. Never skip backups. If you skip backups you will lose your data, sooner or later.


    So I assume that you have good recent backups. If not, make sure to fix that before doing anything else. Otherwise you may lose all your data. Not having good backups is a way to ensure that you are likely to lose data.


    Only then replace the 4TB drives with the 8TB drives.


    Afterwards copy the contents of your most recent backups back to the new filesystem. How you do that depends on how you did the backups. External hard drive or over the network. Do the restore the same way, but in the other direction.


    If you continue to use RAID1 or not is up to you. I assume that you have very good reasons if you do? Using RAID1 instead of backups is a really, really bad reason.

    1. Make sure your backups are up to date.
    2. Replace the drives.
    3. Reconfigure OMV to use the new drives instead of the old.
    4. Restore the backup to the new filesystem.


    Alternatively, if you have no backups, consider the good 4TB drive to be the backup, and restore/copy data from that in step 4.


    It is faster to do a local restore/copy over SATA rather than over GbE.