Newbie trying to build a little NAS for family photos

  • I have to admit that I've never used rsnapshot. So...


    R: If I understand it correctly the rsnapshot works on whole shared folders? I don't like that.
    M: My script will snapshot any folder to any folder destination.


    B: But you can configure rsnapshot in the OMV GUI. You have to get down and dirty on the command line to configure my script and make copies.


    R: Rsnapshot is written in Perl. And the debian source package for rsnapshot consists of 45 files totaling 81 kB. The plugin is 30 kB.
    M: I wrote my script in bash. It is in one single file and excluding comments the whole script is 72 lines long.


    B: But you need one copy of my script for each snapshot target and rsnapshot has a system with configuration files to configure targets. You configure my script by making a copy and edit it.


    R: I think rsnapshot works with separate schedules and folders and rotation for hourly, daily, weekly and monthly snapshots.
    M: My script just puts all snapshots as timestamped sub folders in the destination folder and checks the dates to decide what snapshots to purge.


    B: But my script still handles daily, weekly and monthly snapshots. You can snapshot as often as you like with my script and all snapshots for the current day are retained. But only the oldest for each day is kept as daily snapshot.


    R: Rsnapshot handles yearly snapshots.
    M: My script handles only up to monthly.


    B: But my script allows you to keep for instance 24 monthly snapshots. And you can at any time move a old snapshot out from the destination folder to avoid having it deleted. It will still be hard linked but it will not be automatically purged. Or you could very easily modify the script to handle yearly snapshots as well.


    R: I think rsnapshot keeps careful track of all snapshots in separate "pools" for hourly, daily, weekly, monthly and yearly.
    M: My script just looks at the timestamps of any sub folder in the destination folder.


    B: But I think this means that it can be complicated to delete rsnapshot snapshots. Not sure? Snapshots using my script can deleted freely. Next time all sub folders are scanned again. And purged based on daily, weekly and monthly settings.


    R: Rsnapshot has builtin support for handling ssh keys for remote snapshots. Not sure if the rsnapshot plugin supports it.
    M: My script only works on already mounted filesystems.


    B: But it should be pretty easy to modify my script to use rsync with ssh-keys for remote "pull" snapshots. I use autofs and nfs so all my remote filesystems are always mounted.


    R: Snapshots using rsnapshot are run from the server where rsnapshot is installed. Typically as a "pull" snapshot.
    M: My script can be run anywhere there is a bash shell, rsync is installed and source and destination are mounted.


    B: But I think rsnapshot allows you to have nfs mounts as destination. Not sure if the plugin supports it.


    R: Rsnapshot is great to centralize management and running of snapshots.
    M: My script is great to decentralize management and running of snapshots.


    B: But which is best depends. I have several small SBCs all connected to each other using nfs, autofs and a GbE switch. At night several scripts run in parallel between several pairs of NAS. Together using way more bandwidth than only 1 Gbps. So my script is great for my setup. No coincidence... But to keep things simple I have the exact same setup everywhere. Same folder naming conventions and so on.


    My script continues to improve.


    I now also use it from my laptop to snapshot folders on a 2TB SSD over nfs, autofs and WiFi. Often many small files. E-books in calibre libraries. This is notoriously slow using rsync.


    So I modified my script to reuse the progress of previous partial, never finished, snapshots. So I can remove the laptop from the LAN at any time and later restart a interrupted snapshot without having to start over from scratch. Saves a lot of time. All that was needed was to hide the unfinished snapshot. It was unfinished, now it is .unfinished.


    Also I slightly modified my "launcher" script "all.sh" to automatically prioritize among all the snapshot scripts. So folders that has not been snapshot the longest time are snapshot first. This means that I just double click and run all.sh and then the snapshot scripts in that folder are run in the right order. Oldest snapshot target is always updated first. And any partial interrupted is restarted first.

    Bash: all.sh
    #!/usr/bin/env bash
    for f in `ls -rt *_snapshot.sh`; do
    bash "$f"
    touch "$f"
    done

    Files

    Be smart - be lazy. Clone your rootfs.
    OMV 5: 9 x Odroid HC2 + 1 x Odroid HC1 + 1 x Raspberry Pi 4

  • Hey @Adoby, thanks for the effort you went to with your response! There's an enormous amount of information there and must have taken a while to put together.


    I feel as though I have the information now to make an informed decision. I might start with rsnapshot as a base and then as my knowledge - and confidence - increases. I might switch to using your scripts :)


    Thank you for your detailed response.

Participate now!

Don’t have an account yet? Register yourself now and be a part of our community!