Newbie Setup Questions

  • Hi all, new user here and apologies if these are super basic questions but I can't seem to find any answers? I've just got up and running OMV5 on an RPi4B with a 4TB NAS drive and now I'm a bit stuck with how to achieve what I want to achieve! There's loads of great videos explaining how to install and get OMV up and running which is great, for literally every docker, container, software etc but none that I can see explaining which ones to use where and when! I could do with some advice on what the best thing is to use for my pretty simple needs?! Of course I'm aware that there are many ways of skinning a cat, but some pros and cons for each would be great! ;)


    What I'd like to do is to use the NAS as a backup to the 4tb drive in my PC, and I've also got a 2tb external USB as another backup for really important docs and photos as an offline backup (i'd just plug it in and manual backup). I'd also like to be able to access my photos from anywhere and have them synced between PC, NAS and be able to access them from my phone/laptop etc. I'd also like to automatically backup my camera photos to a folder on the NAS, that will be synced with the PC - every now and then I'll go back in and sort them into folders to be stored in the normal photos library. I've got some movies and as a stretch project, it'd be nice to be able to watch on my TV with it's built in Plex app.


    So, having done some research and reading around I've been looking at Plex to arrange the photos, videos and a few movies I've got and then Nextcloud to do the syncing between PC/NAS? But I read that it may be better to use Syncthing to sync the folders, then use nextcloud on top for a nice gui? Or rsync?!


    Apologies for the long post, but there's lot's of options and I'm not too sure of the best way forward! As I said, there's loads of videos explaining how to install stuff, but not many I could see explaining which ones to use where/when? I've got OMV5 installed (thanks TechnoDadLife!) and I am sharing some folders, so that's all working and I'm pretty happy with installing stuff and getting it working, but what to use?! :/

  • So I've been messing around a bit with it and eventually worked out how to get Syncthing installed and working... took quite a bit of reading and playing around to work out how to map the volumes correctly so the home was on the right drive and folder. Have also got Plex installed and working and am just looking at Nextcloud now to see if I want to install that too. Would be good to have access to my files from outside easily. Any thoughts would be most appreciated for sure.


    I'm certainly discovering the limitations of the Pi tho... it's certainly not quick. But it's OK, as if this works out OK and I can figure it out I think I'll build a new NAS based on something else more powerful.

  • Yes, it would appear that the Raspberry is the gateway drug into “something else more powerful.”

    RAID is NOT a backup and not useful for most home users. Rsync makes true backup and restoration stupid easy, and it's built right in to OMV. Use this command in a Scheduled Job: rsync -av --delete /srv/dev-disk-by-label-NAMEofSOURCEdisk/ /srv/dev-disk-by-label-NAMEofDESTINATIONdisk/

    Hardware: OMV 5 (current) - NanoPi M4: Nextcloud, Plex, & Heimdall - Acer Aspire T180: backup - Odroid XU4: Pi-Hole (DietPi) - Odroid HC2, Raspberry Pi 3B+, and HP dx2400: testing.

  • As you're finding, starting off and building is a discovery process. I started off with an R-PI (an older model, a 2B) and quickly learned of its limitations.


    Frankly, it boils down to what you want to do. For running a straight forward file server, with maybe 2 or so light weight Dockers, an R-PI is fine. It any case, it's a good place to start until you figure out what you want to do. I believe you'd find, "what you want to do" is an evolution of sorts. As Agricola has indicated with the "gateway drug" analogy, a lot of users start out with an R-PI or other SBC type. For some that's enough. Others go on to buy a dedicated workstation or server grade hardware for higher performance.

    What you need to consider, above and beyond all other considerations, is an additional hard drive of equal or larger size to your existing data drive, so you can have simple and easy to restore backup. Once you centrally locate all your files, backing them up is crucial if you want to keep them long term. Rsync can be used for backup. It's easy to use and can be automated. Agricola has the generic command line in his signature above. For a step by step guide to setup Rsync backup and (more importantly) how to restore it, take a look at Data Backup in this -> Guide. There are other items in this guide that you may find of interest, like setting up notifications so if something is going south on the server's data drive you'll get an E-mail notification.


    "If / When" you have a backup drive on your server, I'd work with files locally on my workstation and, when finished, run them up to the server. If your workstation is running Windows there are plenty of free and easy to use file sync app's. One I've used in the past is free file sync.


    If you're not backing up your client, mirroring your client's files to the server will work fine. If you have a backup drive for the server AND plan on backing up your client's boot drive to the server, that gets into additional considerations to prevent eating up a lot of hard drive space.

  • Thanks for the reply, to be fair the RPi is OK now the Syncthing job has completed... 1.5Tb took a while at 10Mb/s tho! From what I read, the hashing and encrypting knocks the performance of the Pi a bit - a straight file transfer from windows explorer samba shared files was about 30 Mb/s. I started at 1Mb/s but figured out I was using a relay, so got a direct connection and it was a lot better :D Now its all synced, it should be fine keeping up I reckon.

    Keeping a backup is certainly at the forefront of my mind, and after doing lots of reading and researching the more important it seems. I started the process thinking that the Syncthing backup would be OK, but quickly realised, just as you point out, that it's not! So I've got an old USB 2 2tb external that will work OK for now, and this RPi will prob become the backup to whatever I buy/make/build next I think.


    Next thing I think I'm going to experiment with is Next Cloud, so I can access my documents from the web. Seems pretty straightforward and should work quite well from what I've seen about it. Plex is up and running too, so pleased with that.


    Been looking at HP Microservers as a reasonably cheap way to get into a 'proper' server at home, probably a gen 7, as doubt I'd need the power/cost of a gen 8. Certainly seems cost effective way for what I will probably want to do with it anyway, just storing files pretty much. If I go down that path, then the RPi will make a good offline backup I would have thought - plug it in every month or so and run and rsync job?


    One other question tho, in the case of a powercut or a PC crash, will rsync/syncthing/any other sync backup recover OK? will it corrupt files on the host end?

  • Power spikes and cuts can hurt your system os. It’s best to have a dd backup of your sd card for this. If stormy weather is headed my way I usually shut the servers down. A UPS would be nice, but that is a different post.


    Nextcloud is great but a bit daunting to set up. This guide is the best one in my opinion. Read through it several times and read the questions supporting page. Here again is where a system os is a must: if you mess up just fall back to your (current) backup and start over.


    On backups: on about page 59 of the guide crashtest linked to above (he wrote it) is a description of a mirrored backup (and restore) that just works. It’s a great first step in the backup department.


    On hardware: crashtest is better able to address that, but my backup is just a 1.5ghz dual core Acer desktop I got for free. I turn it on Sunday evenings and let Rsync backup the shares on my main server with scheduled jobs in the wee hours of Monday. Next morning the backup machine is checked for updates and then shutdown until next week. Just about anything that will run OMV will do.

    RAID is NOT a backup and not useful for most home users. Rsync makes true backup and restoration stupid easy, and it's built right in to OMV. Use this command in a Scheduled Job: rsync -av --delete /srv/dev-disk-by-label-NAMEofSOURCEdisk/ /srv/dev-disk-by-label-NAMEofDESTINATIONdisk/

    Hardware: OMV 5 (current) - NanoPi M4: Nextcloud, Plex, & Heimdall - Acer Aspire T180: backup - Odroid XU4: Pi-Hole (DietPi) - Odroid HC2, Raspberry Pi 3B+, and HP dx2400: testing.

  • Ah OK, good information again... That's tonight's job I think - figure out how to clone the boot SD card without taking the card out if possible - the case that the Pi and HDD are in doesn't have a slot to remove the card, so you've got to unscrew the case to get it out... Once I've done that on to Nextcloud setup.


    Just been reading the guide @chrashtest wrote, don't know how I didn't spot it before, but it's excellent and just what I was after to help explain the basics :) Looks like some bedtime reading for me!


    Great shout on the backup machine... I've not got any old PC's here, but it certainly is the sort of thing that people do tend to give away when they get a bit old. Perfect for backup use tho!

  • The only way I have ever backed up a card was to shut down, pull the card and make an .img file of it on my desktop machine and then burn the image to a new sd card. I then boot on the backup to see if it’s any good. Here’s a good article for that for Windows or Mac.


    There is a thread ongoing here that deals with backing up a live system but I haven’t given it a try yet. If it works it would be a game changer for me. I would still want to boot with the backup to make sure it was good.

    RAID is NOT a backup and not useful for most home users. Rsync makes true backup and restoration stupid easy, and it's built right in to OMV. Use this command in a Scheduled Job: rsync -av --delete /srv/dev-disk-by-label-NAMEofSOURCEdisk/ /srv/dev-disk-by-label-NAMEofDESTINATIONdisk/

    Hardware: OMV 5 (current) - NanoPi M4: Nextcloud, Plex, & Heimdall - Acer Aspire T180: backup - Odroid XU4: Pi-Hole (DietPi) - Odroid HC2, Raspberry Pi 3B+, and HP dx2400: testing.

  • gderf comments quite a bit about that very thing in the thread I linked to above. I’m not sure if it was for a live backup though. The code looks pretty much like what I do with a manual dd and restore in Mac Terminal, but I didn’t look that closely. I get spooked when I see the word “script”. I’ve no experience with setting up and implementing scripts. I need to dig into that but haven’t had the time to sit down and read up.

    RAID is NOT a backup and not useful for most home users. Rsync makes true backup and restoration stupid easy, and it's built right in to OMV. Use this command in a Scheduled Job: rsync -av --delete /srv/dev-disk-by-label-NAMEofSOURCEdisk/ /srv/dev-disk-by-label-NAMEofDESTINATIONdisk/

    Hardware: OMV 5 (current) - NanoPi M4: Nextcloud, Plex, & Heimdall - Acer Aspire T180: backup - Odroid XU4: Pi-Hole (DietPi) - Odroid HC2, Raspberry Pi 3B+, and HP dx2400: testing.

  • Thanks for that, I went through that thread earlier and I've set up the backup plugin and it does appear to be working, not tested yet tho, but the zip file created does seem to be complete.


    Just need to test to confirm it works - I've ordered another SD card so I don't have to overwrite any of my other ones and to keep as a backup card.

  • Sounds like you are on the right track. I was in the process of checking out last night’s copy using the backup plugin. I had just burned it using usbimager and was getting ready to shut down the hc2 when we lost power. So I will check the backup later. I am not committed to the backup plugin yet. I still need proof that the SD card actually works.

    RAID is NOT a backup and not useful for most home users. Rsync makes true backup and restoration stupid easy, and it's built right in to OMV. Use this command in a Scheduled Job: rsync -av --delete /srv/dev-disk-by-label-NAMEofSOURCEdisk/ /srv/dev-disk-by-label-NAMEofDESTINATIONdisk/

    Hardware: OMV 5 (current) - NanoPi M4: Nextcloud, Plex, & Heimdall - Acer Aspire T180: backup - Odroid XU4: Pi-Hole (DietPi) - Odroid HC2, Raspberry Pi 3B+, and HP dx2400: testing.

  • A few last words on backup.

    this RPi will prob become the backup to whatever I buy/make/build next I think.

    The above is a path to the ultimate backup solution in my opinion. My preference is to have a minimum of two servers. The second, a backup server, is configured exactly as the first with the same users, permissions, shared folders and even samba shares set up the same way. (Dockers, if any, can be configured identically as well, and disabled.)

    There are two differences between the main and the backup:
    - Rsync is set up, on a per network share basis at the backup server, to "pull" replicate shares from the main server onto the backup. Sync jobs are setup in a series and time staggered so they don't all run at once.
    - Samba shares (after creation and testing for access) are turned off. In that way, when the backup server is on-line, the second set of samba shares don't show up at LAN clients.

    After an "Rsync day" or after manually triggered sync jobs, the backup server is setup to shutdown (automated) in the early morning hours. I run the series of sync jobs once every two weeks or so, then the backup server shuts itself down. The server becomes a cold backup with hard drives that are aging very slowly.

    If the main server dies, I bring the backup server on line, disable Rsync replication, disable the auto nightly shut down and re-enable Samba shares and Dockers if applicable. So, if / when the main server is discovered to have a serious problem, I can be back on line with the backup server and with data intact, in 5 minutes or so.

    This is just food for thought, as you consider possibilities.

  • Ah that's useful too, thanks.


    I'm just in the process of backing up to my 2TB external drive plugged into the Pi so I've got a backup for now. I'm going to set up individual rsync jobs for each folder I want to backup to the external (Docs, Pictures, Videos etc). Is there a way to setup a job to run them all concurrently? So lets say I plug it in every two weeks or so, can I set it so either it does it as soon as its plugged in, or is it better to have a button/script to run the jobs one after the other?


    Also, is there a way to see the progress of each rsync job? I've got one running now but I don't know how to check on progress?

Participate now!

Don’t have an account yet? Register yourself now and be a part of our community!