How reliable is my backup strategy?

    • OMV 2.x
    • How reliable is my backup strategy?

      Hello,

      I'm running OMV on an Odroid-XU4, with one external hard drive (2TB). For now, there is no backup, and I manually keep copies of my files on other computers. For now, I use ~500GB of space. I use this NAS to store my music, photo/video, project files,...

      I would like to add a second USB hard drive (2TB or more) that would store a backup of the first hard drive.
      Using rsnapshot, I could do incremental backups and have the possibility to keep multiple version of the files.
      Rsnapshot would run daily, and backup, for example, 7 days and 4 weeks of my data.

      This solution is very simple, "straight to the point", and quite cheap. But I'm afraid of losing data in case of hardware error, unexpected power down,... That is, some unexpected events that could corrupt the whole filesystem, or "silent corrupt" the files.

      I'm wondering how reliable is this strategy? Do you think I should change/improve it?

      The first think that comes to my mind would be to connect the backup hard drive on another device than the XU4, on a raspberryPi or my OpenWRT router, for example. This way, if the XU4 fails and corrupt the filesystem, it wouldn't corrupt also the backup hard drive.

      Any thoughts about this?

      Thanks!
    • Unexpected powerdowns are exactly that, unexpected. At least get a good UPS or some sort of battery backup for the Odroid, and for the 2TB drive if it doesn't draw power from the Odroid.

      Connecting the backup HDD to another system wouldn't prevent it from being corrupted by these other systems. Better would be to simply turn it off when not actively being used for a backup, that is, of course, if you don't mind all the manual work of turning it on and re-connecting it to each and every system you want it linked with.

      Rationale: I learnt the hard way that hard drives just aren't reliable.
    • Rationale: I learnt the hard way that hard drives just aren't reliable.

      That's why I'm trying to find a reliable backup strategy :)

      The UPS/battery is a good idea. I'll do some searching about them.

      Connecting the backup HDD to another system wouldn't prevent it from being corrupted by these other systems

      You're right. But I doubt that both systems (NAS-Odroid/Backup-openwrt) will fail and corrupt the drive at the same time.

      Better would be to simply turn it off when not actively being used for a backup

      Physically disconnecting the drive is a good idea, but not very practical. Do you think that mounting the drive before the backup and unmounting it after would be a good compromise?
    • A backup drive that is connected to the main system is not a good idea. At least you should have (1) your main system data with (2) an external backup that is disconnected after backup (in case of lighting/failure of the main system and (3) a 2nd backup off site (not in your house, so with others or in the cloud in case of fire or theft). If you ask me, having less than 3 copies (main system, onsite backup and offsite backup) does not qualify for a proper strategy. Of course it's up to you, just my 2 cents.
      ASRock H97 Pro4 | 8Gb 1600 | i3 4130T | 3x WD Red 4TB with SnapRAID | Backup to Crashplan & External HDD
    • JF002 wrote:


      You're right. But I doubt that both systems (NAS-Odroid/Backup-openwrt) will fail and corrupt the drive at the same time.
      It is even worse than you can imagine: one system will introduce an invisible corruption, the second system won't correct it, the third system will notice but will screw up the backup when attempting to repair it. You may limit damages with a partitioned hard drives, but in the end, things will always turn out worse than expected.

      Do you think that mounting the drive before the backup and unmounting it after would be a good compromise?
      No. As you may have read, ransomwares are clever enough to attempt mounting a connected drive and encrypt the backup as well. They attacked Windows, they attacked Synology (a Linux derivative), and Mac OS X dodged an attack a few days ago.

      I know this isn't practical, but disconnecting is really the safest way to go against these malwares. At the very least, you could encrypt the backup drive and make the system ask for the passphrase each time it is mounted.

      As for the number of copies, it all depends on the replaceability of the documents: local, full backup is a good first step: no Internet connection would be fast enough for full recovery in a decent amount of time. I would say, keep that first-line backup simple: you don't want to deal with proprietary software images or other exotic setups. Some people keep two local copies at a time, and take one with them at the office / trip, etc.

      As MrMister2015 wrote, an offsite backup is also a necessary. If you can, put a decent server at a friend's house (far from yours, ideally), or subscribe to an encrypted storage space service. iDrive and SpiderOak ar good choices, but you'll have to pay an annual fee, and the space offered often isn't enough for a complete backup. Plus, their clients often don't work in Linux. rsync-based backups are good if you trust the remote server (not an issue if the server is yours), but I don't know yet how to remotely encrypt a backup.

      A third-line backup can be made for sensitive documents. Put them in an encrypted container first, then backup this container to a remote storage space. It doesn't matter if the server you're using has doubtful privacy policy, as they can't read the contents anyway. I have a preference for hubiC, which offers 25GB free, and 100GB for 1€ a month, and an easy-to-use client software. I haven't tried to make it run in Linux, but other people have reported it as working.

      A fourth-line copy can reside on any other computer you happen to have with sufficient diskspace and a good sync client, like btsync. While this isn't a backup in the strict sense, it would allow you to keep on working on a different machine if the main one fails.

      You can still add layers of copies, but I don't know of any software that would ease all the manual work required to keep everything running smoothly.
    • JF002 wrote:

      I'm wondering how reliable is this strategy? Do you think I should change/improve it?


      There is always a balance between convenience and safety. It also depends on how often your files are changing. I've come to the following setup over the years.
      - Main OMV NAS: No RAID anymore, just a couple of 4Tb drives. Consider to use snapraid. I use the LUKS plugin to prevent data loss in case of theft.
      - Onsite backup. A couple of external USB drives. Manually connecting and mounting them every couple of weeks and rsyncing them with the NAS. Those USB drives are also LUKS encrypted.
      - Offsite backup. A couple of drives at my parents house. Bring them over and rsync them 2x per year. Also LUKS encrypted.
      - And (very important): I have a paid subscription with Crashplan. Running Crashplan on the OMV NAS box (and backing up almost 3Tb, not movies or downloads). This gives me the peace of mind that there is some sort of automatic continuous backup process running. I can easily restore individual files throughout the week. In case I change or delete a Word file for example. Crashplan costs about $100 per year unlimited. It can use your own encryption key if you want.

      There are little scenario's I can think of that are not covered this way. Hope this helps.

      Ralph
      ASRock H97 Pro4 | 8Gb 1600 | i3 4130T | 3x WD Red 4TB with SnapRAID | Backup to Crashplan & External HDD
    • MrMister2015 wrote:


      - Main OMV NAS: No RAID anymore, just a couple of 4Tb drives. Consider to use snapraid. I use the LUKS plugin to prevent data loss in case of theft.
      If I may, why not RAID anymore? I thought RAID1 with regular parity check was a good way tp protect against unexpected hard drive failure. Is this continuous backup, or intermittent backup? What would be the benefit of snapraid over RAID1, or what Synology calls "Hybrid RAID" (not proprietary, by the way).


      - Offsite backup. A couple of drives at my parents house. Bring them over and rsync them 2x per year. Also LUKS encrypted.
      I would question the frequency of such backups. Should the worst happen, are you still able to get back to work with 6-month old backups? I know I wouldn't, but maybe I am missing something here.
      Crashplan costs about $100 per year unlimited. It can use your own encryption key if you want.
      Glad to know there's another option besides SpiderOak and iDrive. What do you think of them? Does the Crashplan client work properly in OMV?

      When I designed my backup plan, the main objective was to be able to get back to where I was within 2-3 days (worst case scenario involving new computer purchase and syncing back last-used docs from the remote encrypted backup) to 1-hour (simple internal hard drive failure, as I can run from the external clone if needed).
    • Personally, with 500GB of space requirements, I would put my pictures and music to Google for free. Flicr also has 1TB free for pictures and personal video. I would also consider Crashplan for the rest. Even if you are ready to build a NAS, it only offers so much protection. You cannot beat multiple backups of your data including offsite storage.
    • If I may, why not RAID anymore? I thought RAID1 with regular parity check was a good way tp protect against unexpected hard drive failure. Is this continuous backup, or intermittent backup? What would be the benefit of snapraid over RAID1, or what Synology calls "Hybrid RAID" (not proprietary, by the way).

      I've been using RAID5 for years. In the end I think RAID is too complex for my purpose. Keep it simple. I need to be able to access an individual disk if any errors occur. RAID is good for uptime not for backup. It does not prevent you from deleting files. Snapraid could be usefull to detect bits gone wrong.

      Glad to know there's another option besides SpiderOak and iDrive. What do you think of them? Does the Crashplan client work properly in OMV?
      No, you need to run the Crashplan application from Mac or Win. On OMV you install the Crashplan engine (headless). Works great. See support.code42.com/CrashPlan/4…an_On_A_Headless_Computer

      When I designed my backup plan, the main objective was to be able to get back to where I was within 2-3 days (worst case scenario involving new computer purchase and syncing back last-used docs from the remote encrypted backup) to 1-hour (simple internal hard drive failure, as I can run from the external clone if needed).
      True. My data is mainly our home video collection (800Gb) + my ripped and tagged CDs (1500Gb). If the house burns down, I can wait a couple of days to bring in the external disks. Meanwhile I would use Crashplan to restore my documents to any location and computer in minutes.
      ASRock H97 Pro4 | 8Gb 1600 | i3 4130T | 3x WD Red 4TB with SnapRAID | Backup to Crashplan & External HDD
    • MrMister2015 wrote:

      Keep it simple. I need to be able to access an individual disk if any errors occur. RAID is good for uptime not for backup. It does not prevent you from deleting files. Snapraid could be usefull to detect bits gone wrong.
      I first thought the same, but sadly never encountered a solution that would work 100% of time.

      Each drive formatting has deal-breaking limitations that can't be easily overcome without significant tweaks or expense (proprietary drivers):
      FAT32 can only use 4GB max file size, and many files I would want to transfer hard larger than that. Making a split TAR archive for transfer just slows down the whole process (especially on less-powered computers, as I don't tend to throw away decently working ones), and one cannot quickly transfer hundreds of small files over USB due to overhead.
      ExFAT is better, but I had previously experienced reliability issues when exchanging files from Linux to Windows and Mac, and vice-versa. Windows would see the volume as "dirty", and any repair attempt would make it unusable on the other platforms.
      HFS+ obviously only works on Mac.
      ext4 only works properly on Linux-based OSes.

      No, you need to run the Crashplan application from Mac or Win. On OMV you install the Crashplan engine (headless). Works great. See support.code42.com/CrashPlan/4…an_On_A_Headless_Computer
      Great to see that one backup provider at least thought about customer-owned servers. It would be a pain to restore hundreds of gigs over a non-fibre connection. But it seems to be pre-compiled for a given architecture: that wouldn't work on ARM-based NASes…
    • I personally could never get CrashPlan to set up properly on the OMV nas, sadly. I really like code42 and think they have a great platform, but i found IDrive to be the best solution for me as an offsite backup solution for my omv nas since IDrive supports network drive backup.

      I wanted everything to funnel from all the PCs, macs, and laptops to the Nas, then from the Nas to IDrive.

      I would lose track of my backups if i had to pick and chose random backup locations outside my house, which I know crashplan and idrive can do. IDrive also has geophysical data center locations, so if one happens to go down (with your stuff in it), there is another backup on a different data center.

      Its an easy interface to use. I use both the client and web app. You can control backups from the web app if you were on, say, vacation billions of miles away. I got the 1 TB option that was already on sale to a further cost reduction using a university discount since im a student. So far though, I really like it.
    • forLooped80 wrote:

      I personally could never get CrashPlan to set up properly on the OMV nas, sadly. I really like code42 and think they have a great platform, but i found IDrive to be the best solution for me as an offsite backup solution for my omv nas since IDrive supports network drive backup.

      I wanted everything to funnel from all the PCs, macs, and laptops to the Nas, then from the Nas to IDrive.

      I would lose track of my backups if i had to pick and chose random backup locations outside my house, which I know crashplan and idrive can do. IDrive also has geophysical data center locations, so if one happens to go down (with your stuff in it), there is another backup on a different data center.

      Its an easy interface to use. I use both the client and web app. You can control backups from the web app if you were on, say, vacation billions of miles away. I got the 1 TB option that was already on sale to a further cost reduction using a university discount since im a student. So far though, I really like it.
      forLooped80, I know this thread is fairly old, but I'm new to OMV and need some guidance. I am a current iDrive users and am wanting to setup my OMV exactly as you've described to backup everything to OMV and the backup OMV to iDrive. I'm primarily a Windows/Mac user with only rudimentary Linux experience. Can you (or anyone else for that matter) tell me in detail how you set this up?