Question: Can rsync be used without using the GUI?

  • Hi,


    I received a rather complex rsync script that I would like to use on my OMV server
    because I'm new with OMV, I would like to know if it possible to use this "rsync" script directly
    without using the gui?
    If it's possible, is it also advisable? or is using the gui a more stable way of using rsync on OMV?

  • Thanks, I have read that topic, but after reading it was for me not clear if who was to blame these "problems" (user related, software related, or mixture of both)
    Rsync is pretty stable and good system, and many people are using Rsync daily in the world.


    But what I would like to know; is what the best practice of using rsync on OMV?

    • Official Post

    I use rsync a lot on my OMV servers.


    And I have never used rsync from the OMV GUI.


    Either I use rsync on the command line or in scripts. Typically to create and purge rsync snapshots to backup folders.


    Typically the backup snapshot scripts are run daily from a cron job. I edit crontab directly. have never used the OMV GUI to schedule jobs.


    On each backup server I have a folder called snapshots. There I have a bunch of snapshot scripts as well as a "master" script:


    Code: all.sh
    #!/usr/bin/env bash
    for f in *_snapshot.sh; do
       bash "$f"
    done

    And a line in /etc/crontab like this:


    0 3 * * * adoby cd /sharedfolders/nas1/snapshots && /bin/bash all.sh


    Then, at 03:00, every night, all the snapshot scripts are run. Currently that is:


    local_media_snapshot.sh
    shared_media_snapshot.sh
    download_snapshot.sh


    These scripts creates and purges rsync snapshots on nas1 of specific folders from nas0. It is intentionally a rsync "pull" to avoid problems with filled mount points.


    I verify that snapshots are made and purged by looking in /sharedfolders/nas1/snapshots.


    I also attach a copy of local_media.sh. But it is only meant for information! Don't use it! No support is given! You should only use backup scripts that you have written yourself an understand fully!


    Note: All my OMV servers mount all shares of all my OMV servers in /srv/nfs using autofs and NFS.

    • Official Post

    I use rsync from the command line and the web interface a lot. About half of the rsync jobs in the web interface are syncing to another drive in the same server and the other half are syncing to remote servers.

    omv 6.0.46-5 Shaitan | 64 bit | 5.19 proxmox kernel | plugins :: omvextrasorg 6.1.1 | kvm 6.2.5 | mergerfs 6.3.3 | zfs 6.0.12
    omv-extras.org plugins source code and issue tracker - github


    Please try ctrl-shift-R and read this before posting a question.

    Please put your OMV system details in your signature.
    Please don't PM for support... Too many PMs!

  • Thank you Adoby and Ryecoaaron for your answers, this exactly was the information were I was looking for :)
    Also nice to hear thatRsync can be used without the GUI, without problems
    Yesterday my colleague an I tested our own made rsync script on our OMV server, and it
    worked perfectly!



    Our situation:
    To reduce costs, better security and extended lifespan we decided to build our own storage server (with OMV) for the company we work for.
    The server is self build with a 8 bay Server Chassis,with 7 X 6TB SATA hard disks for storage) with no raid controller, the hard disks are connect to the motherboard directly
    (we use OMV software RAID 6)


    We have a lot of confidence in OMV therefore we are going to test our OMV Storage Server in an enterprise server environment, for important/critical purposes.
    (On our own risk of course ;)


    My colleague (who’s experienced with rsync) wrote a rsync scrip (see attachment) that makes incremental backup’s (from the client servers point of view)
    (Also for this script: use this script for study only! and only when you understand the script completely, no support is given)



    The next step for us would be to implement this script in OMV
    So that we can use the GUI functionality. and run the scripts from the OMV server
    and that we can run the scripts from one centralpoint instead of run the scripts from the servers that we want to backup
    Could you maybe give us some advise about this?



    Thanks for you all for sharing your experiences, it helped us alot!

    • Official Post

    I would not try to use the OMV GUI for this. Perhaps only use OMV to set up the shares.


    Instead I would use plain crontab and scripts.


    I would also avoid having a central point from where everything is run.


    Instead I would run the cron jobs and scripts directly on the backup server(s).


    However I would have a central monitoring system. From which I can check that everything works and update scripts and so on. Perhaps even some form of dashboard. If you send log files and/or error messages to some location in the cloud you should be able to use some online dashboard service to provide a nice web based online dashboard. There are plenty of online dashboard services. Some free...


    It is perfectly possible to run scripts remotely from a central system, on the backup servers. SSH can be used for that. Or SaltStack I belive OMV 5 will use/support Salt? But running scripts remotely means that you build an interdependent chain of systems and events. And you can increase the probability of errors drastically. KISS rules!


    But it makes sense to have a central system where you can easily access and edit the remote scripts. Perhaps even directly from a wiki that documents the system and how everything works together. If it is a good wiki, with a programmable backend, it might even be used to create a nice dashboard.


    RAID arrays may also mean that you drastically increase the probability of errors and increase the time to get up and running again. If one drive fail you may loose the contents of the whole array. If you use RAID you should be very comfortable with what to do if the array goes down. It is very easy to set up a RAID array. You can use a nice GUI. But when (not if) something happens you need to use the command line to fix it. And by that time you are likely to have forgotten exactly what you did when testing for errors in the array during setup, unless you work at a data center and do this every day. You may even have a new job somewhere else.

    • Official Post

    I would not try to use the OMV GUI for this. Perhaps only use OMV to set up the shares.

    I use OMV for remote syncs. Works very well. If the systems are connected via local network, I wouldn't use ssh - just rsync protocol.

    omv 6.0.46-5 Shaitan | 64 bit | 5.19 proxmox kernel | plugins :: omvextrasorg 6.1.1 | kvm 6.2.5 | mergerfs 6.3.3 | zfs 6.0.12
    omv-extras.org plugins source code and issue tracker - github


    Please try ctrl-shift-R and read this before posting a question.

    Please put your OMV system details in your signature.
    Please don't PM for support... Too many PMs!

  • Thank you Adoby & Ryecoaaro,



    Thank you for your advice and tips, they helped me very well
    I am going to think carefully about what my next steps will be to improve the Rsync script implementation
    We are very satisfied with OMV, it is a very versatile, reliable and secure storage solution

Participate now!

Don’t have an account yet? Register yourself now and be a part of our community!