USB Backup filesystem practices for reading on multiple platforms

Just finished installing the beta OMV 5 from ISO on an older Dell server and gotten some shares up and working. Looks like a cool project.

I'm now trying to set up USB Backup. I understand from some of the various threads that Rsync doesn't get along well with NTFS or whatever Microsoft is calling their default filesystem these days.

If I must I can format Ext4 or something else that I know is fine for Linux, but in this household I'm the only one with significant Linux experience. My principal reason for setting up a NAS in the first place is for secure storage of family pictures and other keepsake content that would need to be accessible should something happen to me. As such a filesystem that can be readily read by Windows in addition to Linux is much preferred.

What filesystem would you recommend a 5TB external USB hard disk be formatted with in order to accomplish both being usable to OMV and be readily used by Windows should it prove necessary?

For what it's worth we are already doing Blu-ray burns for backups as well, but should a recovery from Blu-ray prove necessary that's going to be quite the ordeal as the number of backup discs grows.

Thanks,

Just to see if the problems I encountered were related to the USB disk having a factory partition table with two partitions, I went ahead and repartitioned and tried an exfat format. Test mounting the partition worked well enough.

When I tried recreating a single USB Backup I received the following error result:

[font='Consolas, Courier New, Courier, monospace'][size=8]Error #0:OMV\ExecException: Failed to execute command 'export PATH=/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/sbin; export LANG=C.UTF-8; omv-salt deploy run usbbackup 2>&1' with exit code '1': /usr/lib/python3/dist-packages/salt/utils/decorators/signature.py:31: DeprecationWarning: `formatargspec` is deprecated since Python 3.5. Use `signature` and the `Signature` object directly *salt.utils.args.get_function_argspec(original_function)/usr/lib/python3/dist-packages/salt/utils/decorators/signature.py:31: DeprecationWarning: `formatargspec` is deprecated since Python 3.5. Use `signature` and the `Signature` object directly *salt.utils.args.get_function_argspec(original_function)debian:---------- ID: create_usbbackup_scripts_dir Function: file.directory Name: /var/lib/openmediavault/usbbackup.d Result: True Comment: Files cleaned from directory /var/lib/openmediavault/usbbackup.d Started: 07:02:50.037034 Duration: 2.296 ms Changes: ---------- removed: - /var/lib/openmediavault/usbbackup.d/systemd-f8dc038d4eb5a08afce891c2b6ac629f---------- ID: remove_usbbackup_rsync_scripts Function: module.run Result: True Comment: file.find: [] Started: 07:02:50.040127 Duration: 1.044 ms Changes: ---------- file.find:---------- ID: remove_usbbackup_systemd_scripts Function: module.run Result: True Comment: file.find: [] Started: 07:02:50.041341 Duration: 1.011 ms Changes: ---------- file.find:---------- ID: disable_usbbackup_systemd_unitfile_openmediavault-usbbackup-f8dc038d4eb5a08afce891c2b6ac629f.service Function: service.disabled Name: openmediavault-usbbackup-f8dc038d4eb5a08afce891c2b6ac629f.service Result: True Comment: Service openmediavault-usbbackup-f8dc038d4eb5a08afce891c2b6ac629f.service has been disabled, and is in the desired state Started: 07:02:50.624462 Duration: 273.75 ms Changes: ---------- openmediavault-usbbackup-f8dc038d4eb5a08afce891c2b6ac629f.service: True---------- ID: remove_usbbackup_systemd_unitfile_openmediavault-usbbackup-f8dc038d4eb5a08afce891c2b6ac629f.service Function: file.absent Name: /lib/systemd/system/openmediavault-usbbackup-f8dc038d4eb5a08afce891c2b6ac629f.service Result: True Comment: Removed file /lib/systemd/system/openmediavault-usbbackup-f8dc038d4eb5a08afce891c2b6ac629f.service Started: 07:02:50.898416 Duration: 0.664 ms Changes: ---------- removed: /lib/systemd/system/openmediavault-usbbackup-f8dc038d4eb5a08afce891c2b6ac629f.service---------- ID: configure_usbbackup_rsync_script_2a6a9c07-dbf9-4fe7-8694-e9a8c82a7d7e Function: file.managed Name: /var/lib/openmediavault/usbbackup.d/rsync-2a6a9c07-dbf9-4fe7-8694-e9a8c82a7d7e Result: False Comment: Unable to manage file: Jinja syntax error: no filter named 'prettify_dir_path'; line 9[/size][/font][size=8][font='Consolas, Courier New, Courier, monospace']

--- [...] {%- set runfile = '/var/run' | path_join(script_prefix ~ job.uuid) -%} {%- set id = job.devicefile | md5 -%} {%- set mount_path = id | make_mount_path -%} {%- if job.mode == 'push' -%} {%- set srcuri = salt['omv_conf.get_sharedfolder_path'](job.sharedfolderref) -%} {%- set desturi = (mount_path ~ salt['omv_conf.get_sharedfolder_name'](job.sharedfolderref) if job.usesubdir else '') | prettify_dir_path -%} <====================== {%- elif job.mode == 'pull' -%} {%- set srcuri = (mount_path ~ salt['omv_conf.get_sharedfolder_name'](job.sharedfolderref) if job.usesubdir else '') | prettify_dir_path -%} {%- set desturi = salt['omv_conf.get_sharedfolder_path'](job.sharedfolderref) -%} {%- endif -%} #!/bin/sh [...] --- Started: 07:02:50.899182 Duration: 63.818 ms Changes:---------- ID: configure_usbbackup_systemd_script_a34b92a0e66e3a470d2e646682183f66 Function: file.managed Name: /var/lib/openmediavault/usbbackup.d/systemd-a34b92a0e66e3a470d2e646682183f66 Result: True Comment: File /var/lib/openmediavault/usbbackup.d/systemd-a34b92a0e66e3a470d2e646682183f66 updated Started: 07:02:50.963109 Duration: 49.197 ms Changes: ---------- diff: New file mode: 0750---------- ID: configure_usbbackup_systemd_unitfile_a34b92a0e66e3a470d2e646682183f66 Function: file.managed Name: /lib/systemd/system/openmediavault-usbbackup-a34b92a0e66e3a470d2e646682183f66.service Result: True Comment: File /lib/systemd/system/openmediavault-usbbackup-a34b92a0e66e3a470d2e646682183f66.service updated Started: 07:02:51.012410 Duration: 34.638 ms Changes: ---------- diff: New file mode: 0644---------- ID: usbbackup_systemctl_daemon_reload Function: module.run Name: service.systemctl_reload Result: True Comment: Started: 07:02:51.047208 Duration: 0.439 ms Changes:---------- ID: enable_usbbackup_systemd_unitfile_a34b92a0e66e3a470d2e646682183f66 Function: service.enabled Name: openmediavault-usbbackup-a34b92a0e66e3a470d2e646682183f66 Result: True Comment: Service openmediavault-usbbackup-a34b92a0e66e3a470d2e646682183f66 has been enabled, and is in the desired state Started: 07:02:51.047749 Duration: 437.074 ms Changes: ---------- openmediavault-usbbackup-a34b92a0e66e3a470d2e646682183f66: True[/font][/size][font='Consolas, Courier New, Courier, monospace'][size=8]

Summary for debian------------Succeeded: 9 (changed=8)Failed: 1------------Total states run: 10Total run time: 863.931 ms in /usr/share/php/openmediavault/system/process.inc:182Stack trace:#0 /usr/share/php/openmediavault/engine/module/serviceabstract.inc(60): OMV\System\Process->execute()#1 /usr/share/openmediavault/engined/rpc/config.inc(164): OMV\Engine\Module\ServiceAbstract->deploy()#2 [internal function]: Engined\Rpc\Config->applyChanges(Array, Array)#3 /usr/share/php/openmediavault/rpc/serviceabstract.inc(123): call_user_func_array(Array, Array)#4 /usr/share/php/openmediavault/rpc/serviceabstract.inc(149): OMV\Rpc\ServiceAbstract->callMethod('applyChanges', Array, Array)#5 /usr/share/php/openmediavault/rpc/serviceabstract.inc(577): OMV\Rpc\ServiceAbstract->OMV\Rpc\{closure}('/tmp/bgstatusYA...', '/tmp/bgoutputb2...')#6 /usr/share/php/openmediavault/rpc/serviceabstract.inc(159): OMV\Rpc\ServiceAbstract->execBgProc(Object(Closure))#7 /usr/share/openmediavault/engined/rpc/config.inc(186): OMV\Rpc\ServiceAbstract->callMethodBg('applyChanges', Array, Array)#8 [internal function]: Engined\Rpc\Config->applyChangesBg(Array, Array)#9 /usr/share/php/openmediavault/rpc/serviceabstract.inc(123): call_user_func_array(Array, Array)#10 /usr/share/php/openmediavault/rpc/rpc.inc(86): OMV\Rpc\ServiceAbstract->callMethod('applyChangesBg', Array, Array)#11 /usr/sbin/omv-engined(536): OMV\Rpc\Rpc::call('Config', 'applyChangesBg', Array, Array, 1)#12 {main}[/size][/font]

It's unclear to me if the error is just the Python warning about the deprecated command, or if there's something more fundamental. When I created the USB backup the only options enabled were Synchronize from/to directory on external storage device, Enable archive mode, Recurse into directories, and Transfer modification times along with the files and update them on the remote system.

Just to see if the problems I encountered were related to the USB disk having a factory partition table with two partitions, I went ahead and repartitioned and tried an exfat format. Test mounting the partition worked well enough.

When I tried recreating a single USB Backup I received the following error result:

Error #0:
OMV\ExecException: Failed to execute command 'export PATH=/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/sbin; export LANG=C.UTF-8; omv-salt deploy run usbbackup 2>&1' with exit code '1': /usr/lib/python3/dist-packages/salt/utils/decorators/signature.py:31: DeprecationWarning: `formatargspec` is deprecated since Python 3.5. Use `signature` and the `Signature` object directly
*salt.utils.args.get_function_argspec(original_function)
/usr/lib/python3/dist-packages/salt/utils/decorators/signature.py:31: DeprecationWarning: `formatargspec` is deprecated since Python 3.5. Use `signature` and the `Signature` object directly
*salt.utils.args.get_function_argspec(original_function)
debian:
----------
ID: create_usbbackup_scripts_dir
Function: file.directory
Name: /var/lib/openmediavault/usbbackup.d
Result: True
Comment: Files cleaned from directory /var/lib/openmediavault/usbbackup.d
Started: 07:02:50.037034
Duration: 2.296 ms
Changes:
----------
removed:
- /var/lib/openmediavault/usbbackup.d/systemd-f8dc038d4eb5a08afce891c2b6ac629f
----------
ID: remove_usbbackup_rsync_scripts
Function: module.run
Result: True
Comment: file.find: []
Started: 07:02:50.040127
Duration: 1.044 ms
Changes:
----------
file.find:
----------
ID: remove_usbbackup_systemd_scripts
Function: module.run
Result: True
Comment: file.find: []
Started: 07:02:50.041341
Duration: 1.011 ms
Changes:
----------
file.find:
----------
ID: disable_usbbackup_systemd_unitfile_openmediavault-usbbackup-f8dc038d4eb5a08afce891c2b6ac629f.service
Function: service.disabled
Name: openmediavault-usbbackup-f8dc038d4eb5a08afce891c2b6ac629f.service
Result: True
Comment: Service openmediavault-usbbackup-f8dc038d4eb5a08afce891c2b6ac629f.service has been disabled, and is in the desired state
Started: 07:02:50.624462
Duration: 273.75 ms
Changes:
----------
openmediavault-usbbackup-f8dc038d4eb5a08afce891c2b6ac629f.service:
True
----------
ID: remove_usbbackup_systemd_unitfile_openmediavault-usbbackup-f8dc038d4eb5a08afce891c2b6ac629f.service
Function: file.absent
Name: /lib/systemd/system/openmediavault-usbbackup-f8dc038d4eb5a08afce891c2b6ac629f.service
Result: True
Comment: Removed file /lib/systemd/system/openmediavault-usbbackup-f8dc038d4eb5a08afce891c2b6ac629f.service
Started: 07:02:50.898416
Duration: 0.664 ms
Changes:
----------
removed:
/lib/systemd/system/openmediavault-usbbackup-f8dc038d4eb5a08afce891c2b6ac629f.service
----------
ID: configure_usbbackup_rsync_script_2a6a9c07-dbf9-4fe7-8694-e9a8c82a7d7e
Function: file.managed
Name: /var/lib/openmediavault/usbbackup.d/rsync-2a6a9c07-dbf9-4fe7-8694-e9a8c82a7d7e
Result: False
Comment: Unable to manage file: Jinja syntax error: no filter named 'prettify_dir_path'; line 9


---
[...]
{%- set runfile = '/var/run' | path_join(script_prefix ~ job.uuid) -%}
{%- set id = job.devicefile | md5 -%}
{%- set mount_path = id | make_mount_path -%}
{%- if job.mode == 'push' -%}
{%- set srcuri = salt['omv_conf.get_sharedfolder_path'](job.sharedfolderref) -%}
{%- set desturi = (mount_path ~ salt['omv_conf.get_sharedfolder_name'](job.sharedfolderref) if job.usesubdir else '') | prettify_dir_path -%} <======================
{%- elif job.mode == 'pull' -%}
{%- set srcuri = (mount_path ~ salt['omv_conf.get_sharedfolder_name'](job.sharedfolderref) if job.usesubdir else '') | prettify_dir_path -%}
{%- set desturi = salt['omv_conf.get_sharedfolder_path'](job.sharedfolderref) -%}
{%- endif -%}
#!/bin/sh
[...]
---
Started: 07:02:50.899182
Duration: 63.818 ms
Changes:
----------
ID: configure_usbbackup_systemd_script_a34b92a0e66e3a470d2e646682183f66
Function: file.managed
Name: /var/lib/openmediavault/usbbackup.d/systemd-a34b92a0e66e3a470d2e646682183f66
Result: True
Comment: File /var/lib/openmediavault/usbbackup.d/systemd-a34b92a0e66e3a470d2e646682183f66 updated
Started: 07:02:50.963109
Duration: 49.197 ms
Changes:
----------
diff:
New file
mode:
0750
----------
ID: configure_usbbackup_systemd_unitfile_a34b92a0e66e3a470d2e646682183f66
Function: file.managed
Name: /lib/systemd/system/openmediavault-usbbackup-a34b92a0e66e3a470d2e646682183f66.service
Result: True
Comment: File /lib/systemd/system/openmediavault-usbbackup-a34b92a0e66e3a470d2e646682183f66.service updated
Started: 07:02:51.012410
Duration: 34.638 ms
Changes:
----------
diff:
New file
mode:
0644
----------
ID: usbbackup_systemctl_daemon_reload
Function: module.run
Name: service.systemctl_reload
Result: True
Comment:
Started: 07:02:51.047208
Duration: 0.439 ms
Changes:
----------
ID: enable_usbbackup_systemd_unitfile_a34b92a0e66e3a470d2e646682183f66
Function: service.enabled
Name: openmediavault-usbbackup-a34b92a0e66e3a470d2e646682183f66
Result: True
Comment: Service openmediavault-usbbackup-a34b92a0e66e3a470d2e646682183f66 has been enabled, and is in the desired state
Started: 07:02:51.047749
Duration: 437.074 ms
Changes:
----------
openmediavault-usbbackup-a34b92a0e66e3a470d2e646682183f66:
True

Summary for debian
------------
Succeeded: 9 (changed=8)
Failed: 1
------------
Total states run: 10
Total run time: 863.931 ms in /usr/share/php/openmediavault/system/process.inc:182
Stack trace:
#0 /usr/share/php/openmediavault/engine/module/serviceabstract.inc(60): OMV\System\Process->execute()
#1 /usr/share/openmediavault/engined/rpc/config.inc(164): OMV\Engine\Module\ServiceAbstract->deploy()
#2 [internal function]: Engined\Rpc\Config->applyChanges(Array, Array)
#3 /usr/share/php/openmediavault/rpc/serviceabstract.inc(123): call_user_func_array(Array, Array)
#4 /usr/share/php/openmediavault/rpc/serviceabstract.inc(149): OMV\Rpc\ServiceAbstract->callMethod('applyChanges', Array, Array)
#5 /usr/share/php/openmediavault/rpc/serviceabstract.inc(577): OMV\Rpc\ServiceAbstract->OMV\Rpc\{closure}('/tmp/bgstatusYA...', '/tmp/bgoutputb2...')
#6 /usr/share/php/openmediavault/rpc/serviceabstract.inc(159): OMV\Rpc\ServiceAbstract->execBgProc(Object(Closure))
#7 /usr/share/openmediavault/engined/rpc/config.inc(186): OMV\Rpc\ServiceAbstract->callMethodBg('applyChanges', Array, Array)
#8 [internal function]: Engined\Rpc\Config->applyChangesBg(Array, Array)
#9 /usr/share/php/openmediavault/rpc/serviceabstract.inc(123): call_user_func_array(Array, Array)
#10 /usr/share/php/openmediavault/rpc/rpc.inc(86): OMV\Rpc\ServiceAbstract->callMethod('applyChangesBg', Array, Array)
#11 /usr/sbin/omv-engined(536): OMV\Rpc\Rpc::call('Config', 'applyChangesBg', Array, Array, 1)
#12 {main}

It's unclear to me if the error is just the Python warning about the deprecated command, or if there's something more fundamental. When I created the USB backup the only options enabled were Synchronize from/to directory on external storage device, Enable archive mode, Recurse into directories, and Transfer modification times along with the files and update them on the remote system.

I'm only planning on using exFAT for backup purposes actually, as his post 22 discusses.

Simply want to make sure that my wife can recover the data from-backup if she needs to with no special steps.

Worst case I can see about writing my own cron job to test for the presence of the USB device, mount the device if-present, synchronize the files if the device is mounted, and then unmount, but if the intent is for OMV to be able to create all of these steps for me without requiring digging into scripts then I'd rather use the project's means to accomplish this.

After all, much of what OMV seeks to accomplish could be done through regular Linux administration; I've been building Linux boxes for various purposes and integrating them into mixed environments for more than twenty years. Admittedly I'm looking at this project in-part because I have plenty of other demands on my time and because I'm trying to overcome my concerns regarding systemd through a project that uses features that it brings to the table.