OneDrive plugin questions / feature request

@mvzut

Zitat

I just upgraded to OMV6 and found out about the OneDrive plugin. Awesome!

...

There is one thing though... My OneDrive is fairly large (almost 600 GB), and my poor Pi 4 (4 GB RAM) is using quite some resources to check for differences at the configured intervals. I have set it to check once per hour, which is the longest possible interval, and every time it takes 15 minutes even if there are no differences. I can see this in the CPU graph, it's not per se User or System but mostly Wait-IO CPU usage.

What client version are you running? OMV initially integrated v2.4.17 .. which, if you were using the OpenSuSE Build Service repository, you would be using v2.4.19. There are some significant performance differences between these versions.

Regarding the differences check .. once per hour is not the longest possible ... please read:

* 'monitor_interval' - https://github.com/abraunegg/o…USAGE.md#monitor_interval - this is the time between checks of OneDrive for online changes

* 'monitor_fullscan_frequency' - https://github.com/abraunegg/o…onitor_fullscan_frequency - controls how often the integrity check is performed. This is where I/O intensive operations come from and again is 100% dependent on what client version you are running as to how big of an impact this is.

Zitat

I don't make changes to my files very often, would it also be possible to set the synchronization interval to e.g. 6, 12 or even 24 hours? That would also allow my harddrive to spin down much more often

Please re-read the above 2 links. You need to configure your client to operate in alignment with your rate of data change. Potentially this may be ideal for your scenario:

* 'monitor_interval' = 3600 (check OneDrive online for changes 1hr after completion of last check)

* 'monitor_fullscan_frequency' = 12 (default, perform integrity check every 12 iterations of checking OneDrive for changes, approx every 12 hours or twice a day)

Zitat

Some time ago I already tried setting up my own synchronization with abraunegg's onedrive service, but was struggling with automating the process and forgot about it.

Next time please raise a Discussion for questions with the client - https://github.com/abraunegg/onedrive/discussions - it is better to post there for assistance.

@mvzut

With the application 'defaults' / OMV defaults and client version v2.4.20 (which you should be running) - what is your current CPU graph showing you for your performance?

IMHO - if your data online 'hardly' changes, you still should be checking OneDrive online at least every hour for any data change. This is reasonable. Telling then your system to perform an integrity check twice a day is also reasonable. This is the suggestion I made above. A tweak on this would be:

* 'monitor_interval' = 3600 (check OneDrive online for changes 1hr after completion of last check)

* 'monitor_fullscan_frequency' = 24 = perform integrity check every 24 iterations of checking OneDrive for changes

I would be really interested to see the following:

* Your current CPU usage graphs - what is the CPU actually doing. Can you graph the 'onedrive' process by itself? What is it actually doing in comparison?

* Your current experience with these settings given your prior issue statement "I have set it to check once per hour, which is the longest possible interval, and every time it takes 15 minutes even if there are no differences." ... as this will be the integrity check performing its job to validate your data

@votdev

Quick question for you.

In OMV, when the service 'starts' or 'restarts' - it appears in some places that you are forcing a 'resync' on the data. Is my understanding correct?

What a resync will do is delete the local cache known state, and force a full integrity check as well - certainly not an ideal position to be in and this may lead to extended CPU loads / disk usage when starting up or restarting the service. If users are concerned about disk sleeps, system sleep, CPU usage running time - this will all add to that.

Additionally, I also see in one of the files, when you are doing an auth process (as per omv-onedrive-auth) - you are manually removing the refresh token, then also forcing a resync.

To perform a reauth, the recommended approach here would be to simplify your existing approach to just the following:

sudo -u onedrive onedrive --confdir /var/cache/onedrive --reauth

and, as per above, it is not recommended to do a resync post reauth ...

A resync should only be done for the following reasons:

* When the application advises because of a configuration change (skip_dir, skip_file for example are triggers)

* When changing the 'sync_dir' path

* When changing the 'drive_id'

* When updating 'sync_list' to control what is included|excluded in the sync process

* When updating OneDrive Business Shared Folders

If none of these client options have been updated, then a resync at startup IMHO should be avoided and not forced upon the application.

If a resync needs to be done, the application will advise the user to use resync .. perhaps there is a way for this to be 'monitored' by OMV to determine if the flag call was requested, so that the application can be restarted automatically with resync

Zitat von votdev

How can check this flag? I think monitd can do that and trigger a resync if necessary.

The 'exit' value of the application will be 126 if a resync is required

So if you check for exit value of the process, if it equals 126, then you can restart the service with the additional resync flags - this would be cleaner than specifying resync for startup

Zitat von votdev

Can you please point me to the code you are talking about? I do not remember that i'm doing this except in the omv-onedrive-auth command.

The omv-onedrive-auth script has been improved according your suggestion. Thx.

It might have been just the auth script - I dont know enough of your code / default startup options etc ... based on my reading it looked like also on service start, the default option was to resync .. hence my question

Zitat von votdev

I assume i have to run onedrive --display-sync-status to get the required exit code, right?

When i have enabled onedrive --monitor via systemd, how can i check if a resync is necessary? If i run onedrive --display-sync-status i get an error message + segfault.

Bash

# /usr/bin/onedrive --confdir /var/cache/onedrive --display-sync-status
Configuration file successfully loaded
Configuring Global Azure AD Endpoints

ERROR: onedrive application is already running - check system process list for active application instances

Segmentation fault

Code

Aug 18 10:05:36 omv6box kernel: onedrive[35844]: segfault at 0 ip 0000000000000000 sp 00007fffbef901f8 error 14 in onedrive[55cecd80f000+224000]
Aug 18 10:05:36 omv6box kernel: Code: Unable to access opcode bytes at RIP 0xffffffffffffffd6.

Code

# dpkg -l | grep onedrive
ii  onedrive                       2.4.20-1+np1                   amd64        folder synchronization with OneDrive
ii  openmediavault-onedrive        6.2.4-1                        all          openmediavault OneDrive plugin

Do i really have to stop the systemd unit, run the --display-sync-status command, do a --resync if necessary and then restart the systemd unit (--monit)? That sound weird to me.

Alles anzeigen

No - you should not have to do that whole song & dance.

OK ... the segmentation fault on --display-sync-status is an odd one ... it should not be doing that. Please report that as a bug to be fixed. That command also tells you if your data / system is in-sync with OneDrive .. not if the application needs a resync.

Essentially, if you 'change' any configuration:

* When the application advises because of a configuration change (skip_dir, skip_file for example are triggers)

* When changing the 'sync_dir' path

* When changing the 'drive_id'

* When updating 'sync_list' to control what is included|excluded in the sync process

* When updating OneDrive Business Shared Folders

Then try and 'restart' the client to use this updated configuration, the exit code should be 126 if a resycnc is required.

systemd services should be sighup post a config change, and this is where you should see then the service not starting with an exit code of 126