omv5 (armbian) freezes when SABnzbd (docker) is unraring downloads

moddy · 29. Oktober 2020

Hi.

After i suceed in installing SABnzbd, omv5 now seems to have the same issue i experienced with dietpi before.

When SABnzbd is unraring the downloaded files, the server freezes.
Kodi is not able to playback files from the server when unraring starts.
Kodi does not even find the shared nfs-folder from the server.

omv5 does not crash. i mean i can login to the web-ui.

In system information i can see, that the usages of cpu (4-5%) and ram (16-17% from 3,55GB). Looks ok.

Unfortunately the performance statistics graphics are empty.
Disks or File systems show only "Loading".

syslogs look like this:

Code

Oct 29 14:31:29 localhost monit[2467]: '\odroidn2' loadavg (5min) of 15.4 matches resource limit [loadavg (5min) > 6.0]
Oct 29 14:31:29 localhost monit[2467]: '\odroidn2' loadavg (1min) of 18.2 matches resource limit [loadavg (1min) > 12.0]
Oct 29 14:31:32 localhost systemd[1]: Started Run anacron jobs.
Oct 29 14:31:32 localhost anacron[9071]: Anacron 2.3 started on 2020-10-29
Oct 29 14:31:32 localhost anacron[9071]: Normal exit (0 jobs run)
Oct 29 14:31:32 localhost systemd[1]: anacron.service: Succeeded.
Oct 29 14:31:39 localhost systemd[1]: serial-getty@ttyS0.service: Succeeded.
Oct 29 14:31:39 localhost systemd[1]: serial-getty@ttyS0.service: Service RestartSec=100ms expired, scheduling restart.
Oct 29 14:31:39 localhost systemd[1]: serial-getty@ttyS0.service: Scheduled restart job, restart counter is at 389.
Oct 29 14:31:39 localhost systemd[1]: Stopped Serial Getty on ttyS0.
Oct 29 14:31:39 localhost systemd[1]: Started Serial Getty on ttyS0.
Oct 29 14:31:49 localhost systemd[1]: serial-getty@ttyS0.service: Succeeded.
Oct 29 14:31:49 localhost systemd[1]: serial-getty@ttyS0.service: Service RestartSec=100ms expired, scheduling restart.
Oct 29 14:31:49 localhost systemd[1]: serial-getty@ttyS0.service: Scheduled restart job, restart counter is at 390.
Oct 29 14:31:49 localhost systemd[1]: Stopped Serial Getty on ttyS0.
Oct 29 14:31:49 localhost systemd[1]: Started Serial Getty on ttyS0.
Oct 29 14:31:59 localhost monit[2467]: '\odroidn2' loadavg (5min) of 16.1 matches resource limit [loadavg (5min) > 6.0]
Oct 29 14:31:59 localhost monit[2467]: '\odroidn2' loadavg (1min) of 20.0 matches resource limit [loadavg (1min) > 12.0]
Oct 29 14:31:59 localhost systemd[1]: serial-getty@ttyS0.service: Succeeded.
Oct 29 14:31:59 localhost systemd[1]: serial-getty@ttyS0.service: Service RestartSec=100ms expired, scheduling restart.
Oct 29 14:31:59 localhost systemd[1]: serial-getty@ttyS0.service: Scheduled restart job, restart counter is at 391.
Oct 29 14:31:59 localhost systemd[1]: Stopped Serial Getty on ttyS0.
Oct 29 14:31:59 localhost systemd[1]: Started Serial Getty on ttyS0.
Oct 29 14:32:09 localhost systemd[1]: serial-getty@ttyS0.service: Succeeded.
Oct 29 14:32:10 localhost systemd[1]: serial-getty@ttyS0.service: Service RestartSec=100ms expired, scheduling restart.
Oct 29 14:32:10 localhost systemd[1]: serial-getty@ttyS0.service: Scheduled restart job, restart counter is at 392.
Oct 29 14:32:10 localhost systemd[1]: Stopped Serial Getty on ttyS0.
Oct 29 14:32:10 localhost systemd[1]: Started Serial Getty on ttyS0.
Oct 29 14:32:20 localhost systemd[1]: serial-getty@ttyS0.service: Succeeded.
Oct 29 14:32:20 localhost systemd[1]: serial-getty@ttyS0.service: Service RestartSec=100ms expired, scheduling restart.
Oct 29 14:32:20 localhost systemd[1]: serial-getty@ttyS0.service: Scheduled restart job, restart counter is at 393.
Oct 29 14:32:20 localhost systemd[1]: Stopped Serial Getty on ttyS0.
Oct 29 14:32:20 localhost systemd[1]: Started Serial Getty on ttyS0.
Oct 29 14:32:29 localhost monit[2467]: '\odroidn2' loadavg (5min) of 16.8 matches resource limit [loadavg (5min) > 6.0]
Oct 29 14:32:29 localhost monit[2467]: '\odroidn2' loadavg (1min) of 21.1 matches resource limit [loadavg (1min) > 12.0]
Oct 29 14:32:30 localhost systemd[1]: serial-getty@ttyS0.service: Succeeded.
Oct 29 14:32:30 localhost systemd[1]: serial-getty@ttyS0.service: Service RestartSec=100ms expired, scheduling restart.
Oct 29 14:32:30 localhost systemd[1]: serial-getty@ttyS0.service: Scheduled restart job, restart counter is at 394.
Oct 29 14:32:30 localhost systemd[1]: Stopped Serial Getty on ttyS0.
Oct 29 14:32:30 localhost systemd[1]: Started Serial Getty on ttyS0.

Alles anzeigen

Any ideas?

ryecoaaron · 29. Oktober 2020

Zitat von moddy

Any ideas?

If your load is that high, it is probably doing a multi-threaded unrar and the storage it is doing that on is either slow or very busy. Faster storage or a faster server is about the only that will fix that.

moddy · 29. Oktober 2020

Zitat von ryecoaaron

If your load is that high, it is probably doing a multi-threaded unrar and the storage it is doing that on is either slow or very busy. Faster storage or a faster server is about the only that will fix that.

i do not agree.

i use an odroid n2 that is better than the odroid i had before and with that one before - i forget the name - i never had those issues.

Same software, same external usb-hdd.

Why do you mean "load is high"?
The numbers that i posted are not that high?

ryecoaaron · 29. Oktober 2020

Zitat von moddy

i do not agree.

Well, I gave you an educated guess based the information you gave me and 22 years as a Linux admin. A load of 18.2 on a system that has 6 cores (really only 4 usable max - I have an N2), means there are either lots of processes asking for cpu time and/or the I/O wait on the storage.

Zitat von moddy

Why do you mean "load is high"?

Your 1 min load average is 18. Unless you have 18 cpus and fast storage, your load is high.

Zitat von moddy

The numbers that i posted are not that high?

Syslog tells me otherwise. cpu percentage and ram use are only part of load on Linux.

moddy · 29. Oktober 2020

Sorry, sounded more disrespectful than i meant it.

Ok, i did not see the load of 18.2 in the syslog?

Good that i accidentally copied this part...

But if you have an odroid n2, don't you think it should handle a little bit unraring?
In htop there were only one core active with around 16%.

Kodi does not find the nfs-shared-folder anymore.
I rebooted but there is still possibility to access the nfs-shares.

ryecoaaron · 29. Oktober 2020

Zitat von moddy

But if you have an odroid n2, don't you think it should handle a little bit unraring?

Yep. Are you sure it is unraring to and from the usb hard drive? If you open another session while it is unraring, what is the value of wa in the line of top that looks like:

%Cpu(s): 1.7 us, 0.7 sy, 0.0 ni, 97.5 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st

moddy · 31. Oktober 2020

Zitat von ryecoaaron

Yep. Are you sure it is unraring to and from the usb hard drive? If you open another session while it is unraring, what is the value of wa in the line of top that looks like:

%Cpu(s): 1.7 us, 0.7 sy, 0.0 ni, 97.5 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st

First of all: THANK YOU FOR YOUR SUPPORT1

I skipped to nzbget to see if the new SABnzbd is maybe the problem but had the same issues.

The wa is around 32 while unraring.

ryecoaaron · 31. Oktober 2020

Zitat von moddy

The wa is around 32 while unraring.

That is very high. Your storage is not keeping up.

moddy · 31. Oktober 2020

Ok. Can you tell me more about this?
"wa" means "waiting for I/O" so it is an issue with the external usb-disk?

EDIT:
I mentioned that the wa=32 does not change much, also when the unraring has ended.

Another issue i have is, that the reboot does not work.
I have to draw the powercable.
Now - after you mentioned the I/O - i draw the USB-cable from the external hdd and the reboot happened!

So it seems to be the external usb-hdd.

moddy · 1. November 2020

I watched the unraring process again:

At the beginning unraring works with only 2-3 wa, but then the unrar-process in iotop diappears and the wa goes up to 31-33.

Seems to crash or freeze here.
The wa stays there and does not change anymore.

What can i do?

ryecoaaron · 1. November 2020

Zitat von moddy

"wa" means "waiting for I/O" so it is an issue with the external usb-disk?

It might be. Do you see any drive errors in dmesg?

Zitat von moddy

What can i do?

Try a different drive?

Jetzt mitmachen!