Some context that might or might not be helpful:
I've been setting up a docker container with a wireguard client which I wanted to send some containers' internet traffic through, my apache web server container amongst others. To set up the IP routing rules, I first had to install iproute2 on the container - not via Dockerfile yet, but on the running container, for testing purposes. During the installation process though there was some strange, unexpected occurrence where the terminal completely froze and all other services (DNS, nginx,...) - mostly running on containers - failed to respond for several minutes. Only after killing the PuTTY terminal the situation normalized - not sure though if this is just timing coincidence though. I have no idea what actually caused this stuckup, I had installed iproute2 on containers before without any issue. IIRC the stuckup happened while the libpam-cap package was being set up.
Now after everything seemed to be back up running stably, I tried to spin up that wireguard client container that I already had spun up several times with its configuration unchanged. While starting the container, the exact same occurence happened again: Terminal freezing, services failing to respond... this time waiting for some time or killing the terminal didn't make a difference. Connecting to the OMV web GUI was loading very slow, and any login attempts did fail with a timeout. As I had no way to communicate with my NAS now - forgetting I could just have tried to hook up screen and keyboard to the device itself - I plugged out the network cable, hoping to trigger some timeout which might bring the system back up again. Instead after a few seconds, the NAS signalled through beeps that it ungracefully rebooted itself.
The current problem:
Now the result of all of this hassle for some reason is that connecting via SSH on port 22 won't work anymore. After some research I found that I still could connect if I changed the port to some different one on the web GUI, but not without triggering some error first which goes like "Failed to execute command 'export... - I'm assuming some logging process is failing here, the GUI keeps informing me about pending configuration changes since. I think the relevant part of this log correctly formatted looks like this though:
----------
ID: configure_sshd_config
Function: file.managed
Name: /etc/ssh/sshd_config
Result: True
Comment: File /etc/ssh/sshd_config updated
Started: 06:18:04.375249
Duration: 120.635 ms
Changes:
----------
diff:
---
+++
@@ -23,7 +23,7 @@
UsePAM yes
AllowGroups root ssh
AddressFamily any
-Port 22
+Port 26
PermitRootLogin yes
AllowTcpForwarding no
Compression no
----------
ID: divert_sshd_config
Function: omv_dpkg.divert_add
Name: /etc/ssh/sshd_config
Result: True
Comment: Leaving 'local diversion of /etc/ssh/sshd_config to /etc/ssh/sshd_config.distrib'
Started: 06:18:04.497091
Duration: 22.644 ms
Changes:
----------
ID: test_sshd_config
Function: cmd.run
Name: sshd -t
Result: False
Comment: Command \"sshd -t\" run
Started: 06:18:04.522577
Duration: 34.569 ms
Changes:
----------
pid:
78972
retcode:
255
stderr:
Missing privilege separation directory: /run/sshd
stdout:
----------
ID: start_ssh_service
Function: service.running
Name: ssh
Result: True
Comment: Service ssh is already enabled, and is running
Started: 06:18:04.604062
Duration: 238.338 ms
Changes:
----------
ssh:
True
Summary for debian
------------
Succeeded: 7 (changed=7)
Failed: 1
------------So running sshd -t throws the error Missing privilege separation directory: /run/sshd, but the SSH service start just fine.
On the other hand, if I switch the SSH port back to 22, another error comes up, this time like this:
----------
ID: configure_sshd_config
Function: file.managed
Name: /etc/ssh/sshd_config
Result: True
Comment: File /etc/ssh/sshd_config updated
Started: 06:09:04.432631
Duration: 122.287 ms
Changes:
----------
diff:
---
+++
@@ -23,7 +23,7 @@
UsePAM yes
AllowGroups root ssh
AddressFamily any
-Port 26
+Port 22
PermitRootLogin yes
AllowTcpForwarding no
Compression no
----------
ID: divert_sshd_config
Function: omv_dpkg.divert_add
Name: /etc/ssh/sshd_config
Result: True
Comment: Leaving 'local diversion of /etc/ssh/sshd_config to /etc/ssh/sshd_config.distrib'
Started: 06:09:04.556140
Duration: 23.328 ms
Changes:
----------
ID: test_sshd_config
Function: cmd.run
Name: sshd -t
Result: True
Comment: Command \"sshd -t\" run
Started: 06:09:04.582858
Duration: 36.385 ms
Changes:
----------
pid:
74176
retcode:
0
stderr:
stdout:
----------
ID: start_ssh_service
Function: service.running
Name: ssh
Result: False
Comment: Job for ssh.service failed because the control process exited with error code.
See \"systemctl status ssh.service\" and \"journalctl -xe\" for details.
Started: 06:09:04.729608
Duration: 3059.629 ms
Changes:
Summary for debian
------------
Succeeded: 7 (changed=6)
Failed: 1
------------This time, sshd -t is silent, but starting the SSH service fails.
Now that I finally got the idea to hook up to the NAS directly to check on that condition when I cannot connect via SSH, sshd -t and systemctl status ssh.service provide me this:
... which is not saying a lot. journalctl -xe's output is very verbose, but doesn't seem to have any hint either - grepping for ssh gives me nothing. Trying to start SSH gives me the same error as in the log above.
Now while I could deal with SSH no longer running through port 22, this whole situation seems rather spooky to me and I fear that I might run into further issues down the road, given the premise that lead to this issue. That's why I'd like to evaluate what could have caused all of this havoc and what I can do to fix and prevent issues like this in the future. So... does anyone have a clue what is up with all that? Are there any logs I should check for further information? (Still kind of a novice to OMV and Linux in general, so please bare with me.)
NAS model: Terramaster F2-221 (Intel Celeron J3355)
OMV version: 6.3.0-2 (Shaitan) (the latest one to date)
I set up this NAS about 6 weeks ago, so the system is still kind of fresh.
Any help is greatly appreciated!
