Connecting to github timed out during omv-release-upgrade

foreach · Jun 19th 2024

Hello,

I've been running OMV on a proxmox VM without major issues since a few years ago. Now I tried to upgrade from 6 to 7 and the upgrade process gets stuck at this point. On OMV6 (tested before the upgrade) there are no problems with github at all. It's during the upgrade that something broke, and can't figure out what's causing it. I've already tried to reset iptables removing all docker stuff etc. without any luck. Also uninstalled wireguard to make sure that it's not messing with some network configuration and same result. I've restored a backup and tried to run the upgrade multiple times, and always gets stuck at the same point. The only plugins/extras that I have are, docker, wireguard, snapraid, minidlna and mergerfs

Thank you.

Code

Checking yq version ...
Correct version of yq installed - 'v4.35.2'
Downloading regctl ...
--2024-06-16 22:09:40--  https://github.com/regclient/regclient/releases/download/v0.6.0/regctl-linux-amd64
Resolving github.com (github.com)... 140.82.121.4
Connecting to github.com (github.com)|140.82.121.4|:443... failed: Connection timed out.
Retrying.

--2024-06-16 22:11:50--  (try: 2)  https://github.com/regclient/regclient/releases/download/v0.6.0/regctl-linux-amd64
Connecting to github.com (github.com)|140.82.121.4|:443... failed: Connection timed out.
Retrying.

Display More

ryecoaaron · Jun 19th 2024

Your system is failing to connect to github. There is something on your network likely causing this. Wireguard is incoming connection and shouldn't block outgoing. Did you change any firewall settings in the OMV web interface?

foreach · Jun 20th 2024

Quote from ryecoaaron

Your system is failing to connect to github. There is something on your network likely causing this. Wireguard is incoming connection and shouldn't block outgoing. Did you change any firewall settings in the OMV web interface?

Hello, I don't see any rules on OMV firewall web interface, and I've never had one that I know of. It's weird because on the other LXC containers or VM's, github connection is working without any problems, and in OMV6 prior to do omv-release-upgrade it's working too… any more ideas... hints? Thank you.

ryecoaaron · Jun 20th 2024

Quote from foreach

any more ideas... hints?

Sorry. Anything I would look at is outside of OMV. The plugin is doing a simple wget. Not much can go wrong with it other than network settings.

What is the output of:

wget "https://github.com/regclient/regclient/releases/download/v0.6.0/regctl-linux-amd64"

foreach · Jun 20th 2024

Here you can see both results from OMV6 and OMV7

Executing wget from OMV6:

Code

root@openmediavault:~# wget "https://github.com/regclient/regclient/releases/download/v0.6.0/regctl-linux-amd64"
--2024-06-20 13:59:15--  https://github.com/regclient/regclient/releases/download/v0.6.0/regctl-linux-amd64
Resolving github.com (github.com)... 140.82.121.3
Connecting to github.com (github.com)[140.82.121.3]:443... connected.
HTTP request sent, awaiting response... 302 Found
Location: https://objects.githubusercontent.com/github-production-release-asset-2e65be/294181202/42376875-f4ac-4487-9513-a4c29c613342?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=releaseassetproduction%2F20240620%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20240620T115915Z&X-Amz-Expires=300&X-Amz-Signature=b4e7a76b44cef22ed6a499ae32d4f5ed7fbf842e87274bdc73b7d8e4f4e6ef71&X-Amz-SignedHeaders=host&actor_id=0&key_id=0&repo_id=294181202&response-content-disposition=attachment%3B%20filename%3Dregctl-linux-amd64&response-content-type=application%2Foctet-stream [following]
--2024-06-20 13:59:15--  https://objects.githubusercontent.com/github-production-release-asset-2e65be/294181202/42376875-f4ac-4487-9513-a4c29c613342?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=releaseassetproduction%2F20240620%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20240620T115915Z&X-Amz-Expires=300&X-Amz-Signature=b4e7a76b44cef22ed6a499ae32d4f5ed7fbf842e87274bdc73b7d8e4f4e6ef71&X-Amz-SignedHeaders=host&actor_id=0&key_id=0&repo_id=294181202&response-content-disposition=attachment%3B%20filename%3Dregctl-linux-amd64&response-content-type=application%2Foctet-stream
Resolving objects.githubusercontent.com (objects.githubusercontent.com)... 185.199.109.133, 185.199.110.133, 185.199.111.133, ...
Connecting to objects.githubusercontent.com (objects.githubusercontent.com)[185.199.109.133]:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 9740423 (9,3M) [application/octet-stream]
Saving to: «regctl-linux-amd64»

regctl-linux-amd64 100%[================>]   9,29M  41,3MB/s    en 0,2s    

2024-06-20 13:59:16 (41,3 MB/s) - «regctl-linux-amd64» saved [9740423/9740423]

root@openmediavault:~#

Display More

Executing wget from the same VM after doing omv-release-upgrade:

Code

root@openmediavault:~# wget "https://github.com/regclient/regclient/releases/download/v0.6.0/regctl-linux-amd64"
--2024-06-20 14:51:00--  https://github.com/regclient/regclient/releases/download/v0.6.0/regctl-linux-amd64
Resolving github.com (github.com)... 140.82.121.3
Connecting to github.com (github.com)[140.82.121.3]:443... failed: Connection timed out.
Retrying.

--2024-06-20 14:53:11--  (try: 2)  https://github.com/regclient/regclient/releases/download/v0.6.0/regctl-linux-amd64
Connecting to github.com (github.com)[140.82.121.3]:443...

ryecoaaron · Jun 20th 2024

Quote from foreach

Executing wget from the same VM after doing omv-release-upgrade:

I understand your OMV6 system works and your OMV7 system does not but I can't see your system. So, I don't know why the network settings on your omv 7 system aren't allowing a simple wget to work. OMV doesn't configure wget ever and it works on all of my OMV7 systems. You will need to post more info on your network settings.

cat /etc/netplan/*

cat /etc/resolv.conf

sudo omv-salt deploy run systemd-networkd

foreach · Jun 20th 2024

Code

root@openmediavault:~# cat /etc/netplan/*
network:
  version: 2
  renderer: networkd
network:
  ethernets:
    ens18:
      match:
        macaddress: **:**:**:**:**:76
      addresses:
      - 192.168.1.100/24
      routes:
      - to: 0.0.0.0/0
        via: 192.168.1.1
      dhcp4: no
      dhcp6: no
      link-local: []
      nameservers:
        addresses:
        - 192.168.1.1
      mtu: 6

Display More

Code

root@openmediavault:~# cat /etc/resolv.conf
# This file is managed by man:systemd-resolved(8). Do not edit.
#
# This is a dynamic resolv.conf file for connecting local clients to the
# internal DNS stub resolver of systemd-resolved. This file lists all
# configured search domains.
#
# Run "resolvectl status" to see details about the uplink DNS servers
# currently in use.
#
# Third party programs should typically not access this file directly, but only
# through the symlink at /etc/resolv.conf. To manage man:resolv.conf(5) in a
# different way, replace this symlink by a static file or a different symlink.
#
# See man:systemd-resolved.service(8) for details about the supported modes of
# operation for /etc/resolv.conf.

nameserver 127.0.0.53
options edns0 trust-ad
search .

Display More

Code

root@openmediavault:~# omv-salt deploy run systemd-networkd 
debian:
----------
          ID: unmask_systemd_networkd
    Function: service.unmasked
        Name: systemd-networkd
      Result: True
     Comment: Service systemd-networkd was already unmasked
     Started: 15:23:38.387495
    Duration: 19.78 ms
     Changes:   
----------
          ID: enable_systemd_networkd
    Function: service.enabled
        Name: systemd-networkd
      Result: True
     Comment: Service systemd-networkd is already enabled, and is in the desired state
     Started: 15:23:38.407539
    Duration: 22.535 ms
     Changes:   
----------
          ID: remove_systemd_networkd_defaults
    Function: file.absent
        Name: /etc/systemd/network/05-openmediavault-default.link
      Result: True
     Comment: File /etc/systemd/network/05-openmediavault-default.link is not present
     Started: 15:23:38.433982
    Duration: 0.987 ms
     Changes:   
----------
          ID: unmask_systemd_resolved
    Function: service.unmasked
        Name: systemd-resolved
      Result: True
     Comment: Service systemd-resolved was already unmasked
     Started: 15:23:38.435175
    Duration: 18.47 ms
     Changes:   
----------
          ID: enable_systemd_resolved
    Function: service.enabled
        Name: systemd-resolved
      Result: True
     Comment: Service systemd-resolved is already enabled, and is in the desired state
     Started: 15:23:38.453896
    Duration: 15.198 ms
     Changes:   
----------
          ID: configure_etc_network_interfaces
    Function: file.managed
        Name: /etc/network/interfaces
      Result: True
     Comment: File /etc/network/interfaces is in the correct state
     Started: 15:23:38.469361
    Duration: 91.707 ms
     Changes:   
----------
          ID: remove_netplan_systemd_networkd_config_files
    Function: module.run
      Result: True
     Comment: file.find: []
     Started: 15:23:38.562332
    Duration: 2.147 ms
     Changes:   
              ----------
              file.find:
----------
          ID: remove_netplan_empty_systemd_networkd_config_files
    Function: module.run
      Result: True
     Comment: file.find: []
     Started: 15:23:38.564719
    Duration: 1.54 ms
     Changes:   
              ----------
              file.find:
----------
          ID: remove_netplan_config_files
    Function: module.run
      Result: True
     Comment: file.find: ['/etc/netplan/10-openmediavault-default.yaml', '/etc/netplan/20-openmediavault-ens18.yaml']
     Started: 15:23:38.566426
    Duration: 1.937 ms
     Changes:   
              ----------
              file.find:
                  - /etc/netplan/10-openmediavault-default.yaml
                  - /etc/netplan/20-openmediavault-ens18.yaml
----------
          ID: configure_netplan_default
    Function: file.managed
        Name: /etc/netplan/10-openmediavault-default.yaml
      Result: True
     Comment: File /etc/netplan/10-openmediavault-default.yaml updated
     Started: 15:23:38.568525
    Duration: 193.797 ms
     Changes:   
              ----------
              diff:
                  New file
----------
          ID: configure_netplan_ethernet_ens18
    Function: file.managed
        Name: /etc/netplan/20-openmediavault-ens18.yaml
      Result: True
     Comment: File /etc/netplan/20-openmediavault-ens18.yaml updated
     Started: 15:23:38.762543
    Duration: 232.302 ms
     Changes:   
              ----------
              diff:
                  New file
----------
          ID: apply_netplan_config
    Function: cmd.run
        Name: netplan apply
      Result: True
     Comment: Command "netplan apply" run
     Started: 15:23:38.996400
    Duration: 1683.66 ms
     Changes:   
              ----------
              pid:
                  125907
              retcode:
                  0
              stderr:
                  Cannot call openvswitch: ovsdb-server.service is not running.
              stdout:
----------
          ID: symlink_systemd_resolvconf
    Function: file.symlink
        Name: /etc/resolv.conf
      Result: True
     Comment: Symlink /etc/resolv.conf is present and owned by root:root
     Started: 15:23:40.680325
    Duration: 1674.736 ms
     Changes:   

Summary for debian
-------------
Succeeded: 13 (changed=6)
Failed:     0
-------------
Total states run:     13
Total run time:    3.959 s

Display More

foreach · Jun 20th 2024

This is the last part of the upgrade process:

Code

--2024-06-20 15:29:52--  (try:18)  https://github.com/regclient/regclient/releases/download/v0.6.0/regctl-linux-amd64
Connecting to github.com (github.com)|140.82.121.3|:443... failed: Connection timed out.
Retrying.

--2024-06-20 15:32:11--  (try:19)  https://github.com/regclient/regclient/releases/download/v0.6.0/regctl-linux-amd64
Connecting to github.com (github.com)|140.82.121.3|:443... failed: Connection timed out.
Retrying.

--2024-06-20 15:34:30--  (try:20)  https://github.com/regclient/regclient/releases/download/v0.6.0/regctl-linux-amd64
Connecting to github.com (github.com)|140.82.121.3|:443... failed: Connection timed out.
Giving up.

ERROR: Failed to create the default configuration: Command '['/usr/share/openmediavault/confdb/create.d/conf.service.compose.sh']' returned non-zero exit status 4.
dpkg: error processing package openmediavault-compose (--configure):
 installed openmediavault-compose package post-installation script subprocess returned error exit status 1
Processing triggers for ca-certificates (20230311) ...
Updating certificates in /etc/ssl/certs...
0 added, 0 removed; done.
Running hooks in /etc/ca-certificates/update.d...
done.
Processing triggers for initramfs-tools (0.142) ...
update-initramfs: Generating /boot/initrd.img-6.1.0-0.deb11.21-amd64
Processing triggers for dbus (1.14.10-1~deb12u1) ...
Processing triggers for php8.2-cli (8.2.7-1~deb12u1) ...
Processing triggers for php8.2-phpdbg (8.2.7-1~deb12u1) ...
Processing triggers for php8.2-fpm (8.2.7-1~deb12u1) ...
Processing triggers for php8.2-cgi (8.2.7-1~deb12u1) ...
Processing triggers for rsyslog (8.2302.0-1) ...
Processing triggers for openmediavault (7.2.1-1) ...
Updating workbench configuration files ...
Restarting engine daemon ...
Errors were encountered while processing:
 openmediavault-compose
E: Sub-process /usr/bin/dpkg returned an error code (1)

Display More

ryecoaaron · Jun 20th 2024

I don't see anything that points to the VM's network causing the problem. Can you wget from other site?

foreach · Jun 20th 2024

Yes. I can.

Code

root@openmediavault:~# wget google.com
--2024-06-20 17:04:11--  http://google.com/
Resolving google.com (google.com)... 142.250.200.78, 2a00:1450:4003:80e::200e
Connecting to google.com (google.com)[142.250.200.78]:80... connected.
HTTP request sent, awaiting response... 301 Moved Permanently
Location: http://www.google.com/ [following]
--2024-06-20 17:04:11--  http://www.google.com/
Resolving  www.google.com (www.google.com)... 216.58.215.164, 2a00:1450:4003:80f::2004
Connecting to google.com (www.google.com)[216.58.215.164]:80... connected.
HTTP request sent, awaiting response... 200 OK

Maybe is something related with known_hosts?

ryecoaaron · Jun 20th 2024

Quote from foreach

Maybe is something related with known_hosts?

What do you mean? The known_hosts file used by ssh? Or something else? Does wget https://www.github.com work?

foreach · Jun 20th 2024

Quote from ryecoaaron

The known_hosts file used by ssh?

Yes.

Quote from ryecoaaron

Does wget https://www.github.com work?

It doesn't work

ryecoaaron · Jun 20th 2024

Quote from foreach

Yes.

That is only used for ssh. wget doesn't use ssh. If you were setting .netrc settings, it might affect wget.

foreach · Jun 21st 2024

Hello, finally managed to upgrade successfully. The problem was MTU value on 'ens18' interface netplan. I don't know why, it had a value of 6, and it should be 1500 in my case... Changing it to 1500 now wget github.com works... just intrigued why worked till today with this value on OMV6 but on OMV7 don't. Anyway… Thank you!

foreach · Jan 31st 2025

On the latest upgrade I did, once again the mtu value changed to 6, anybody knows why this is happening?

ryecoaaron · Feb 1st 2025

Is the mtu set to 6 in the web interface? I can't find anything in the code that could change the value to anything other than 0.

foreach · Feb 1st 2025

Quote from ryecoaaron

Is the mtu set to 6 in the web interface? I can't find anything in the code that could change the value to anything other than 0.

I've taken a look and yes, MTU is defined to 6 there..

I should set here this value to 1500 instead setting it in the netplan file? I don't know why this field has a value.. I think I never typed any to be honest.

Edit: I set it to 0 and applied changes, mtu disappeared from netplan file and now seems it's working good too. Maybe best option is to let it be the default value.

ryecoaaron · Feb 1st 2025

Quote from foreach

I should set here this value to 1500 instead setting it in the netplan file?

Yes. OMV will just overwrite any change you put in the netplan file.

Quote from foreach

Edit: I set it to 0 and applied changes, mtu disappeared from netplan file and now seems it's working good too. Maybe best option is to let it be the default value.

good to hear.

foreach · Feb 1st 2025

Thank you!

Connecting to github timed out during omv-release-upgrade

foreach Jun 19th 2024

foreach Jun 21st 2024

Participate now!