Unable to log into web gui after update

KM0201 · 25. März 2019

Zitat von andyw

That's strange. Please see attached.

I just happened to see this, but just FYI... I would never trust Windows to accurately display information about a Linux filesystem... it will almost always be wrong (as it is in this case).

andyw · 25. März 2019

Thank you for your assistance.

andyw · 25. März 2019

Am I missing something? I have just run sudo du -h --max-depth=1 in the directory /var/lib/docker and can't find the offending file. Please see the output below.

root@lh-omv-nas:/var/lib/docker# sudo du -h --max-depth=1
4.0K ./swarm
4.0K ./runtimes
20K ./builder
3.5G ./overlay2
360K ./containerd
180K ./network
1.4M ./containers
4.0K ./tmp
9.1M ./image
72K ./buildkit
28K ./volumes
4.0K ./openmediavault
4.0K ./trust
20K ./plugins
3.5G .

geaves · 25. März 2019

Do it again but from root, it will give you a better idea

andyw · 25. März 2019

Here is the output from root.

root@lh-omv-nas:/# sudo du -h --max-depth=1
16K ./opt
4.0K ./mnt
4.0K ./export
3.6G ./var
du: cannot read directory './proc/15597/task/15597/net': Invalid argument
du: cannot read directory './proc/15597/net': Invalid argument
du: cannot read directory './proc/15598/task/15598/net': Invalid argument
du: cannot read directory './proc/15598/net': Invalid argument
du: cannot read directory './proc/15599/task/15599/net': Invalid argument
du: cannot read directory './proc/15599/net': Invalid argument
du: cannot read directory './proc/15600/task/15600/net': Invalid argument
du: cannot read directory './proc/15600/net': Invalid argument
du: cannot access './proc/15618/task/15618/fd/3': No such file or directory
du: cannot access './proc/15618/task/15618/fdinfo/3': No such file or directory
du: cannot access './proc/15618/fd/4': No such file or directory
du: cannot access './proc/15618/fdinfo/4': No such file or directory
0 ./proc
173M ./boot
4.0K ./home
452G ./sharedfolders
0 ./dev
0 ./tmp
1.2G ./lib
0 ./sys
16K ./lost+found
9.3M ./run
13M ./bin
8.0K ./media
4.0K ./lib64
68K ./root
950M ./usr
15M ./sbin
245G ./srv
7.3M ./etc
702G .

geaves · 25. März 2019

I'm running out of ideas, Ok du -sh /* it will probably output the same as above

andyw · 25. März 2019

Here is the output of du -sh /*

root@lh-omv-nas:/# du -sh /*
13M /bin
173M /boot
0 /dev
7.3M /etc
4.0K /export
4.0K /home
0 /initrd.img
0 /initrd.img.old
1.2G /lib
4.0K /lib64
16K /lost+found
8.0K /media
4.0K /mnt
16K /opt
du: cannot read directory '/proc/18285/task/18285/net': Invalid argument
du: cannot read directory '/proc/18285/net': Invalid argument
du: cannot read directory '/proc/18286/task/18286/net': Invalid argument
du: cannot read directory '/proc/18286/net': Invalid argument
du: cannot read directory '/proc/18287/task/18287/net': Invalid argument
du: cannot read directory '/proc/18287/net': Invalid argument
du: cannot read directory '/proc/18288/task/18288/net': Invalid argument
du: cannot read directory '/proc/18288/net': Invalid argument
du: cannot access '/proc/18289/task/18289/fd/3': No such file or directory
du: cannot access '/proc/18289/task/18289/fdinfo/3': No such file or directory
du: cannot access '/proc/18289/fd/3': No such file or directory
du: cannot access '/proc/18289/fdinfo/3': No such file or directory
0 /proc
68K /root
9.3M /run
15M /sbin
452G /sharedfolders
60K /srv
0 /sys
0 /tmp
950M /usr
3.6G /var
0 /vmlinuz
0 /vmlinuz.old

geaves · 25. März 2019

I am running out of ideas so I've searched the forum here you could also install ncdu apt-get install ncdu then run from the cli ncdu /

andyw · 25. März 2019

I have just looked at the /proc folder and there appears to be hundreds of entry's with today's date. could this be the issue?

KM0201 · 26. März 2019

What is the output of du -sh /var/*

andyw · 26. März 2019

The output of du -sh /var/*

root@lh-omv-nas:/# du -sh /var/*
1.7M /var/backups
8.6M /var/cache
3.5G /var/lib
4.0K /var/local
0 /var/lock
25M /var/log
4.0K /var/mail
4.0K /var/opt
0 /var/run
120K /var/spool
4.0K /var/tmp
11M /var/www

KM0201 · 26. März 2019

Well, I'm at a loss.... Post #5 clearly indicates the issue is in /var/lib/docker

andyw · 26. März 2019

I am unable to run 'apt-get install ncdu'. The output is below:

root@lh-omv-nas:/# apt-get install ncdu
Reading package lists... Done
Building dependency tree
Reading state information... Done
E: Unable to locate package ncdu

Thanks again all for your help.

geaves · 26. März 2019

Zitat von andyw

I am unable to run 'apt-get install ncdu'. The output is below:

My bad of course it won't because the boot disk is full and as per your first post you can't therefore run apt-clean nor update to update the repositories, I'm also at a loss now, my only suggestion is that link I posted.

KM0201 · 26. März 2019

Zitat von andyw

I am unable to run 'apt-get install ncdu'. The output is below:

root@lh-omv-nas:/# apt-get install ncdu
Reading package lists... Done
Building dependency tree
Reading state information... Done
E: Unable to locate package ncdu

Thanks again all for your help.

Alles anzeigen

That's saying it can't find the package.. pretty sure a disk full error would be different. Not sure what is going on.

Regardless, i don't think ncdu really solves the problem.. it's likely just going to duplicate what the other commands are saying.

KM0201 · 26. März 2019

Hold on a second... going back to your first command...

/var/lib/docker/overlay2/1bb93d795d93790555cade4c27f2b28765673b594ba15e4d5b7786044f450e4a/merged
overlay 110G 108G 0 100%

Are you using mergerFS? I'm not sure how that would be on your OS drive... but I don't have that "merged" in my /overlay2 path for any containers.

geaves · 26. März 2019

Zitat von KM0201

Are you using mergerFS?

Can't be it's not showing the mergerfs mount point in df -h this is the output from mine;

Code

df -h
Filesystem      Size  Used Avail Use% Mounted on
udev            1.9G     0  1.9G   0% /dev
tmpfs           383M   21M  362M   6% /run
/dev/sdg1        28G  5.7G   21G  22% /
tmpfs           1.9G     0  1.9G   0% /dev/shm
tmpfs           5.0M     0  5.0M   0% /run/lock
tmpfs           1.9G     0  1.9G   0% /sys/fs/cgroup
tmpfs           1.9G  4.0K  1.9G   1% /tmp
2:3:4           3.6T  1.2T  2.5T  33% /srv/ce990cca-026e-422d-989b-ec50e1fba465
/dev/sde1       916G  512G  405G  56% /srv/dev-disk-by-label-Backup
/dev/sdc1       916G  124G  793G  14% /srv/dev-disk-by-label-disk3
/dev/sdf1       293G   11G  283G   4% /srv/dev-disk-by-label-Docker
/dev/sdb1       1.8T  842G  992G  46% /srv/dev-disk-by-label-disk2
/dev/sda1       1.8T 1002G  831G  55% /srv/dev-disk-by-label-disk1
/dev/sdd1       916G  213G  704G  24% /srv/dev-disk-by-label-disk4
folder2ram      1.9G   86M  1.8G   5% /var/log
folder2ram      1.9G     0  1.9G   0% /var/tmp
folder2ram      1.9G  2.4M  1.9G   1% /var/lib/openmediavault/rrd
folder2ram      1.9G  1.7M  1.9G   1% /var/spool
folder2ram      1.9G   29M  1.9G   2% /var/lib/rrdcached
folder2ram      1.9G   12K  1.9G   1% /var/lib/monit
folder2ram      1.9G     0  1.9G   0% /var/lib/php
folder2ram      1.9G     0  1.9G   0% /var/lib/netatalk/CNID
folder2ram      1.9G  640K  1.9G   1% /var/cache/samba
overlay          28G  5.7G   21G  22% /var/lib/docker/overlay2/350c6e7b978faaa077b93e6538cf24ad0d08960c1e3e06b043f2e31f0e7c2fb6/merged
shm              64M     0   64M   0% /var/lib/docker/containers/a2032298aa8e3aceb1cc5bfde906a718521db7c880b5308a364a230f3180464e/mounts/shm
tmpfs           383M     0  383M   0% /run/user/0
overlay          28G  5.7G   21G  22% /var/lib/docker/overlay2/1fe399e880615a3d4828e16abd327330a3a4e5db013656315eb8a6e4169a1716/merged
shm              64M     0   64M   0% /var/lib/docker/containers/d3d46b1105f01dd7a1bf6f2b7b1bc90d0a64d50b148d9bab89d76fb244c7f368/mounts/shm
overlay          28G  5.7G   21G  22% /var/lib/docker/overlay2/5fd2aef017370e245b5424d7919dde291982b2fcfe1ed16af3acf8bcd1700bc7/merged
shm              64M     0   64M   0% /var/lib/docker/containers/8d2e9c10c69c599b4334845a3aae1193f07de26ecc9181860dbe63657db3c94f/mounts/shm

Alles anzeigen

KM0201 · 26. März 2019

Any clue what that merged path is then? I don't have that in any of my paths.

Edit: Nevermind.. It's apparently not in all the paths, but I just found it in a couple.

I was off base there... just out of ideas and grasping at straws.

geaves · 26. März 2019

Zitat von KM0201

Any clue what that merged path is then?

Best guess reading this it's the number of image layers a particular image has, therefore they merge, looking at the 3 I use they all have a number of image layers.

If you look at the output from du -h --max-depth=1 and then from du -sh /* they are different, the sharedfolders are the same which matches the first post if you add up the used space on the three drives.

However, du -h --max-depth=1 shows ./srv at 245G but du -sh /* shows /srv at 60K TBH my knowledge of linux directory/file listings is virtually zero and I have to search.

ryecoaaron · 27. März 2019

What is the output of: sudo du -d1 -x -h / | sort -h

Jetzt mitmachen!