Beiträge von Dejh0n

Dejh0n · 4. August 2021

And further:

Code

root@Optimus:~# du -sh /var/lib/docker/containers/6e201ea2cd22203fa59e6454039bec83f5d965584239c46a880ad5c2fd161a6b/*
6.5G    /var/lib/docker/containers/6e201ea2cd22203fa59e6454039bec83f5d965584239c46a880ad5c2fd161a6b/6e201ea2cd22203fa59e6454039bec83f5d965584239c46a880ad5c2fd161a6b-json.log
4.0K    /var/lib/docker/containers/6e201ea2cd22203fa59e6454039bec83f5d965584239c46a880ad5c2fd161a6b/checkpoints
8.0K    /var/lib/docker/containers/6e201ea2cd22203fa59e6454039bec83f5d965584239c46a880ad5c2fd161a6b/config.v2.json
4.0K    /var/lib/docker/containers/6e201ea2cd22203fa59e6454039bec83f5d965584239c46a880ad5c2fd161a6b/hostconfig.json
4.0K    /var/lib/docker/containers/6e201ea2cd22203fa59e6454039bec83f5d965584239c46a880ad5c2fd161a6b/hostname
4.0K    /var/lib/docker/containers/6e201ea2cd22203fa59e6454039bec83f5d965584239c46a880ad5c2fd161a6b/hosts
4.0K    /var/lib/docker/containers/6e201ea2cd22203fa59e6454039bec83f5d965584239c46a880ad5c2fd161a6b/mounts
4.0K    /var/lib/docker/containers/6e201ea2cd22203fa59e6454039bec83f5d965584239c46a880ad5c2fd161a6b/resolv.conf
4.0K    /var/lib/docker/containers/6e201ea2cd22203fa59e6454039bec83f5d965584239c46a880ad5c2fd161a6b/resolv.conf.hash

Dejh0n · 4. August 2021

Code

root@Optimus:~# du -sh /var/lib/docker/*
20K     /var/lib/docker/builder
104K    /var/lib/docker/buildkit
7.7G    /var/lib/docker/containers
12M     /var/lib/docker/image
136K    /var/lib/docker/network
3.6G    /var/lib/docker/overlay2
24K     /var/lib/docker/plugins
4.0K    /var/lib/docker/runtimes
4.0K    /var/lib/docker/swarm
4.0K    /var/lib/docker/trust
364K    /var/lib/docker/volumes

Alles anzeigen

Dejh0n · 4. August 2021

Because I am unsure / not confident in what I’m doing!

But thanks for the tip, followed that and I got:

Code

root@Optimus:~# du -sh /var/lib/docker/containers/*
7.6M    /var/lib/docker/containers/20232ba35a530f39438d8ab8e653918219b99e117bc51c21a97ec240e065796a
921M    /var/lib/docker/containers/2ada272f5f7399cdfe59e0b58a325bfec7753f070ea9fbe8f15236784c7ab758
104K    /var/lib/docker/containers/3485627b6b8e822175e060b6bb81c037fff7860e6f6a47cd91a53af03b04818e
22M     /var/lib/docker/containers/3a03cec0504e4247b280282aa1a316f0c0664e782b11442d51438b916e388f72
93M     /var/lib/docker/containers/598f89fcde32c1a4d942001ee6b2572ba0d34a929117e474a406640c7a1cc962
6.5G    /var/lib/docker/containers/6e201ea2cd22203fa59e6454039bec83f5d965584239c46a880ad5c2fd161a6b
104M    /var/lib/docker/containers/99882eb5a88120004edec44aaabf84ca87b1d10c05cc5710d48217b9ceddff33
2.4M    /var/lib/docker/containers/9ddc925bae6493475364a194c863b96ee829b9c377e2c00b577e03f0250daf7e
44K     /var/lib/docker/containers/bb045d66ca3beb022919d5eae5969afea2ade81737d372078396a7361a92f39d
72K     /var/lib/docker/containers/d6b4f4021bfd8d2365a5f6ca29c8bdb26e8bfa0be187fd614c3df49754e5ce00
32M     /var/lib/docker/containers/e9c7e68e38e7e14e30ffcb5858b3e9d55b340032e8ffcb78f1da7bbcdf6daf62
4.3M    /var/lib/docker/containers/fdc6cbc427836a36a0faeeb8d17259ef9f10d5f3adad0ca0f2d97753abf4c9f8

Alles anzeigen

Dejh0n · 4. August 2021

tried the following:

Code

root@Optimus:/# df -h /
Filesystem      Size  Used Avail Use% Mounted on
/dev/sdj2        19G   19G     0 100% /

Code

oot@Optimus:/# sudo du -h --max-depth=1
4.0K    ./archive
7.0M    ./etc
16K    ./opt
137M    ./root
16K    ./lost+found
5.3G    ./usr
4.0K    ./home
4.0K    ./mnt
0    ./dev
12G    ./var
1015M    ./boot
0    ./sys
4.0K    ./export
702M    ./run
110T    ./srv
0    ./tmp
du: cannot access './proc/3553/task/3553/fd/3': No such file or directory
du: cannot access './proc/3553/task/3553/fdinfo/3': No such file or directory
du: cannot access './proc/3553/fd/4': No such file or directory
du: cannot access './proc/3553/fdinfo/4': No such file or directory
du: cannot read directory './proc/3746/task/3746/net': Invalid argument
du: cannot read directory './proc/3746/net': Invalid argument
du: cannot read directory './proc/3747/task/3747/net': Invalid argument
du: cannot read directory './proc/3747/net': Invalid argument
du: cannot read directory './proc/3748/task/3748/net': Invalid argument
du: cannot read directory './proc/3748/net': Invalid argument
du: cannot read directory './proc/3749/task/3749/net': Invalid argument
du: cannot read directory './proc/3749/net': Invalid argument
du: cannot read directory './proc/3750/task/3750/net': Invalid argument
du: cannot read directory './proc/3750/net': Invalid argument
du: cannot read directory './proc/3751/task/3751/net': Invalid argument
du: cannot read directory './proc/3751/net': Invalid argument
du: cannot read directory './proc/3752/task/3752/net': Invalid argument
du: cannot read directory './proc/3752/net': Invalid argument
du: cannot read directory './proc/3753/task/3753/net': Invalid argument
du: cannot read directory './proc/3753/net': Invalid argument
du: cannot read directory './proc/3754/task/3754/net': Invalid argument
du: cannot read directory './proc/3754/net': Invalid argument
0    ./proc
8.0K    ./media
110T    .

Alles anzeigen

Code

root@Optimus:/# du -sh /*
4.0K    /archive
0    /bin
1015M    /boot
0    /dev
7.0M    /etc
4.0K    /export
4.0K    /home
0    /initrd.img
0    /initrd.img.old
0    /lib
0    /lib32
0    /lib64
0    /libx32
16K    /lost+found
8.0K    /media
4.0K    /mnt
16K    /opt
du: cannot read directory '/proc/3781/task/3781/net': Invalid argument
du: cannot read directory '/proc/3781/net': Invalid argument
du: cannot read directory '/proc/3782/task/3782/net': Invalid argument
du: cannot read directory '/proc/3782/net': Invalid argument
du: cannot read directory '/proc/3783/task/3783/net': Invalid argument
du: cannot read directory '/proc/3783/net': Invalid argument
du: cannot read directory '/proc/3784/task/3784/net': Invalid argument
du: cannot read directory '/proc/3784/net': Invalid argument
du: cannot read directory '/proc/3785/task/3785/net': Invalid argument
du: cannot read directory '/proc/3785/net': Invalid argument
du: cannot read directory '/proc/3786/task/3786/net': Invalid argument
du: cannot read directory '/proc/3786/net': Invalid argument
du: cannot read directory '/proc/3787/task/3787/net': Invalid argument
du: cannot read directory '/proc/3787/net': Invalid argument
du: cannot read directory '/proc/3788/task/3788/net': Invalid argument
du: cannot read directory '/proc/3788/net': Invalid argument
du: cannot read directory '/proc/3789/task/3789/net': Invalid argument
du: cannot read directory '/proc/3789/net': Invalid argument
du: cannot access '/proc/3790/task/3790/fd/4': No such file or directory
du: cannot access '/proc/3790/task/3790/fdinfo/4': No such file or directory
du: cannot access '/proc/3790/fd/3': No such file or directory
du: cannot access '/proc/3790/fdinfo/3': No such file or directory
0    /proc
137M    /root
710M    /run
0    /sbin
110T    /srv
0    /sys
0    /tmp
5.3G    /usr
12G    /var
0    /vmlinuz
0    /vmlinuz.old

Alles anzeigen

Code

root@Optimus:/# du -sh /srv/*
12T	/srv/dev-disk-by-label-DATA1
12T	/srv/dev-disk-by-label-DATA2
12T	/srv/dev-disk-by-label-DATA3
2.9T	/srv/dev-disk-by-label-DATA5
2.9T	/srv/dev-disk-by-label-DATA6
2.9T	/srv/dev-disk-by-label-DATA7
2.9T	/srv/dev-disk-by-label-DATA8
2.9T	/srv/dev-disk-by-label-DATA9
12T	/srv/dev-disk-by-label-PARITY1
49T     /srv/ed0e27e4-ee6b-478a-b1d2-3da0707c2346
8.0K	/srv/ftp
32K	/srv/pillar
1.7M	/srv/salt

Alles anzeigen

Code

root@Optimus:/# du -sh /var/*
180K	/var/backups
14M	/var/cache
12G	/var/lib
4.0K	/var/local
0	/var/lock
270M	/var/log
4.0K	/var/mail
4.0K	/var/opt
0	/var/run
356K	/var/spool
4.0K	/var/tmp
6.8M	/var/www

Alles anzeigen

Code

root@Optimus:/# sudo du -d1 -x -h / | sort -h
4.0K	/archive
4.0K	/export
4.0K	/home
4.0K	/mnt
8.0K	/media
16K	/lost+found
16K	/opt
1.7M	/srv
7.0M	/etc
137M	/root
1015M	/boot
5.3G	/usr
12G	/var
19G	/

Alles anzeigen

Code

root@Optimus:/# blkid
/dev/sdj2: UUID="68118a3d-cb37-4ce4-86fa-3b5e990f6cec" TYPE="ext4" PARTUUID="a131f1de-bf74-483d-a874-ee0cfa800c39"
/dev/sdj3: UUID="03ae3308-ab16-42b7-8863-78d845153cb6" TYPE="swap" PARTUUID="62ca5e7a-e34d-4871-bdd0-9fded74b240f"
/dev/sde1: LABEL="DATA8" UUID="060f8263-2674-4cdf-b966-297904fcbfd7" TYPE="ext4" PARTUUID="1ea66cd0-1560-48c7-8b14-ef03e2657124"
/dev/sdb1: LABEL="DATA2" UUID="2444c410-685e-4f36-933e-c0f62a05c811" TYPE="ext4" PARTUUID="7ec6f706-f126-4119-b696-05e240272c58"
/dev/sdi1: LABEL="DATA7" UUID="ccbfc8ee-7a44-45ff-9bb9-0973c8db505d" TYPE="ext4" PARTUUID="a7ff6829-3a7d-4a3d-8334-2a2920d5ccb7"
/dev/sdf1: LABEL="DATA9" UUID="41ed4af9-eb76-49dc-91dc-bfe840f9e059" TYPE="ext4" PARTUUID="40f88b39-2445-495e-be86-b4310042b99d"
/dev/sdh1: LABEL="DATA6" UUID="b9995e0d-4daf-42a0-8c25-04e41e701ae9" TYPE="ext4" PARTUUID="57244f74-ab8a-4a7a-b28a-47930edd45e4"
/dev/sdd1: LABEL="PARITY1" UUID="e8cc9c92-1f7e-41a0-905a-4173f662f75c" TYPE="ext4" PARTUUID="c63340b5-1aef-43d7-8d89-14706ceb7c15"
/dev/sdc1: LABEL="DATA1" UUID="bc4523ce-2d74-406b-a3f2-cc4987a32d1f" TYPE="ext4" PARTUUID="33b9c58b-c543-4575-961b-fe7bc2e3a16f"
/dev/sda1: LABEL="DATA3" UUID="2c95f714-7151-4085-8c98-45696e6b28ca" TYPE="ext4" PARTUUID="0ed33adb-e099-4d71-861f-30e287b8011d"
/dev/sdg1: LABEL="DATA5" UUID="03af9c75-07ec-49f2-98cf-5a81202ea939" TYPE="ext4" PARTUUID="ff478ea1-0447-46a8-a171-ecf95791cad9"
/dev/sdj1: PARTUUID="9db144df-985e-4410-9691-7f21c2d9d5f7"

Alles anzeigen

Code

root@Optimus:/# tune2fs -l /dev/sdj2
tune2fs 1.46.2 (28-Feb-2021)
Filesystem volume name:   <none>
Last mounted on:          /
Filesystem UUID:          68118a3d-cb37-4ce4-86fa-3b5e990f6cec
Filesystem magic number:  0xEF53
Filesystem revision #:    1 (dynamic)
Filesystem features:      has_journal ext_attr resize_inode dir_index filetype needs_recovery extent 64bit flex_bg sparse_super large_file huge_file dir_nlink extra_isize metadata_csum
Filesystem flags:         signed_directory_hash
Default mount options:    user_xattr acl
Filesystem state:         clean
Errors behavior:          Continue
Filesystem OS type:       Linux
Inode count:              1286144
Block count:              5120000
Reserved block count:     255999
Free blocks:              4096
Free inodes:              968439
First block:              0
Block size:               4096
Fragment size:            4096
Group descriptor size:    64
Reserved GDT blocks:      1024
Blocks per group:         32768
Fragments per group:      32768
Inodes per group:         8192
Inode blocks per group:   512
Flex block group size:    16
Filesystem created:       Mon Jul 27 20:56:59 2020
Last mount time:          Tue Aug  3 19:10:22 2021
Last write time:          Tue Aug  3 19:10:19 2021
Mount count:              16
Maximum mount count:      -1
Last checked:             Sun Aug 23 16:41:00 2020
Check interval:           0 (<none>)
Lifetime writes:          2291 GB
Reserved blocks uid:      0 (user root)
Reserved blocks gid:      0 (group root)
First inode:              11
Inode size:              256
Required extra isize:     32
Desired extra isize:      32
Journal inode:            8
Default directory hash:   half_md4
Directory Hash Seed:      11cba8e2-159d-4628-8901-480070806aa3
Journal backup:           inode blocks
Checksum type:            crc32c
Checksum:                 0xd15e9377

Alles anzeigen

Going down the rabbit hole here...

Code

root@Optimus:/# df -h /mnt
Filesystem      Size  Used Avail Use% Mounted on
/dev/sdj2        19G   19G     0 100%

Dejh0n · 4. August 2021

Thanks for the reply.

Here is my results:

"Cleaning Up dockers"

Ran:

Code

docker system df

Returned:

Code

root@Optimus:~# docker system df
Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?
root@Optimus:~#

Then moved onto "moving locker storage"

and got the following:

root@Optimus:~# cat /etc/docker/daemon.json

{

"runtimes": {

"nvidia": {

"path": "nvidia-container-runtime",

"runtimeArgs": []

}

Portainer or any of my services are not running..

Dejh0n · 3. August 2021

To follow up, I am happy using terminal but I do not know what commands to use where; however am good at following instructions

Dejh0n · 3. August 2021

Hi team,

So I’ve had OMV up and running for a year now and it’s been pretty great.

I’ve installed all my things (containers etc) and got it set up and just how I like and it’s been rock solid for a few months now.

Now here is the issue. The volume my containers are stored on has seemingly gotten full even with regular image pruning and apt-cleaning. Now it is so full none of my containers will start, the web UI won’t load and I can only access it via SSH.

Can someone please help me get this back online and then help me migrate the small volume on an old 3TB HDD onto a new 500gb SSD (in that order please!)

Beiträge von Dejh0n

Can’t access services, suspected volume full

Can’t access services, suspected volume full

Can’t access services, suspected volume full

Can’t access services, suspected volume full

Can’t access services, suspected volume full

Can’t access services, suspected volume full

Can’t access services, suspected volume full