Search files and sort them by cli

Zitat von riff-raff

the option -i should not be neccessary or is it? Just in case there is triplet of files?

The latter. But I would do it differently and compare also the file's contents. Something like

MD5Sum=$(md5sum "${REPLY}" | awk -F" " '{print $1}')

creates a hash over the contents so you could do fine grained comparisons and sort also based on contents.

I usually do tasks like this in this way

#!/bin/bash
find /srv/dev-disk-by-xxx/folder1 -type f -name '*.cvs' | while read ; do
	MD5Sum=$(md5sum "${REPLY}" | awk -F" " '{print $1}')
	echo -e "${MD5Sum}\t${REPLY##*/}"
done >/tmp/list.txt

And then let list.txt be processed (sorting by column combined with 'uniq -d' to get only duplicates listed)

Searching for files with same contents would then be simply this

#!/bin/bash
awk '{print $1}' </tmp/list.txt | sort | uniq -d | while read ; do
	grep "^${REPLY}" /tmp/list.txt
done

Searching for file with same filenames then just

#!/bin/bash
awk '{print $2}' </tmp/list.txt | sort | uniq -d | while read ; do
	grep "${REPLY}$" /tmp/list.txt
done

Zitat von riff-raff

This list only contains duplicates with same content?

The list contains all filenames and all MD5 hashes. Last post above with the two script snippets shows how you can use sort/uniq to further process the list to find everything that's interesting. Since nothing is actually moved you can simply try it out.

If you want to start postprocessing the above stuff should be somewhat save wrt 'creative filenames' (I do this for a living, crawling through directories on Unix servers where macOS users store their stuff. They use literally every special character possible )

Finding files with identical name and also contents is then simply

sort </tmp/list.txt | uniq -d

Zitat von riff-raff

hashing all the files seems to take quite a while

Yeah, it's essentially reading all contents of the files matching the name pattern and the find run is also already some sort of a benchmark.

This variant would provide some user feedback to stdout while writing to list.txt at the same time:

#!/bin/bash
find /srv/dev-disk-by-xxx/folder1 -type f -name '*.cvs' | while read ; do
	MD5Sum=$(md5sum "${REPLY}" | awk -F" " '{print $1}')
	echo -e "${MD5Sum}\t${REPLY##*/}" | tee -a /tmp/list.txt
done

(but I dealt once with a tee variant that implemented fsync() pretty closely and writing the list file became almost a bottleneck)

Zitat von riff-raff

By listing the files with same content a later on process to remove the duplicate

Now guess what could've happened if you would've asked for 'how to find and possibly delete duplicates?' in the first place?

At least now I'll recommend doing an

sudo apt install fdupes
man fdupes

Zitat von riff-raff

I want to move all folders (containing files) with beginning letter "A" to another location.

find $lala -name "A*" -type d | $filter | while read; do
    mv "${REPLY}" $somewhere
done

('| $filter' being optional of course)

Zitat von riff-raff

Does not work

It does work this or that way

Well, this is basic scripting stuff totally unrelated to OMV. Maybe you need -maxdepth or -mindepth operators too. IMO it's worth to learn these basics if you have to do this stuff more often (otherwise it's a waste of time!)