Missing Raid

  • Should have been du -xhd 1 /srv/dev-disk-by-uuid-F48E91208E90DD04/ without the space.

    Code
    root@NAS:~# du -xhd 1 /srv/dev-disk-by-uuid-F48E91208E90DD04/
    2.4T    /srv/dev-disk-by-uuid-F48E91208E90DD04/Anna
    96M     /srv/dev-disk-by-uuid-F48E91208E90DD04/Config
    0       /srv/dev-disk-by-uuid-F48E91208E90DD04/Download
    0       /srv/dev-disk-by-uuid-F48E91208E90DD04/Downloads
    0       /srv/dev-disk-by-uuid-F48E91208E90DD04/lost+found
    57M     /srv/dev-disk-by-uuid-F48E91208E90DD04/Riccardo
    2.4T    /srv/dev-disk-by-uuid-F48E91208E90DD04/
    root@NAS:~#

    There should be more than 1 TiB of data in the 'Riccardo' folder!

  • Your Debiuan RAID has 2.3TB, you rsyncc it to this Disk, why would you expect 2.4TB + 1 TB ofdata.

    Execute the same command for the other disk du -xhd 1 /srv/dev-disk-by-id-md-name-debian-Raind/


    And once you see, the used space decrease, do the same.


    On thought about the changing space usage:

    Is it possible, rsync has never executed properly and you are seeing the regular space usage changes in the directory Anna?

    If you got help in the forum and want to give something back to the project click here (omv) or here (scroll down) (plugins) and write up your solution for others.

  • why would you expect 2.4TB + 1 TB ofdata.

    Actually, I don't expect 2.4 + 1 TB, but what the command du -xhd 1 /srv/dev-disk-by-uuid-F48E91208E90DD04/

    returns to me is different from what I actually see in the folders. The 2.3 TB must be divided between the "Anna" folder and the "Riccardo" folder. In the "Richard" folder there is about 1 TB.


    And once you see, the used space decrease, do the same.

    I'm sorry, I don't quite understand what you mean; could you please explain it again?


    Is it possible, rsync has never executed properly and you are seeing the regular space usage changes in the directory Anna?

    Sorry, I don't quite understand the question; can you explain what you mean, please? Thank you

  • Lets disect it:

    Code
    root@NAS:~# du -xhd 1 /srv/dev-disk-by-uuid-F48E91208E90DD04/
    2.4T    /srv/dev-disk-by-uuid-F48E91208E90DD04/Anna
    96M     /srv/dev-disk-by-uuid-F48E91208E90DD04/Config
    0       /srv/dev-disk-by-uuid-F48E91208E90DD04/Download
    0       /srv/dev-disk-by-uuid-F48E91208E90DD04/Downloads
    0       /srv/dev-disk-by-uuid-F48E91208E90DD04/lost+found
    57M     /srv/dev-disk-by-uuid-F48E91208E90DD04/Riccardo
    2.4T    /srv/dev-disk-by-uuid-F48E91208E90DD04/
    root@NAS:~#
    • Line 2: There are 2.4T in directory Anna
    • Line 7: There a 57M in directory Riccardo
    • Line 8: The total is 2.4T

    You told before, the you see the disk usage for this drive get larger and smaller. (between 1.9T and 2.4T)

    Rsync does not delete files which still exist in the source even with --delete set.

    In Riccardo there are not enough files (disk usage) to explan the difference you see, so it must be in Anna.

    Is Anna deleteing and creating files in her directory and you see this in the disk usage?


    To analyse a bit further:


    du -xhd 1 /srv/dev-disk-by-uuid-F48E91208E90DD04/Anna will show how the disk space is used.

    du -xhd 1 /srv/dev-disk-by-id-md-name-debian-Raid/  will show what you would copy with rsync

    And to get closer we look into Anna ind Riccardo on the source as well:

    du -xhd 1 /srv/dev-disk-by-id-md-name-debian-Raid/Anna 

    du -xhd 1 /srv/dev-disk-by-id-md-name-debian-Raid/Riccardo

    If you got help in the forum and want to give something back to the project click here (omv) or here (scroll down) (plugins) and write up your solution for others.

  • Is Anna deleteing and creating files in her directory and you see this in the disk usage?

    No one is operating on the Raid at this time.



    Code
    root@NAS:~# du -xhd 1 /srv/dev-disk-by-id-md-name-debian-Raid/
    24K     /srv/dev-disk-by-id-md-name-debian-Raid/Download
    16K     /srv/dev-disk-by-id-md-name-debian-Raid/lost+found
    1.2T    /srv/dev-disk-by-id-md-name-debian-Raid/Anna
    1001G   /srv/dev-disk-by-id-md-name-debian-Raid/Riccardo
    604M    /srv/dev-disk-by-id-md-name-debian-Raid/Config
    102G    /srv/dev-disk-by-id-md-name-debian-Raid/Downloads
    2.3T    /srv/dev-disk-by-id-md-name-debian-Raid/

    Code
    root@NAS:~# du -xhd 1 /srv/dev-disk-by-id-md-name-debian-Raid/Riccardo
    932G    /srv/dev-disk-by-id-md-name-debian-Raid/Riccardo/Foto-Video
    55G     /srv/dev-disk-by-id-md-name-debian-Raid/Riccardo/Sistemi Operativi & Sw
    95M     /srv/dev-disk-by-id-md-name-debian-Raid/Riccardo/Libri Fotografia
    15G     /srv/dev-disk-by-id-md-name-debian-Raid/Riccardo/PcGame
    1001G   /srv/dev-disk-by-id-md-name-debian-Raid/Riccardo
  • The disks do not match at all! Has the target disk been empty before you did the rsync?


    On the target disk, all space is used here:

    Code
    2.4T    /srv/dev-disk-by-uuid-F48E91208E90DD04/Anna/CLUB ACADEMY


    and on the source it looks completly different. It is distributed on several directories:

    Code
    root@NAS:~# du -xhd 1 /srv/dev-disk-by-id-md-name-debian-Raid/
    24K     /srv/dev-disk-by-id-md-name-debian-Raid/Download
    16K     /srv/dev-disk-by-id-md-name-debian-Raid/lost+found
    1.2T    /srv/dev-disk-by-id-md-name-debian-Raid/Anna
    1001G   /srv/dev-disk-by-id-md-name-debian-Raid/Riccardo
    604M    /srv/dev-disk-by-id-md-name-debian-Raid/Config
    102G    /srv/dev-disk-by-id-md-name-debian-Raid/Downloads
    2.3T    /srv/dev-disk-by-id-md-name-debian-Raid/


    Can you check the rsync job if you do not sync into the shared folder somwhere deeply nested inside /srv/dev-disk-by-uuid-F48E91208E90DD04/Anna/CLUB ACADEMY


    Step down further into this directory to see, what is really going on. Allways check for the larges directory and execute the

    du -xhd 1/srv/dev-disk-by-uuid-F48E91208E90DD04/Anna/CLUB\ ACADEMY/... command in it. Use TAB-completion to type the directory names to avoid typos.

    If you got help in the forum and want to give something back to the project click here (omv) or here (scroll down) (plugins) and write up your solution for others.

  • To be clear:
    To do an rsync disk to disk mirror, the disk mount point is used followed by a "/"
    For example:

    /srv/dev-disk-by-uuid-F48E91208E90DD04/

    _________________________________________________________________

    The device name with a following back slash can be used as well, for example:


    /dev/md0/

    __________________________________________________________________


    The slash " / " is the only thing added to the mount point or device name.  The same thing is done with the destination mount point or device name. Adding anything after the slash, in either the source or the destination, will NOT produce a mirror in the destination drive.


  • The disks do not match at all! Has the target disk been empty before you did the rsync?

    Yes, it was empty


    Code
    root@NAS:~# /srv/dev-disk-by-uuid-F48E91208E90DD04/Anna/CLUB ACADEMY
    -bash: /srv/dev-disk-by-uuid-F48E91208E90DD04/Anna/CLUB: No such file or directory


    Code
    root@NAS:~# du -xhd 1/srv/dev-disk-by-uuid-F48E91208E90DD04/Anna/CLUB ACADEMY/
    du: cannot access 'ACADEMY/': No such file or directory


    Something is not working



    I put the slash in the rsync command!

    (If you mean this command)

    rsync -av /srv/dev-disk-by-id-md-name-debian-Raid/ /srv/dev-disk-by-uuid-F48E91208E90DD04/

  • you did not use tab-completion to enter the commands. The directory name has a space in it which must be escape by backslash space or you just hit the tab key several times to complete the path.


    I am nearly 100% sure the files have not been copied with the rsync command given above, otherwise you would have at least the same directories on top level.

    If you got help in the forum and want to give something back to the project click here (omv) or here (scroll down) (plugins) and write up your solution for others.

  • I put the slash in the rsync command!

    (If you mean this command)

    rsync -av /srv/dev-disk-by-id-md-name-debian-Raid/ /srv/dev-disk-by-uuid-F48E91208E90DD04/

    Note that I've never had rsync issues like you're describing. However, I use rsync to copy from EXT4 to EXT4 (or another Linux / OMV compatible disk format).

    All I can I can think of that might cause some of these problems are:
    - The NTFS format which can result in "foreign volume" permission issues.

    - You might check the SMART stat's of the external drive to make sure it's healthy.

    Otherwise, if NTFS must be use with the external disk, maybe it might be best to use -> FreeFileSync mentioned before, coping network shares over the network to a Windows Client. You'll get the progress meter and error reporting, up front, in the foreground.

  • I'm back. After a busy week, today I got back on track.

    I decided to copy folder by folder using WinSCP. It will be a long job but at least I know it will be done.

    There are some files that I can't copy in any way, probably damaged, but I can open them with the various video players or photo viewers. Is it possible that there's no way to force them to be copied?

  • WinSCP can be used for file copies but occasionally, as you're seeing, it seems to choke on numerous files.

    If doing numerous / large file copies locally, it's better to use Midnight Commander. It's a command line / graphic utility for file management. You can read about it -> here.

  • Here I am again guys.


    So far I've been trying to save a copy of all the various files to hd. I had almost managed to complete the backup of all the most important files.


    Unfortunately, however, it seems that the hd with the O.S. has failed.


    The system rebooted but OMV no longer loads.


    I tried connecting the hd via USB to the PC and Windows can see it, or at least it senses that there is an hd connected and asks me to format it.


    I ask again: is it possible in case I can read the contents of this hd, to clone OMV to another hd and continue the backup?

  • To be honest, I do not recal what the original problem was and am too lazy to read up this thread.


    OMV fortmats disks with ext4 file system whic hcan be read on every linux system, so if you plug the drive in question (not the os disk) into a linux computer you can copy the contents of it.


    Or you can try this tool to read the data on the ext4 disk from windows: https://www.diskinternals.com/linux-reader/ (never used it myself).

    If you got help in the forum and want to give something back to the project click here (omv) or here (scroll down) (plugins) and write up your solution for others.

  • To be honest, I do not recal what the original problem was and am too lazy to read up this thread.


    OMV fortmats disks with ext4 file system whic hcan be read on every linux system, so if you plug the drive in question (not the os disk) into a linux computer you can copy the contents of it.


    Or you can try this tool to read the data on the ext4 disk from windows: https://www.diskinternals.com/linux-reader/ (never used it myself).

    The problem was that the raid had stopped working. I had a broken sata cable and some damaged hds.

    After several trials, thankful to your help, I managed to solve the problem with the raid and get the files backed up on a usb hd.

    Now I still haven't finished backing up and it seems that the hd OMV is installed on is broken. But, as I said in the previous post, Windows seems to recognise it.

    Now if possible I would like to copy the whole system to a new hd and resume the backup

    I will try the software you recommended and let you know.

  • If the software is on a RAID array, the recommendations will not help.

    If you system boots from an USB stick, make a basic install of OMV6 onto an USB stick, mount the raid, create a sahred folder and continue from there.

    If you got help in the forum and want to give something back to the project click here (omv) or here (scroll down) (plugins) and write up your solution for others.

  • If the software is on a RAID array, the recommendations will not help.

    If you system boots from an USB stick, make a basic install of OMV6 onto an USB stick, mount the raid, create a sahred folder and continue from there.

    OMV is installed on a boot hd.

    I'll try to see what I can do in the hope of succeeding.

    Thanks

  • I installed OMV on a new hard drive. I was able to mount the old RAID and see the folders and all the data.

    I don't understand how to see the usb hard drive I started the backup on. If I recreate it in File System it formats it for me and I don't have to format it!

Participate now!

Don’t have an account yet? Register yourself now and be a part of our community!