Backup Plugin Errors

Silvaire · 8. November 2015

Hi, I have had two problems with the Backup plugin.

1. I have PartedMagic so I used the plugin gui to load the iso into grub. When testing it out, PartedMagic failed to boot its gui due to a mount error and kicked me to command-line. The same iso burned to a USB key works fine.

2. The system backup is set to back up to a hard drive that is completely empty and recently formatted. The drive has over 120 GB free. However, halfway through the rsync it starts kicking out "no space available" errors and the backup fails.

Here is a snippet from the end of the transfer:

Code

rsync: recv_generator: mkdir "/media/6b153e75-f80f-42b7-8aad-4f4033094706/omvbackup/var/mail" failed: No space left on device (28)
*** Skipping any contents from this failed directory ***
var/opt/
rsync: recv_generator: mkdir "/media/6b153e75-f80f-42b7-8aad-4f4033094706/omvbackup/var/opt" failed: No space left on device (28)
*** Skipping any contents from this failed directory ***
var/spool/
rsync: recv_generator: mkdir "/media/6b153e75-f80f-42b7-8aad-4f4033094706/omvbackup/var/spool" failed: No space left on device (28)
*** Skipping any contents from this failed directory ***
var/tmp/
rsync: recv_generator: mkdir "/media/6b153e75-f80f-42b7-8aad-4f4033094706/omvbackup/var/tmp" failed: No space left on device (28)
*** Skipping any contents from this failed directory ***
var/www/
rsync: recv_generator: mkdir "/media/6b153e75-f80f-42b7-8aad-4f4033094706/omvbackup/var/www" failed: No space left on device (28)
*** Skipping any contents from this failed directory ***


sent 1974991864 bytes  received 467271 bytes  33768532.22 bytes/sec
total size is 2647076831  speedup is 1.34
rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1070) [sender=3.0.9]

Alles anzeigen

Any ideas? Thanks.

Silvaire · 8. November 2015

On a related note, #2 will consistently crap out my web-GUI for OMV requiring a restart of NginX in SSH.

ryecoaaron · 8. November 2015

Sounds like something is wrong with the system. What kind of drive is OMV installed on? Any errors in the logs?

Silvaire · 9. November 2015

OMV is installed on a 256 GB Samsung 830 SSD -- a 20 GB partition with 15.93 GB free. The destination for the backup is on another SSD, a 120 GB Samsung 840, which now has 109.9 GB free after the failed rsync.

I don't see any relevant errors in the log. I just ran another failed backup and nothing showed up in syslog.

ryecoaaron · 9. November 2015

Is the backup drive mounted read only for some reason? Can you try manually copying a few files in command line from the OS drive to the data drive?

Silvaire · 9. November 2015

Sure enough when trying to copy a deb file to that drive from the OS drive, I get a out of space error:

cp: cannot create regular file `/media/6b153e75-f80f-42b7-8aad-4f4033094706/plexmediaserver_0.9.12.19.1537-f38ac80_amd64.deb': No space left on device

It's weird because I had completely wiped the partitions, and formatted it with ext4 before bringing it into this OMV. Every other too, including OMV's webgui itself, that reports drive health and partitions shows it as having 111.77 GB usable space total with 109.9 GB avaiable (the used space is the part of the rsync that succeeds before it reports drive full). I also have never had issues with this drive immediately before converting to this 100% OMV build (I used to virtualize OMV).

EDIT: The only other weird thing I've noticed is just now I have a bogus entry in my filesystem tab in OMV that is filled with "n/a" at the top of the list. I'm scared to delete it but I have no idea what it is.

ryecoaaron · 9. November 2015

For now, don't worry about the n/a filesystem. It isn't hurting anything. What is the output of mount and blkid

Silvaire · 9. November 2015

http://pastebin.com/HWBtqTyp

Thanks. Only other thing I can think of is I did disable quota checks when I installed OMV because in the past I've had issues with it taking awhile. I don't know if it's related. Also using your flashmemory plugin for my SSD.

ryecoaaron · 9. November 2015

I don't think the quotas being disabled is causing the problem. I have done that many times. I see you are using mergerfs. When you were trying to copy files, were you trying to copy to the mergefs pool? If so, can you try copying a file to an individual drive? What kernel are you using - 3.2 or 3.16?

Silvaire · 9. November 2015

1. MergerFS was not involved at all with the rsync operation between OMV drive and OMV backup drive. Neither are on the pool. I have 6 mechanical drives in the mergerFS pool.

2. 3.2.0-4-AMD64

ryecoaaron · 9. November 2015

Is it a newer system? I use the backports 3.16 kernel on all of my systems. Just wondering if it would help. If that doesn't help, I would probably boot systemrescuecd, mount a few of the drives, and see if I could write to them. They are mounted rw. I don't understand why you can't write to them especially if there is no error logs. What is the output of: cat /etc/fstab

Silvaire · 9. November 2015

I have no general issues writing to any of the drives outside of this one particular SSD during RSYNC through the plugin. My issue in the other thread seems narrowly-focused on a particular circumstance but generally speaking I have had no issues, for example, doing large samba file transfers between the drives on the mergerfs pool and between the pool and another drive. The backup SSD drive is empty so I can easily nuke it again and report back further tests. Last time I formatted it, I did it in PartedMagic and the following: "makefs.ext4 -m 0 -T largefile4" after creating a partition in parted. I will recreate the filesystem.

As for the system, it is a Supermicro X10SL7-F with a Xeon E3-1231 3.4 GHZ. I had been running OMV in an ESXI VM for years with PCI passhtrough of my LSI HBA but recently decided to reduce down to baremetal and run everything through OMV.

Code

root@Paragon:~# cat /etc/fstab
# /etc/fstab: static file system information.
#
# Use 'blkid' to print the universally unique identifier for a
# device; this may be used with UUID= as a more robust way to name devices
# that works even if disks are added and removed. See fstab(5).
#
# <file system> <mount point>   <type>  <options>       <dump>  <pass>
# / was on /dev/sda1 during installation
UUID=9fa1dc98-db47-4e7e-901a-e80a967079f0 /               ext4    noatime,nodiratime,errors=remount-ro 0       1
# swap was on /dev/sda5 during installation
#UUID=5f50599d-4e5e-47b6-91fb-00aa563876db none            swap    sw              0       0
/dev/sdc1       /media/usb0     auto    rw,user,noauto  0       0
/dev/sdc2       /media/usb1     auto    rw,user,noauto  0       0
/dev/sdc5       /media/usb2     auto    rw,user,noauto  0       0
/dev/sdd1       /media/usb3     auto    rw,user,noauto  0       0
# >>> [openmediavault]
UUID=c3293904-29bd-4334-aa0e-ab8476867f17 /media/c3293904-29bd-4334-aa0e-ab8476867f17 ext4 defaults,nofail,user_xattr,noexec,usrjquota=aquota.user,grpjquota=aquota.group,jqfmt=vfsv0,acl 0 2
UUID=733ee5f3-9dc9-472d-9055-7614e788db1a /media/733ee5f3-9dc9-472d-9055-7614e788db1a ext4 defaults,nofail,user_xattr,noexec,usrjquota=aquota.user,grpjquota=aquota.group,jqfmt=vfsv0,acl 0 2
UUID=944d29bf-bc87-416d-b335-1b8888be37ef /media/944d29bf-bc87-416d-b335-1b8888be37ef ext4 defaults,nofail,user_xattr,noexec,usrjquota=aquota.user,grpjquota=aquota.group,jqfmt=vfsv0,acl 0 2
UUID=65fab1ad-8125-40bd-9dbc-af40496a99fa /media/65fab1ad-8125-40bd-9dbc-af40496a99fa ext4 defaults,nofail,user_xattr,noexec,usrjquota=aquota.user,grpjquota=aquota.group,jqfmt=vfsv0,acl 0 2
UUID=192d2dad-e2a6-4011-ad7c-ebabb36b192e /media/192d2dad-e2a6-4011-ad7c-ebabb36b192e ext4 defaults,nofail,user_xattr,noexec,usrjquota=aquota.user,grpjquota=aquota.group,jqfmt=vfsv0,acl 0 2
UUID=438c3496-c190-40c9-9fda-3c5cc408aa41 /media/438c3496-c190-40c9-9fda-3c5cc408aa41 ext4 defaults,nofail,user_xattr,noexec,usrjquota=aquota.user,grpjquota=aquota.group,jqfmt=vfsv0,acl 0 2
UUID=1c7962e6-19ad-419a-84a7-7ca93d651804 /media/1c7962e6-19ad-419a-84a7-7ca93d651804 ext4 defaults,nofail,user_xattr,noexec,usrjquota=aquota.user,grpjquota=aquota.group,jqfmt=vfsv0,acl 0 2
UUID=462ab19c-5e03-4091-8318-9b1b9ad0267e /media/462ab19c-5e03-4091-8318-9b1b9ad0267e ext4 defaults,nofail,user_xattr,noexec,usrjquota=aquota.user,grpjquota=aquota.group,jqfmt=vfsv0,acl 0 2
UUID=26c544b4-2fe1-469b-882f-685b03c0d2e8 /media/26c544b4-2fe1-469b-882f-685b03c0d2e8 ext4 defaults,nofail,user_xattr,noexec,usrjquota=aquota.user,grpjquota=aquota.group,jqfmt=vfsv0,acl 0 2
UUID=6b153e75-f80f-42b7-8aad-4f4033094706 /media/6b153e75-f80f-42b7-8aad-4f4033094706 ext4 defaults,nofail,user_xattr,noexec,usrjquota=aquota.user,grpjquota=aquota.group,jqfmt=vfsv0,acl 0 2
/media/c3293904-29bd-4334-aa0e-ab8476867f17:/media/733ee5f3-9dc9-472d-9055-7614e788db1a:/media/65fab1ad-8125-40bd-9dbc-af40496a99fa:/media/438c3496-c190-40c9-9fda-3c5cc408aa41:/media/192d2dad-e2a6-4011-ad7c-ebabb36b192e:/media/1c7962e6-19ad-419a-84a7-7ca93d651804 /media/5e3a0513-b56d-4b02-a635-50e6ca1622af fuse.mergerfs defaults,allow_other,category.create=epmfs,minfreespace=4G 0 0
# <<< [openmediavault]

Alles anzeigen

Silvaire · 9. November 2015

OK so using the OMV-GUI to delete the filesystem on the SSD and create a new one, the filesystem creation window says it completed successfully but the webui does throw an error upon completion:

Code

Creating the file system, please wait ...
Warning: The kernel is still using the old partition table.
The new table will be used at the next reboot.
GPT data structures destroyed! You may now partition the disk using fdisk or
other utilities.
Creating new GPT entries.
Information: Moved requested sector from 34 to 2048 in
order to align on 2048-sector boundaries.
Disk /dev/sdj: 234441648 sectors, 111.8 GiB
Logical sector size: 512 bytes
Disk identifier (GUID): 4DC3DC19-EA80-4929-BABB-4E18EC26442F
Partition table holds up to 128 entries
First usable sector is 34, last usable sector is 234441614
Partitions will be aligned on 2048-sector boundaries
Total free space is 2014 sectors (1007.0 KiB)


Number  Start (sector)    End (sector)  Size       Code  Name
   1            2048       234441614   111.8 GiB   8300  
Warning: The kernel is still using the old partition table.
The new table will be used at the next reboot.
The operation has completed successfully.

Alles anzeigen

Code

Error #4000:
exception 'OMVException' with message 'Failed to execute command 'export LANG=C; partprobe '/dev/sdj'': ' in /usr/share/openmediavault/engined/rpc/filesystemmgmt.inc:614
Stack trace:
#0 [internal function]: OMVRpcServiceFileSystemMgmt->create(Array, Array)
#1 /usr/share/php/openmediavault/rpcservice.inc(125): call_user_func_array(Array, Array)
#2 /usr/share/php/openmediavault/rpc.inc(79): OMVRpcServiceAbstract->callMethod('create', Array, Array)
#3 /usr/sbin/omv-engined(500): OMVRpc::exec('FileSystemMgmt', 'create', Array, Array, 1)
#4 {main}

Rebooted and the new filesystem does not show up in the tab. The HDD is still reported correctly in Physical Disks but no reference to it in filesystems. The bogus filesystem of n/a still shows up with a status of 'missing'.

EDIT: OK, I wiped it again in OMV, and made another EXT4 partition and this time it shows up.

Silvaire · 9. November 2015

Sorry for the post spam. Good news though. I just ran the rsync again on the new filesystem and it worked fine. I'm not sure why it took 3 filesystem creation's (1 in partedmagic, 2 in omv) to get the stink off my SSD but I won't look a gift horse in the mouth. I'll run the sync a few more times to be sure.

If so, that eliminates 1 of 3 problems. Remaining are the bogus filesystem, and the other thread issue.

ryecoaaron · 9. November 2015

That makes more sense. Glad it is working now. I have that same motherboard. I still like the 3.16 backports kernel on it.

You can try deleting the n/a filesystem in the web interface. If it won't let you, you need to manually edit the /etc/openmediavault/config.xml file to remove it. Just make a backup before doing that. Can't help much with the other thread.

Silvaire · 9. November 2015

Thanks for the help! I'll see if I can narrow the issue down re: nzbget on the nzbget forum before I come crawling back to see if anyone here has had similar issues.

With regard to deleting the n/a filesystem, it does allow me to do it, but it has no impact on removing it. I'm int he XML file now but I'm not sure where an edit would be made. The <mntend> section details all my mounts. The <filesystem> section doesn't seem to have anything I could granularly remove either.

I've seen in the forum other people seem to have this issue with the n/a--missing. I am not sure what caused it for me. It only appeared today.

EDIT: I think I see what I have to do. Running through various tests to see if it gets rid of it.

EDIT2: Got the little bugger! 2 problems down, 1 to go. Thanks again.

Jetzt mitmachen!