SMB Recycle Bin on mergerfs pool share not working

kranich · 15. Dezember 2015

I recently switched my disk pool to mergerfs. Now I realized that the recycle bin stopped working on my smb shares.

SW-Status:
openmediavault/stoneburner uptodate 2.1.20
openmediavault-unionfilesystems/stoneburner uptodate 2.3

What I did to reproduce:

Create shared folder on mergerfs pool
Give permissions to a user through a group
Create samba share on this shared folder.
Enable recycle bin
Enable inherited permissions
Log in using smb from Windows 7 client
Create a file on the share
Delete the file

Expected behavior: File moved to share/.recycle/username/file.xyz
Actual behavior: share/.recycle/username is created, but stays empty.

Why I think this has sth. to do with mergerfs? When I create an analog share directly on a disk and not on the pool, the file is correctly moved to the recycle bin as expected. To give all the details: The disk on which I created the direct share is still part of the pool. However, I am not very experienced with OMV so I do not rule out a general misconfiguration.

Second non-deterministic deletion issue:
While analyzing the issue above I sometimes I had the issue with files not being deleted at all. I observed this the last time even on the not-pool share. The deleted file disappeared on the client side while it was actually never deleted on the server. If you refresh the client's view the file is still present. I then restarted the smb daemon and out of a sudden the file was moved to the .recycle bin. I was not able to reproduce this scenario deterministically.

trapexit · 18. Dezember 2015

Have you checked the logs? I suspect Samba is expecting that a share is a single drive which mergerfs does not currently represent. That is... rename returns EXDEV. I'm betting that Samba is creating the directory path and then tries to `rename` the file into that path. Mergerfs returns EXDEV and then it just gives up. It should be performing a move but probably isn't.

I'm going to be putting out a new version of mergerfs soon which will handle this better but EXDEV is a totally legit return value and I will limit when it's returned but not entirely.

kranich · 19. Dezember 2015

Thanks for investigating @trapexit. Indeed you are absolutely right. I set the "log level" and "debug level" to 10 after your hint. In parallel I tried to find out what Samba is doing. I found some documentation that it hooks the unlink function. I checked out the source code of samba and searched for recycling functionality. Inside the vfs_recycle.c I found the corresponding actions.

The log file revealed exactly what you assumed:

Code

[2015/12/19 19:41:19.986667, 10] modules/vfs_recycle.c:609(recycle_unlink)
  recycle: recycled file name: .recycle/ruben///unke.odg
[2015/12/19 19:41:19.987103, 10] modules/vfs_recycle.c:640(recycle_unlink)
  recycle: Moving unke.odg to .recycle/ruben///unke.odg
[2015/12/19 19:41:19.987471,  3] modules/vfs_recycle.c:646(recycle_unlink)
  recycle: Move error 18 (Ungültiger Link über Gerätegrenzen hinweg), purging file unke.odg (.recycle/ruben///unke.odg)

According to /usr/include/asm-generic/errno-base.h 18 is exactly EXDEV. I looked into the source and it reads:

Code

DEBUG(10, ("recycle: Moving %s to %s\n", smb_fname_str_dbg(smb_fname),
        smb_fname_str_dbg(smb_fname_final)));
    rc = SMB_VFS_NEXT_RENAME(handle, smb_fname, smb_fname_final);

So obviously you are right in that it calls rename only. I looked into the source code of gnu mv and found a comment that they copy in case a rename does not succeed exactly for the reason you listed (EXDEV). I'll try to contact folks from samba to ask them why this only uses rename. Seems to be a bit naive.

PS: Just to have it linked in both directions: See also Issue 182 on mergerfs github.

trapexit · 20. Dezember 2015

Thanks for investigating. I hadn't had the chance yet but when it comes to moving files around lots of software fail to hand EXDEV correctly. As I said I'm trying to handle it better in a future version of mergerfs.

I'm trying to provide a hybrid solution which doesn't just straight up replicate the unlink target, copy files, unlink source that mv does on EXDEV.

If not using a path preserving policy then I'll just clone the target path on the source drive and rename. If a path preserving policy is being used try the rename; if it fails but a create would end up on that drive, clone the path on the source drive from target and then rename. If all that fails then return EXDEV.

Jetzt mitmachen!