Greyhole problem, slow network transfer

Ok, I have been throwing files around for a while and am seeing some odd behaviour, the problem is that i am totally unsure if it's Greyhole, Samba or Windows related.

Transfer rates I have been getting when copying a large folder (38GB, 364 files, various sizes) are as follows:

WindowsHomeServer 2011 -> Windows 8.1 = ~100MB/s at peak but dropping to ~50MB/s for the small files. Average is around 80-90MB/s
WindowsHomeServer 2011 -> OMV greyhole share = fairly steady ~35MB/s
Windows 8.1 -> OMV greyhole share = very steady ~75-80MB/s
WHS 2011 -> OMV share outside of the greyhole pool (but on same drive) = ~35MB/s

Now this would seem to point to WHS2011 being the problem but the slow file transfer from that is only to OMV. The WHS machine is my main longterm server and has been reliably transferring data to and from Windows and Mac OS X machines, at the speed I would expect, for years without issue.

I am seeing the same thing here, large files are fine @ ~100MB/s but when the transfer hits a small file it's dropping right down to ~10MB/s

WHS2011 is an HP N54L Microserver - Embedded NC107i PCI Express Gigabit Ethernet Server Adapter
OMV is HP N40L - as above.
Windows 8.1 machine self built, NIC is onboard and is - Intel 82579V Gigabit NIC

I'm starting to see a couple of PCIE Intel NICs in my future.. :?

Yep, I'm thinking this is more of a Linux/dodgy NIC issue than anything samba or greyhole related. I am throwing some more files around to confirm that the file duplication is working ok but I think the Greyhole issue is pretty much solved. Just need to wait for this build to roll out to the repo once Mr Boudreau is happy with it.

Ok, sorry to drag this thread back from the dead but I am still not seeing the performance that I would expect.
I have changed the NIC in both the WHS2011 machine (HP N54L) and the OMV server (HP N40L) for PCIe Intel Gigabit Network Cards which I was hoping would address the poor network performance. Unfortunately this has had a minimal effect, network transfer to the OMV has gone up only 10MB/s to around 40MB/s

I am at a loss to explain why this is the case, the only thing I can think of is that there is some kind of inherent Greyhole overhead that is effecting network performance, possibly the interaction between Greyhole and Samba could be causing the performance degradation.!?!?
As a next step I was going to setup an NFS share and attempt to transfer via NFS to see what the performance is like. Can anyone give me some tips as to how to connect to an NFS share on the OMV from windows.? I have done some digging and found a few guides but they are rather confusing. I should point out that I want to connect to the OMV from WHS2011 via NFS, NFS is already setup on WHS2011 and has been used to stream from there to an XBMC machine quite happily for a few years now so as I understand it all I need is the sequence for creating a networked drive on WHS2011 that connects to the NFS share on OMV.
I have already added 1 of the shared folders on OMV via the NFS tab although I am not 100% sure it's correct.

I realise that this is drifting off topic a touch but hopefully this will be able to help identify the problem.

To be honest I wouldn't be that bothered as 40MB/s isn't catastophic but this is to be used as a media server and I am initially wanting to transfer close to 4TB of data onto it across the network.. would rather it didn't take all week.!!!!!

One other thing I have just noticed.. when checking for those mounting options you mentioned earlier (tekkbebe) I noticed an item in the Greyhole documentation stating that :

Zitat

Including any drive mounted as /media/Something in your storage pool is usually a bad idea.

Those mounts are created by the gnome-automounter, which requires you to be logged in into X (Gnome) to become available. This will create issues with Greyhole, which expects drives to always be available, and will take action when some of them are missing.

I notice that in my fstab, the section added by OMV has mounted drives as /media/UUID.. see below. Is this a problem.?

# /etc/fstab: static file system information.
#
# Use 'blkid' to print the universally unique identifier for a
# device; this may be used with UUID= as a more robust way to name devices
# that works even if disks are added and removed. See fstab(5).
#
# <file system> <mount point>   <type>  <options>       <dump>  <pass>
proc            /proc           proc    defaults        0       0
# / was on /dev/sda1 during installation
UUID=71055c55-065b-49f6-9e7e-734c7b8708aa /               ext4    errors=remount-ro 0       1
# swap was on /dev/sda5 during installation
UUID=99716b2b-37ef-4635-9e94-4e7f0a2daa06 none            swap    sw              0       0
# /dev/sdb1       /media/usb0     auto    rw,user,noauto  0       0
UUID=11e09e30-8ba0-45cb-ba63-7bc9f1f48731       /media/usb0     auto    rw,user,noauto  0       0
tmpfs           /tmp            tmpfs   defaults        0       0
# >>> [openmediavault]
UUID=ce5d3c39-2ac7-4d8c-a041-5ea933bd073d /media/ce5d3c39-2ac7-4d8c-a041-5ea933bd073d ext4 defaults,noatime,data=writeback,nofail,acl,user_xattr,noexec,usrjquota=a$
UUID=46e2c9ca-6236-4900-84f3-60c34b206fae /media/46e2c9ca-6236-4900-84f3-60c34b206fae ext4 defaults,noatime,data=writeback,nofail,acl,user_xattr,noexec,usrjquota=a$
UUID=11e09e30-8ba0-45cb-ba63-7bc9f1f48731 /media/11e09e30-8ba0-45cb-ba63-7bc9f1f48731 ext4 defaults,noatime,data=writeback,nofail,acl,user_xattr,noexec,usrjquota=a$
UUID=676f2e24-6b7e-4b8a-9ef4-372959a4c1ca /media/676f2e24-6b7e-4b8a-9ef4-372959a4c1ca ext4 defaults,noatime,data=writeback,nofail,acl,user_xattr,noexec,usrjquota=a$
/media/46e2c9ca-6236-4900-84f3-60c34b206fae/Video/ /export/Video none bind 0 0
# <<< [openmediavault]

That's an idea of course.. Can't believe I didn't think of that..!!! :oops:

I am not actually using the file duplication feature on most of the files as they are purely for serving to an XBMC. I was using the drive pooling functionality primarily in this case in order to prevent having to manage the files across the drives..

Hopefully one last thing.
I have got the majority of data transferred to the OMV server now and ran the 'Files Check' on each of the shares. I have also got it to run 'Files Balance' once the check is complete.. My question is, how long should this take.? It has been running now for around 18 hours..? It is around 3TB of data so I don't expect it to take 5 minutes but is 18 hours right.? As far as I can see it is still doing the file check and hasn't even started the balancing yet.

Sorry for the slow response (son's birthday party today)...

I think there is definately something wrong here.. the greyhole log is still repeating the same message..

Mar 08 18:38:02 INFO fsck: Now working on task ID 23799: fsck /media/46e2c9ca-6236-4900-84f3-60c34b206fae/Video/
Mar 08 18:38:02 INFO fsck: Will execute all (364) pending fsck_file operations for /media/46e2c9ca-6236-4900-84f3-60c34b206fae/Video before running this fsck (task ID 23799).

The task id hasn't changed since last night..!?
It's still running the same right now, that's almost 24 hours now.

Ok, sorry but I'm not aware of how to do either of those things in order to give you some helpful log output.!?
As far as I am aware I haven't changed the default log output.

Thanks for that. Have changed the log output to DEBUG and restarted the server.. I did that because the log output did not change.
Having rebooted the server, the log output for greyhole is exactly the same as above...

Ahh ok, think I've figured out at least part of the problem and it's my own stupid fault.. While messing about with the Greyhole config I mistakenly deleted an SMB share before removing it from the Greyhole config. I thought I had cleared it up by removing the leftover bits if share data in the smb.conf and greyhole.conf files but it seems that it's still causing an issue..

Mar 08 20:54:15 WARN fsck_file:   Found a share (Video) with no path in /etc/samba/smb.conf, or missing it's num_copies[Video] config in /etc/greyhole.conf. Skipping.
Mar 08 20:54:15 DEBUG fsck_file:   Got 0 metadata files.
Mar 08 20:54:15 WARN fsck_file: Found a task on a share (Video) that disappeared from /etc/greyhole.conf. Skipping.
Mar 08 20:54:15 CRITICAL fsck_file: Tried to fsck a share that is missing from greyhole.conf. Quitting.

Any help in removing the evidence of my stupidity would be appreciated. The 'Video' SMB share no longer exists and does not show up in either the SMB config tab or the Greyhole one.