[HOWTO] Instal ZFS-Plugin & use ZFS on OMV

Hello,

checked the improvements.

1. importing by-id as a default works fine now:

1a. with omv3.0.13 and the openmediavault-zfs 3.0.1 plugin "zpool status" looks as follows after pressing the button "import" in the webui:

root@mediatank:~# zpool status
  pool: mediatank
 state: ONLINE
  scan: scrub repaired 0 in 5h42m with 0 errors on Mon Apr 25 17:52:00 2016
config:


        NAME        STATE     READ WRITE CKSUM
        mediatank   ONLINE       0     0     0
          raidz2-0  ONLINE       0     0     0
            sde     ONLINE       0     0     0
            sdi     ONLINE       0     0     0
            sdh     ONLINE       0     0     0
            sdd     ONLINE       0     0     0
            sdf     ONLINE       0     0     0
            sdg     ONLINE       0     0     0
            sdj     ONLINE       0     0     0
            sdc     ONLINE       0     0     0


errors: No known data errors

1b. with omv3.0.40 and the openmediavault-zfs 3.0.3 plugin "zpool status" looks as follows after pressing the button "import" in the webui:

root@omv:~# zpool status
  pool: mediatank
 state: ONLINE
  scan: scrub repaired 0 in 5h42m with 0 errors on Mon Apr 25 17:52:00 2016
config:


        NAME                                          STATE     READ WRITE CKSUM
        mediatank                                     ONLINE       0     0     0
          raidz2-0                                    ONLINE       0     0     0
            ata-WDC_WD40EFRX-68WT0N0_WD-WCC4E2YJY76J  ONLINE       0     0     0
            ata-WDC_WD40EFRX-68WT0N0_WD-WCC4E3NV2D2F  ONLINE       0     0     0
            ata-WDC_WD40EFRX-68WT0N0_WD-WCC4E5EC1TZ9  ONLINE       0     0     0
            ata-WDC_WD40EFRX-68WT0N0_WD-WCC4E6LA42K7  ONLINE       0     0     0
            ata-WDC_WD40EFRX-68WT0N0_WD-WCC4E6LA4ZJ8  ONLINE       0     0     0
            ata-WDC_WD40EFRX-68WT0N0_WD-WCC4E7HP68ZE  ONLINE       0     0     0
            ata-WDC_WD40EFRX-68WT0N0_WD-WCC4E7XKY0DJ  ONLINE       0     0     0
            ata-WDC_WD40EFRX-68WT0N0_WD-WCC4E7XKY1AN  ONLINE       0     0     0


errors: No known data errors

Great!

2. file systems are shown correct now:

My pool is configured as shown in the following screenshot:

2a. with omv3.0.13 and the openmediavault-zfs 3.0.1 plugin "File Systems" section in the webui looks as follows:

This view doesn't fit to the pool, but everthing I need worked as expected.

2b. with omv3.0.40 and the openmediavault-zfs 3.0.3 plugin "File Systems" section in the webui looks as follows:

WOW, with the new plugin the file systems view gets repaired.

@luxflow Great work, well done!

For the moment everything looks as expected.

This weekend I am going to scrub my file systems and report back here.

Greetings Hoppel

Here is my way to install kernel 4.6 and headers
Basic idea is to install packages from debian snapshot repo
(see http://snapshot.debian.org/)

vim /etc/apt/sources.list.d/backports.list
#deb http://httpredir.debian.org/debian jessie-backports main contrib non-free
deb http://snapshot.debian.org/archive/debian/20160801T044620Z/ jessie-backports main contrib non-free


#this command install packages for amd64 if you want pakcages for other arch, change it
apt-get -o Acquire::Check-Valid-Until=false update
apt-get install -t jessie-backports linux-headers-amd64 linux-image-amd64


#After install headers and images you can use original repo by uncomentting 
#deb http://httpredir.debian.org/debian jessie-backports main contrib non-free

P.S For changing mountpoint, no success .. hmm maybe should I request to volker to patch some point

Zitat von luxflow

Here is my way to install kernel 4.6 and headers

Ok, so we have two ways to install a newer kernel. Thanks for sharing that.

Tonight I scrubed my file systems. The scrub process worked fine, repaired 8k with 0 errors and was as fast as with openmediavault on bare metal. So the kvm performance is good. Anyway I am going to use the proxmox 4.4 kernel, because it has long term support.

Waht do you think about the repaired 8k? The last time i sometimes removed the two miniSAShd-cables from from the mainboard to setup the proxmox machine. I didn't want proxmox with it's own zfs capabilities to touch my pool. Maybe this was the trigger for that? I am going to observe my pool and do another scrub the next days. Are there any possibilities to analyze these errors?

root@omv:~# zpool status
  pool: mediatank
 state: ONLINE
status: One or more devices has experienced an unrecoverable error.  An
        attempt was made to correct the error.  Applications are unaffected.
action: Determine if the device needs to be replaced, and clear the errors
        using 'zpool clear' or replace the device with 'zpool replace'.
   see: http://zfsonlinux.org/msg/ZFS-8000-9P
  scan: scrub repaired 8K in 5h32m with 0 errors on Sun Oct  2 09:13:38 2016
config:


        NAME                                          STATE     READ WRITE CKSUM
        mediatank                                     ONLINE       0     0     0
          raidz2-0                                    ONLINE       0     0     0
            ata-WDC_WD40EFRX-68WT0N0_WD-WCC4E2YJY76J  ONLINE       0     0     0
            ata-WDC_WD40EFRX-68WT0N0_WD-WCC4E3NV2D2F  ONLINE       0     0     1
            ata-WDC_WD40EFRX-68WT0N0_WD-WCC4E5EC1TZ9  ONLINE       0     0     0
            ata-WDC_WD40EFRX-68WT0N0_WD-WCC4E6LA42K7  ONLINE       0     0     0
            ata-WDC_WD40EFRX-68WT0N0_WD-WCC4E6LA4ZJ8  ONLINE       0     0     1
            ata-WDC_WD40EFRX-68WT0N0_WD-WCC4E7HP68ZE  ONLINE       0     0     0
            ata-WDC_WD40EFRX-68WT0N0_WD-WCC4E7XKY0DJ  ONLINE       0     0     0
            ata-WDC_WD40EFRX-68WT0N0_WD-WCC4E7XKY1AN  ONLINE       0     0     0


errors: No known data errors

Greetings Hoppel

Zitat von hoppel118

Waht do you think about the repaired 8k? The last time i sometimes removed the two miniSAShd-cables from from the mainboard to setup the proxmox machine. I didn't want proxmox with it's own zfs capabilities to touch my pool. Maybe this was the trigger for that? I am going to observe my pool and do another scrub the next days. Are there any possibilities to analyze these errors?

http://www.freenas.org/blog/yes-you-can-virtualize-freenas/

Precautions

• If you are not using PCI passthrough (more on that below), then you must disable the scrub tasks in ZFS. The hardware can “lie” to ZFS so a scrub can do more damage than good, possibly even permanently destroying your zpool.

I don't know exactly since I dont know very well about virtualization
But maybe this is the reason I think..

Zitat von luxflow

I don't know exactly since I dont know very well about virtualizationBut maybe this is the reason I think..

Hello @luxflow

my sas-controller (LSI SAS3008) is passed through to the openmediavault kvm. So this should not be the problem.

Greetings Hoppel

Hello,

I see the following errors in the syslog, when I access a smb-share and my hdds have to wake up after a spindown:

I found the following closed issue, describing the same problem:

https://github.com/zfsonlinux/zfs/issues/3785

And I found this current issue with zfs 0.6.5.6:

https://github.com/zfsonlinux/zfs/issues/4638

There I described my error in detail. What do you think about this?

Greetings Hoppel

Why not grab additional hdd drive, clone omv image to that hdd and test it in barebone?

Steps to reproduce:
1. Put disks in vdev in standby (e.g. hdparm -y /dev/sd[b-i])
2. Attempt to access a filesystem in that pool
3. Observe errors

I followed above step in my environment
(I install zfs-dkms, spl-dkms from latest debian backports, and install kernel from snapshot since 4.7 don't works well with my computer)

root@openmediavault:/tank/media# dpkg -l | grep dkms
ii dkms 2.2.0.3-2 all Dynamic Kernel Module Support Framework
ii spl-dkms 0.6.5.8-2~bpo8+2 all Solaris Porting Layer kernel modules for Linux
ii zfs-dkms 0.6.5.8-1~bpo8+1 all OpenZFS filesystem kernel modules for Linux

root@openmediavault:/tank/media# uname -a
Linux openmediavault 4.6.0-0.bpo.1-amd64 #1 SMP Debian 4.6.3-1~bpo8+1 (2016-07-13) x86_64 GNU/Linux

In my environment, no error appear
I always spin down all drives after 30 minutes, and no issue yet
I also didn't experience this issue when zfs-dkms, spl-dkms is 0.6.5.7

Hello @luxflow,

thanks for answering and the approach.

Zitat von luxflow

Why not grab additional hdd drive, clone omv image to that hdd and test it in barebone?

Steps to reproduce:
1. Put disks in vdev in standby (e.g. hdparm -y /dev/sd[b-i])
2. Attempt to access a filesystem in that pool
3. Observe errors

I will try that if I have the time. I am not sure if this is really related to zfs, even if it seems so. Maybe it is related to the pci passthrough. I have an additional hdd here, but never cloned a kvm to an hdd.

For the moment I disabled the spindown and now I don't see the error messages. But for sure I want to spindown my disks in future.

Zitat von luxflow

I followed above step in my environment(I install zfs-dkms, spl-dkms from latest debian backports, and install kernel from snapshot since 4.7 don't works well with my computer)

root@openmediavault:/tank/media# dpkg -l | grep dkms
ii dkms 2.2.0.3-2 all Dynamic Kernel Module Support Framework
ii spl-dkms 0.6.5.8-2~bpo8+2 all Solaris Porting Layer kernel modules for Linux
ii zfs-dkms 0.6.5.8-1~bpo8+1 all OpenZFS filesystem kernel modules for Linux

root@openmediavault:/tank/media# uname -a
Linux openmediavault 4.6.0-0.bpo.1-amd64 #1 SMP Debian 4.6.3-1~bpo8+1 (2016-07-13) x86_64 GNU/Linux

In my environment, no error appear
I always spin down all drives after 30 minutes, and no issue yet
I also didn't experience this issue when zfs-dkms, spl-dkms is 0.6.5.7

Alles anzeigen

OK, I will check the snapshot kernel the three packages you mentioned above.

Do you use any kind of virtaulization? Which "spindown mode" (Advanced Power Management) do you use? My hdds were configured the following way:

Thanks and greetings Hoppel

by the way... I found some other no priority things:

Is it somehow possible to get back the default values in the properties? It would be nice.

The file systems view doesn't show the right values under "total". It looks like a mixup.

If I have a look at the pool. It looks fine:

No priority things at all...

Greetings Hoppel

Zitat von hoppel118

Do you use any kind of virtaulization? Which "spindown mode" (Advanced Power Management) do you use? My hdds were configured the following way:

I'm using omv on baremetal (no virtualization),
spindown mode APM,ACM = disable, only spindown time is set to 30 minutes
hdds are conntected to motherboard sata ports

I will consider UI improve in `edit properties`
and also thanks for reporting file systems view bug
Maybe fixed future release, but since it is not critical, I cannot confirm when to release

Zitat von luxflow

Why not grab additional hdd drive, clone omv image to that hdd and test it in barebone?

Steps to reproduce:
1. Put disks in vdev in standby (e.g. hdparm -y /dev/sd[b-i])
2. Attempt to access a filesystem in that pool
3. Observe errors

OK, I checked it with my "old" bare metal setup (omv 3.0.13 and omv-zfs from that stage of development). When the disks get into standby and I try to access a file system of my pool, I see the same messages in the log file as in the kvm. So this issue is not related to my virtualization environment. I don't know why I didn't recognize the errors in the bare metal setup. Maybe I didn't want see them.

OK, this is offtopic here.

Greetings Hoppel

omv-zfs is uploaded in repo
==========

3.0.4 released

what's new?

• Refactor code
  it is not important for end-user
• Change dependencies (4.7 kernel is now supported)
  if you're upgrading from old openmediavault-zfs, shuold have to remove previous package following command
  
  Code

  apt-get autoremove debian-zfs libnvpair1 libuutil1 libzfs2 libzpool2 zfsutils spl-dkms zfs-dkms openmediavault-zfs

• Add mountpoint properties for filesystem
  Do not change mountpoint which has omv share
• Fix rename filesystem
• Fix Filesystem size information ( hoppel118 reported this issue)

As soon as @ryecoaaron upload omv-omvextrasorg 3.4 and omv-zfs 3.0.4 to bintray
It can be downloaded

Just a quick question for my understanding here:

If I install erasmus and omv-ZFS, which Kernel will I receive and ZOL version as default?

Backports jessie is Kernel 4.7, and ZOL version is 0.6.5.8-1.

will this work?

Zitat von bug11

If I install erasmus and omv-ZFS, which Kernel will I receive and ZOL version as default?

omv-zfs doesn't depend on kernel but depends on dkms-zfs
dkms is kind of a automatic building system, so
Whenever you install kernel header (linux-headers-amd64) or install new dkms pakcage (for example dkms-zfs)
It build all dkms module(including dkms-zfs) for kernel headers

For example if you have a header for backports kernel (you can get with below command)
apt-get install -t jessie-backports linux-headers-amd64
You will get dkms for backports kernel (currently 4.7)

You can freely choose your kernel, anytime you want

Alright, let me rewrite my question:

will the default backports 4.7 kernel and ZFS in OMV work out of the box or do I need to rekompile etc?.

work out of box since it is recompiled automatically, you don't need to recompile,

I just tried to upgrade OpenMediaVault 3 + ZFS. It failed with errors, so I started over by reinstalling from openmediavault_3.0.36-amd64.iso and it failed again with the same errors:

- updated everything.
- uploaded openmediavault-omvextrasorg_latest_all3.deb
- installed omv-extras under "plugins"
- enabled OMV-Extras -> ZFS + ZFS Testing
- Update everything again
- then installed openmediavault-zfs 3.0.4 under plugins which threw the same errors when trying to upgrade. Here is the part that shows what is going wrong:

....
Loading new spl-0.6.5.7 DKMS files...
First Installation: checking all kernels...
Building only for 3.16.0-4-amd64
Building initial module for 3.16.0-4-amd64
configure: error: in `/var/lib/dkms/spl/0.6.5.7/build':
configure: error: C compiler cannot create executables
See `config.log' for more details
Error! Bad return status for module build on kernel: 3.16.0-4-amd64 (x86_64)
Consult /var/lib/dkms/spl/0.6.5.7/build/make.log for more information.
Processing triggers for libc-bin (2.19-18+deb8u6) ...
Selecting previously unselected package zfs-dkms.
(Reading database ... (Reading database ... 5%(Reading database ... 10%(Reading database ... 15%(Reading database ... 20%(Reading database ... 25%(Reading database ... 30%(Reading database ... 35%(Reading database ... 40%(Reading database ... 45%(Reading database ... 50%(Reading database ... 55%(Reading database ... 60%(Reading database ... 65%(Reading database ... 70%(Reading database ... 75%(Reading database ... 80%(Reading database ... 85%(Reading database ... 90%(Reading database ... 95%(Reading database ... 100%(Reading database ... 44578 files and directories currently installed.)
Preparing to unpack .../zfs-dkms_0.6.5.7-8-jessie_all.deb ...
Unpacking zfs-dkms (0.6.5.7-8-jessie) ...
Selecting previously unselected package libuutil1linux.
Preparing to unpack .../libuutil1linux_0.6.5.8-1~bpo8+1_amd64.deb ...
Unpacking libuutil1linux (0.6.5.8-1~bpo8+1) ...
dpkg: error processing archive /var/cache/apt/archives/libuutil1linux_0.6.5.8-1~bpo8+1_amd64.deb (--unpack):
 trying to overwrite '/lib/libuutil.so.1.0.1', which is also in package libuutil1 0.6.5.7-8-jessie
Selecting previously unselected package libnvpair1linux.
Preparing to unpack .../libnvpair1linux_0.6.5.8-1~bpo8+1_amd64.deb ...
Unpacking libnvpair1linux (0.6.5.8-1~bpo8+1) ...
dpkg: error processing archive /var/cache/apt/archives/libnvpair1linux_0.6.5.8-1~bpo8+1_amd64.deb (--unpack):
 trying to overwrite '/lib/libnvpair.so.1.0.1', which is also in package libnvpair1 0.6.5.7-8-jessie
Selecting previously unselected package libzpool2linux.
Preparing to unpack .../libzpool2linux_0.6.5.8-1~bpo8+1_amd64.deb ...
Unpacking libzpool2linux (0.6.5.8-1~bpo8+1) ...
dpkg: error processing archive /var/cache/apt/archives/libzpool2linux_0.6.5.8-1~bpo8+1_amd64.deb (--unpack):
 trying to overwrite '/lib/libzpool.so.2.0.0', which is also in package libzpool2 0.6.5.7-8-jessie
dpkg-deb: error: subprocess paste was killed by signal (Broken pipe)
Selecting previously unselected package libzfs2linux.
Preparing to unpack .../libzfs2linux_0.6.5.8-1~bpo8+1_amd64.deb ...
Unpacking libzfs2linux (0.6.5.8-1~bpo8+1) ...
dpkg: error processing archive /var/cache/apt/archives/libzfs2linux_0.6.5.8-1~bpo8+1_amd64.deb (--unpack):
 trying to overwrite '/lib/libzfs_core.so.1.0.0', which is also in package libzfs2 0.6.5.7-8-jessie
dpkg-deb: error: subprocess paste was killed by signal (Broken pipe)
Selecting previously unselected package zfsutils-linux.
Preparing to unpack .../zfsutils-linux_0.6.5.8-1~bpo8+1_amd64.deb ...
Unpacking zfsutils-linux (0.6.5.8-1~bpo8+1) ...
Selecting previously unselected package openmediavault-zfs.
Preparing to unpack .../openmediavault-zfs_3.0.4_amd64.deb ...
Unpacking openmediavault-zfs (3.0.4) ...
Processing triggers for man-db (2.7.0.2-5) ...
Processing triggers for openmediavault (3.0.50) ...
Restarting engine daemon ...
Errors were encountered while processing:
 /var/cache/apt/archives/libuutil1linux_0.6.5.8-1~bpo8+1_amd64.deb
 /var/cache/apt/archives/libnvpair1linux_0.6.5.8-1~bpo8+1_amd64.deb
 /var/cache/apt/archives/libzpool2linux_0.6.5.8-1~bpo8+1_amd64.deb
 /var/cache/apt/archives/libzfs2linux_0.6.5.8-1~bpo8+1_amd64.deb
>>> *************** Error ***************
<<< *************************************
>>> *************** Error ***************
<<< *************************************
.......

Here is a part from /var/lib/dkms/spl/0.6.5.7/build/config.log where I saw errors:

...
Thread model: posix
gcc version 4.9.2 (Debian 4.9.2-10) 
configure:3421: $? = 0
configure:3410: gcc -V >&5
gcc: error: unrecognized command line option '-V'
gcc: fatal error: no input files
compilation terminated.
configure:3421: $? = 4
configure:3410: gcc -qversion >&5
gcc: error: unrecognized command line option '-qversion'
gcc: fatal error: no input files
compilation terminated.
configure:3421: $? = 4
configure:3441: checking whether the C compiler works
configure:3463: gcc    conftest.c  >&5
/usr/bin/ld: cannot find crt1.o: No such file or directory
/usr/bin/ld: cannot find crti.o: No such file or directory
/usr/bin/ld: cannot find -lc
/usr/bin/ld: cannot find crtn.o: No such file or directory
collect2: error: ld returned 1 exit status
configure:3467: $? = 1
configure:3505: result: no
configure: failed program was:
| /* confdefs.h */
| #define PACKAGE_NAME "spl"
| #define PACKAGE_TARNAME "spl"
| #define PACKAGE_VERSION "0.6.5.7"
| #define PACKAGE_STRING "spl 0.6.5.7"
| #define PACKAGE_BUGREPORT ""
| #define PACKAGE_URL ""
| #define SPL_META_NAME "spl"
| #define SPL_META_VERSION "0.6.5.7"
| #define SPL_META_RELEASE "5-jessie"
| #define SPL_META_LICENSE "GPL"
| #define SPL_META_ALIAS "spl-0.6.5.7-5-jessie"
| #define SPL_META_AUTHOR "OpenZFS on Linux"
| #define PACKAGE "spl"
| #define VERSION "0.6.5.7"
| /* end confdefs.h.  */
| 
| int
| main ()
| {
| 
|   ;
|   return 0;
| }
configure:3510: error: in `/var/lib/dkms/spl/0.6.5.7/build':
configure:3512: error: C compiler cannot create executables
See `config.log' for more details
...

Here are my versions:

root@nas4:/var/log# dpkg -l |grep zfs
ii  libzfs2                        0.6.5.7-8-jessie          amd64        Native ZFS filesystem library for Linux
iU  openmediavault-zfs             3.0.4                     amd64        OpenMediaVault plugin for ZFS
iU  zfs-dkms                       0.6.5.7-8-jessie          all          Native ZFS filesystem kernel modules for Linux
iU  zfsutils-linux                 0.6.5.8-1~bpo8+1          amd64        command-line tools to manage OpenZFS filesystems
root@nas4:/var/log# dpkg -l |grep openm
ii  openmediavault                 3.0.50                    all          Open network attached storage solution
ii  openmediavault-keyring         1.0                       all          GnuPG archive keys of the OpenMediaVault archive
ii  openmediavault-omvextrasorg    3.4.4                     all          OMV-Extras.org Package Repositories for OpenMediaVault
iU  openmediavault-zfs             3.0.4                     amd64        OpenMediaVault plugin for ZFS
root@nas4:/var/log# zpool status
The ZFS modules are not loaded.
Try running '/sbin/modprobe zfs' as root to load them.
root@nas4:~# /sbin/modprobe zfs
modprobe: FATAL: Module zfs not found.

How can I fix this?

Thanks!

@roccur did you try this command?

apt-get autoremove debian-zfs libnvpair1 libuutil1 libzfs2 libzpool2 zfsutils spl-dkms zfs-dkms openmediavault-zfs

before installing new omv-zfs?

It seems you use old version of zfs (0.6.5.7)

After completely remove all zfs related things (include plugins) with above command, try install again

Zitat von luxflow

@roccur did you try this command?

Bash

apt-get autoremove debian-zfs libnvpair1 libuutil1 libzfs2 libzpool2 zfsutils spl-dkms zfs-dkms openmediavault-zfs

before installing new omv-zfs?

Thanks @luxflow , it worked! I think my mistake was previously to select both "ZFS" and "ZFS Testing" under "OMV-Extras". This time I only selected "ZFS Testing".