[HOWTO] Instal ZFS-Plugin & use ZFS on OMV

raulfg3 · Feb 15th 2015

2016.11.09 actualization: please read this post first before to install: Error trying to install ZFS-Plugin

2018.09.08 actualization: Good video of HOW to Install from @Technodadlife.

2023.04.01 actualization: from a @ShamusLit :RE: How do I get ZFS in OMV?

2023.09.01 actualization: Always install kernell-plugin & PROXMOX KERNEL, prior to install ZFS-plugin.

Well This post is intendent to show how to start to work with ZFS & ZFS Plugin in OMV, Is usefull for beginers & Users that come from BSD ( FreeNAs & Nas4Free) World to understand certain diferrences between ZFS and ZFS on Linux.

Some usefull links:

Issues in ZFS-Plugins: https://github.com/OpenMediaVa…openmediavault-zfs/issues
Name conventions when creating pools: http://zfsonlinux.org/faq.html…uldIUseWhenCreatingMyPool
ZFS-HandBook: http://www.allanjude.com/zfs_handbook/zfs-zpool.html
Info About Performance: http://open-zfs.org/wiki/Performance_tuning
Article about ZED: http://louwrentius.com/category/zfs.html

Requeriments:

To Use ZFS you need at least OMV 1.12 & OMV_Extras 1.10

If you have an updated OMV, go to step 2

1 -
If start from actual OMV.ISO: http://www.openmediavault.org/download.html

your first screen is:

So you nee to update to letas version and install all updates:

Finally you can have OMV 1.12 or up:

Now you nee to install OMV-Extras: OMV-Extras.org Plugin

once done you have OMV-Extras 1.10 or up

2 -

Now you are ready to instal ZFS-Plugin.

Select ZFS Testing Repo:

Update system:

Install ZFS (0.6.3.6 when writting this guide):

raulfg3 · Feb 15th 2015

If all works fine & expected, you must see your ZFS icon:

Now you have 2 possible paths,

1- Import your existing Pool ( use option in ZFS menu) ; remember that latest FreeNAS pools (9.3 and up) can't be imported due a Feature Flag not still implemented on ZFS for Linux (9.2 and down can be imported without problem), So please revise what feature Flags have your pool beforo to try to import on OMV
2 - Create a new Pool.

If you want to create a pool your disk must be clean, to be sure use wipe from menu:

Now you are ready to create pools, some examples:

Now a more complicated pool: a 4x3TB vdev + other 4+3TB vdev in one big pool.

You need to create in 2 steps

First you need to create a pool using only 4 disk (first vdev), in my case I select to use ashift=12 = 4K disk

Once created, use Grow icon to add 4 more disk in raidZ1

This second vdev is added to first vdev so finally you have the pool taht you want , and in future, you can grow RPoll in the same way

Now time to create some filesystems to share it:

raulfg3 · Feb 15th 2015

Import is really easy, only go to button import and import your previsoly exported/created pool.

DISCLAIMER: Actual ZFS version on linux DO not import latest FreeNAS pool because FreeNAS use some featured Flags NOT supported by actual ZFS version of ZFSonLinux.

I test a scrub too, previously I corrupt some files:

Code

Pool status (zpool status):


  pool: Rpool
 state: ONLINE
status: One or more devices has experienced an error resulting in data
    corruption.  Applications may be affected.
action: Restore the file in question if possible.  Otherwise restore the
    entire pool from backup.
   see: http://zfsonlinux.org/msg/ZFS-8000-8A
  scan: scrub repaired 326M in 4h11m with 0 errors on Fri Feb 20 12:02:53 2015
config:


    NAME                                  STATE     READ WRITE CKSUM
    Rpool                                 ONLINE       0     0     6
      raidz1-0                            ONLINE       0     0    68
        ata-TOSHIBA_DT01ACA300_Z2P58ZSAS  ONLINE       0     0   116
        ata-TOSHIBA_DT01ACA300_Z2P592PAS  ONLINE       0     0   106
        ata-TOSHIBA_DT01ACA300_Z2P4LS0GS  ONLINE       0     0   134
        ata-TOSHIBA_DT01ACA300_Z2P5909AS  ONLINE       0     0   119
      raidz1-1                            ONLINE       0     0    59
        ata-TOSHIBA_DT01ACA300_63NZKLSKS  ONLINE       0     0   148
        ata-TOSHIBA_DT01ACA300_63NZ4Z9GS  ONLINE       0     0   107
        ata-TOSHIBA_DT01ACA300_63QZR0TGS  ONLINE       0     0   127
        ata-TOSHIBA_DT01ACA300_63NZK2UGS  ONLINE       0     0    99


errors: 6 data errors, use '-v' for a list


Pool details (zpool get all):

Display More

A lot of CPU & RAM is used when scrub is initiated:

raulfg3 · Feb 15th 2015

Some captures of file transfer (Videos , avi, and mkv):

raulfg3 · Feb 15th 2015

documentation, ZFS Evil tunning guide: http://www.solarisinternals.co…php/ZFS_Evil_Tuning_Guide

Some tunning that you can use in function of your RAM

Please read: http://wiki.gentoo.org/wiki/ZFS
http://arstechnica.com/informa…-gen-filesystem-on-linux/

in my case I use:

Code

echo "options zfs zfs_arc_max=6312427520" >> /etc/modprobe.d/zfs.conf
echo "options zfs zfs_arc_min=6312427520" >> /etc/modprobe.d/zfs.conf

and then reboot NAS so next time use new zfs_arc_max & min values

other good option is for 4GB:

Code

echo "options zfs zfs_arc_max=3221225472" >> /etc/modprobe.d/zfs.conf
echo "options zfs zfs_arc_min=3221225472" >> /etc/modprobe.d/zfs.conf

I use values that experience recommended on BSD= Nas4Free if you search for a good tunning tool called ZFSKerntune and analize, you can see that only fixes zfs_arc_max and zfs_arc_min to desirables values.

Matrix of desirables values is:

Code

"X32_1024MB" => array("kmem" =>  "512M", "arcmin" => "128M", "arcmax" => "128M"),
        "X32_1536MB" => array("kmem" => "1024M", "arcmin" => "256M", "arcmax" => "256M"),
        "X32_2048MB" => array("kmem" => "1400M", "arcmin" => "400M", "arcmax" => "400M"),
        "X64_2GB"    => array("kmem" => "1536M", "arcmin" => "512M", "arcmax" => "512M"),
        "X64_3GB"    => array("kmem" => "2048M", "arcmin" => "1024M", "arcmax" => "1024M"),
        "X64_4GB"    => array("kmem" => "2560M", "arcmin" => "1536M", "arcmax" => "1536M"),
        "X64_6GB"    => array("kmem" => "4608M", "arcmin" => "3072M", "arcmax" => "3072M"),
        "X64_8GB"    => array("kmem" => "6656M", "arcmin" => "5120M", "arcmax" => "5120M"),
        "X64_12GB"   => array("kmem" => "10752M", "arcmin" => "9216M", "arcmax" => "9216M"),
        "X64_16GB"   => array("kmem" => "14336M", "arcmin" => "12288M", "arcmax" => "12288M"),
        "X64_24GB"   => array("kmem" => "22528M", "arcmin" => "20480M", "arcmax" => "20480M"),
        "X64_32GB"   => array("kmem" => "30720M", "arcmin" => "28672M", "arcmax" => "28672M"),
        "X64_48GB"   => array("kmem" => "47104M", "arcmin" => "45056M", "arcmax" => "45056M"),
        "X64_64GB"   => array("kmem" => "62464M", "arcmin" => "59392M", "arcmax" => "59392M"),
        "X64_96GB"   => array("kmem" => "95232M", "arcmin" => "92160M", "arcmax" => "92160M"),
        "X64_128GB"   => array("kmem" => "128000M", "arcmin" => "124928M", "arcmax" => "124928M"),
        "X64_192GB"   => array("kmem" => "193536M", "arcmin" => "190464M", "arcmax" => "190464M"),
        "X64_256GB"   => array("kmem" => "259072M", "arcmin" => "256000M", "arcmax" => "256000M")

Display More

nicjo814 · Feb 16th 2015

Really good info you have provided here! I think this will help out everyone who is looking at implementing ZFS on their OMV system.

betz0r · Feb 20th 2015

I followed your guide, i have most recent versions installed: omv 1.12 and omv-extras 1.11

When i try to setup omv-extras my settings windows looks very different than yours:

I can't find ZFS plugin anywhere in the plugin list, too. i installed definietly omv-extras 1.11

davidh2k · Feb 20th 2015

ARM?

Greetings
David

betz0r · Feb 20th 2015

yes, bananapi arm7

ryecoaaron · Feb 20th 2015

zfs is amd64 only. arm is nowhere near powerful enough for zfs.

betz0r · Feb 20th 2015

i have ZFS running on my bananapi quite well (dulacore 1ghz) but the omv plugin is missing...

ryecoaaron · Feb 20th 2015

I have lots of arm boards and used to have a bananapi. I assume you have it running with fuse? I would hate to see dedup try and run on an arm board.

The repo we get the zfs debian packages from is amd64 only. So, that will be the only supported arch for the plugin. The plugin code is here if you want to experiment with it on arm.

ellnic · Feb 25th 2015

Nice. Got this up and running and it's working well with 3 x 4TB Seagate's in a RAID-Z1. So far it's stable, but I keep an offline backup as well.

Thanks

ellnic · Feb 25th 2015

This is interesting

On a reboot my ZFS pool isn't mounted. The pool isn't listed in the ZFS plugin panel... I try to reimport and I get:

Code

Error
 #0:
exception 'OMVModuleZFSException' with message 'cannot create 'Media': 
missing dataset name' in /usr/share/omvzfs/Dataset.php:353
Stack trace:
#0 /usr/share/omvzfs/Dataset.php(191): OMVModuleZFSDataset->exec('zfs
 create -p "...', Array, 1)
#1 /usr/share/omvzfs/Dataset.php(77): OMVModuleZFSDataset->create()
#2 /usr/share/omvzfs/OMVStorageZvol.php(342): 
OMVModuleZFSDataset->__construct('Media')
#3 /usr/share/openmediavault/engined/rpc/filesystemmgmt.inc(231): 
OMVFilesystemZFS->isMounted()
#4 [internal function]: 
OMVRpcServiceFileSystemMgmt->enumerateMountedFilesystems(Array, 
Array)
#5 /usr/share/php/openmediavault/rpcservice.inc(125): 
call_user_func_array(Array, Array)
#6 /usr/share/php/openmediavault/rpc.inc(79): 
OMVRpcServiceAbstract->callMethod('enumerateMounte...', Array, Array)
#7 /usr/sbin/omv-engined(500): OMVRpc::exec('FileSystemMgmt', 
'enumerateMounte...', Array, Array, 1)
#8 {main}

Display More

Hrm... 'zpool status Media' gives me:

Code

pool: Media
 state: UNAVAIL
status: One or more devices could not be used because the label is missing 
    or invalid.  There are insufficient replicas for the pool to continue
    functioning.
action: Destroy and re-create the pool from
    a backup source.
   see: http://zfsonlinux.org/msg/ZFS-8000-5E
  scan: none requested
config:


    NAME        STATE     READ WRITE CKSUM
    Media       UNAVAIL      0     0     0  insufficient replicas
      raidz1-0  UNAVAIL      0     0     0  insufficient replicas
        sdd     ONLINE       0     0     0
        sde     UNAVAIL      0     0     0
        sdf     UNAVAIL      0     0     0

Display More

It appears that this is because the drives were originally: sdd, sde, and sdf. They now appear to have been reassigned sdb, sdc, and and sdd.

This could be a failing on my part, I expect I should have selected 'By ID' not 'By Path' but the "Specifies which device alias should be used. Don't change unless needed." made me leave it alone... surely it should always be by ID and this should be the default? Anyway, fixed with:

Code

zpool export Media

then imported the pool again.

By importing the pool again, it appears that the drives have now been imported by ID:

Code

/mnt/Media # zpool status Media                                                                                    
  pool: Media
 state: ONLINE
  scan: none requested
config:


    NAME                                 STATE     READ WRITE CKSUM
    Media                                ONLINE       0     0     0
      raidz1-0                           ONLINE       0     0     0
        ata-ST4000DM000-1F2168_XXXXXXX  ONLINE       0     0     0
        ata-ST4000DM000-1F2168_XXXXXXX  ONLINE       0     0     0
        ata-ST4000DM000-1F2168_XXXXXXX  ONLINE       0     0     0

Display More

Thus preventing this from occuring again.

Is it possible to have the default option for 'Device alias' changed to 'By ID'? - especially if the user is warned not to touch it.

alex · Mar 10th 2015

DISCLAIMER: Actual ZFS version on linux DO not import latest FreeNAS pool because FreeNAS use some featured Flags NOT supported by actual ZFS version of ZFSonLinux.

Is there anyway to solve this problem?

ellnic · Mar 10th 2015

No. Others will correct me if I am wrong, but I believe the pool version for FreeBSD 10.1, on which FreeNAS is based, is version 5000 whereas Debian Wheezy is on version 28.

I am not sure why the version increase is so large, but 5000 seems to come after 28 from what I can tell. More info in the wiki: http://en.m.wikipedia.org/wiki/ZFS#Platforms

raulfg3 · Mar 10th 2015

really , both are version 5000 to be more complex, only diference between latest FreeNAS pools and the rest is the use of feature flags:

Quote

Actual ZFS version on linux DO not import latest FreeNAS pool because FreeNAS use some featured Flags NOT supported by actual ZFS version of ZFSonLinux.

ellnic · Mar 10th 2015

Ah, I stand corrected.

Drible79 · Mar 23rd 2015

Hi,

Is ZFS considered stable or still in development on omv? Also, does it require a cron job for a monthly scrub or is that part of the plugin?

thanks

OMV 1.14 - MSI ITX - 8GB RAM - 16GB SSD Boot - 3x2tb in RAID 5

ryecoaaron · Mar 23rd 2015

In my opinion, zfs is stable but I'm not using it for production. The plugin may also be missing features. The plugin does not add a cron job for scrubbing.

[HOWTO] Instal ZFS-Plugin & use ZFS on OMV

Participate now!

Tags