[HOWTO] Instal ZFS-Plugin & use ZFS on OMV

    • OMV 1.0
    • ryecoaaron wrote:

      Try the proxmox kernel from the omv-extras kernel tab. No compiling needed.

      Brainfuck Source Code

      1. May 16 16:49:51 smallnas kernel: [ 8.726498] ZFS: Loaded module v0.7.13-pve1~bpo1, ZFS pool version 5000, ZFS filesystem version 5
      2. May 16 16:49:51 smallnas kernel: [ 9.468956] softdog: initialized. soft_noboot=0 soft_margin=60 sec soft_panic=0 (nowayout=0)
      3. May 16 16:49:53 smallnas kernel: [ 11.346368] e1000e: enp6s0f0 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: None
      4. May 16 16:52:04 smallnas kernel: [ 141.640072] ------------[ cut here ]------------
      5. May 16 16:52:04 smallnas kernel: [ 141.640235] invalid opcode: 0000 [#1] SMP PTI
      6. May 16 16:52:04 smallnas kernel: [ 141.650112] Modules linked in: softdog gpio_ich coretemp ppdev kvm radeon irqbypass ttm snd_pcm drm_kms_helper snd_timer input_leds snd lpc_ich soundcore serio_raw pcspkr drm zfs(PO) i2c_algo_bit fb_sys_fops syscopyarea sysfillrect sysimgblt zunicode(PO) zavl(PO) icp(PO) i5000_edac i5k_amb shpchp parport_pc parport mac_hid zcommon(PO) znvpair(PO) spl(O) sunrpc ip_tables x_tables autofs4 btrfs zstd_compress raid10 raid456 async_raid6_recov async_memcpy async_pq async_xor async_tx xor raid6_pq libcrc32c raid1 raid0 multipath linear ahci i2c_i801 libahci pata_acpi e1000e(O) ptp pps_core sata_mv
      7. May 16 16:52:04 smallnas kernel: [ 141.693352] CPU: 6 PID: 209 Comm: scsi_eh_7 Tainted: P O 4.15.18-14-pve #1
      8. May 16 16:52:04 smallnas kernel: [ 141.704517] Hardware name: Supermicro X7DB8/X7DB8, BIOS 2.1a 12/20/2008
      9. May 16 16:52:04 smallnas kernel: [ 141.715761] RIP: 0010:mv_qc_prep+0x1f3/0x210 [sata_mv]
      10. May 16 16:52:04 smallnas kernel: [ 141.727042] RSP: 0018:ffffa08943a6f9d0 EFLAGS: 00010006
      11. May 16 16:52:04 smallnas kernel: [ 141.738226] RAX: ffff899302bea6c0 RBX: ffff899302069d48 RCX: 0000000000000047
      12. May 16 16:52:04 smallnas kernel: [ 141.749477] RDX: ffff899302bea6ca RSI: ffff899302bea400 RDI: ffff899302069d48
      13. May 16 16:52:04 smallnas kernel: [ 141.760668] RBP: ffffa08943a6f9f0 R08: 0000000000000008 R09: 0000000000000007
      14. May 16 16:52:04 smallnas kernel: [ 141.771809] R10: ffffa08943a6fac8 R11: ffff89930ec036e8 R12: ffff899302068000
      15. May 16 16:52:04 smallnas kernel: [ 141.782922] R13: ffff899302069e80 R14: 0000000000000001 R15: ffff899302068000
      16. May 16 16:52:04 smallnas kernel: [ 141.794154] FS: 0000000000000000(0000) GS:ffff89932fd80000(0000) knlGS:0000000000000000
      17. May 16 16:52:04 smallnas kernel: [ 141.805423] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
      18. May 16 16:52:04 smallnas kernel: [ 141.816646] CR2: 000055fe34b2a454 CR3: 000000080ab74000 CR4: 00000000000006e0
      19. May 16 16:52:04 smallnas kernel: [ 141.827995] Call Trace:
      20. May 16 16:52:04 smallnas kernel: [ 141.839369] ? ata_qc_issue+0x17a/0x230
      21. May 16 16:52:04 smallnas kernel: [ 141.850818] ata_exec_internal_sg+0x33f/0x650
      22. May 16 16:52:04 smallnas kernel: [ 141.862371] ata_exec_internal+0x64/0xa0
      23. May 16 16:52:04 smallnas kernel: [ 141.873927] ? ata_exec_internal+0x64/0xa0
      24. May 16 16:52:04 smallnas kernel: [ 141.885529] ata_read_log_page.part.35+0x18b/0x1c0
      25. May 16 16:52:04 smallnas kernel: [ 141.897219] ata_read_log_page+0x22/0x30
      26. May 16 16:52:04 smallnas kernel: [ 141.908786] ata_eh_analyze_ncq_error+0xe0/0x2f0
      27. May 16 16:52:04 smallnas kernel: [ 141.920186] ata_eh_link_autopsy+0x9c/0xaf0
      28. May 16 16:52:04 smallnas kernel: [ 141.931352] ? ttwu_do_wakeup+0x1e/0x140
      29. May 16 16:52:04 smallnas kernel: [ 141.942359] ? ttwu_do_activate+0x77/0x80
      30. May 16 16:52:04 smallnas kernel: [ 141.953226] ata_eh_autopsy+0x2b/0xf0
      31. May 16 16:52:04 smallnas kernel: [ 141.963986] sata_pmp_error_handler+0x37/0xa20
      32. May 16 16:52:04 smallnas kernel: [ 141.974746] ? flush_work+0x5b/0x1e0
      33. May 16 16:52:04 smallnas kernel: [ 141.985362] ? __switch_to_asm+0x40/0x70
      34. May 16 16:52:04 smallnas kernel: [ 141.995873] ? __switch_to_asm+0x34/0x70
      35. May 16 16:52:04 smallnas kernel: [ 142.006331] ? __switch_to_asm+0x40/0x70
      36. May 16 16:52:04 smallnas kernel: [ 142.016731] ? __switch_to_asm+0x34/0x70
      37. May 16 16:52:04 smallnas kernel: [ 142.027077] ? lock_timer_base+0x7a/0xa0
      38. May 16 16:52:04 smallnas kernel: [ 142.037348] mv_pmp_error_handler+0x8e/0xa0 [sata_mv]
      39. May 16 16:52:04 smallnas kernel: [ 142.047606] ata_scsi_port_error_handler+0x474/0x7d0
      40. May 16 16:52:04 smallnas kernel: [ 142.057839] ? ata_scsi_cmd_error_handler+0xec/0x140
      41. May 16 16:52:04 smallnas kernel: [ 142.068087] ata_scsi_error+0x95/0xd0
      42. May 16 16:52:04 smallnas kernel: [ 142.078321] scsi_error_handler+0xda/0x5c0
      43. May 16 16:52:04 smallnas kernel: [ 142.088557] kthread+0x105/0x140
      44. May 16 16:52:04 smallnas kernel: [ 142.098715] ? scsi_eh_get_sense+0x240/0x240
      45. May 16 16:52:04 smallnas kernel: [ 142.108855] ? kthread_create_worker_on_cpu+0x70/0x70
      46. May 16 16:52:04 smallnas kernel: [ 142.118994] ret_from_fork+0x35/0x40
      47. May 16 16:52:04 smallnas kernel: [ 142.129114] Code: ca 0f 85 4f ff ff ff 0f b6 4f 2b 48 8d 50 0c 80 cd 12 66 89 48 0a e9 47 ff ff ff f3 c3 c6 47 35 30 c3 0f 0b e9 a0 fe ff ff f3 c3 <0f> 0b 48 83 e2 df 48 89 57 20 e9 65 fe ff ff 0f 1f 40 00 66 2e
      48. May 16 16:52:04 smallnas kernel: [ 142.162216] ---[ end trace fdec5b72bba524b0 ]---
      Display All
    • ryecoaaron wrote:

      What kind of hardware? Let me guess... bleeding edge desktop hardware?

      Source Code

      1. lshw -short
      2. H/W path Device Class Description
      3. =====================================================
      4. system X7DB8
      5. /0 bus X7DB8
      6. /0/0 memory 108KiB BIOS
      7. /0/4 processor Intel(R) Xeon(R) CPU L5420 @ 2.50GHz
      8. /0/4/6 memory 16KiB L1 cache
      9. /0/4/7 memory 12MiB L2 cache
      10. /0/5 processor Intel(R) Xeon(R) CPU L5420 @ 2.50GHz
      11. /0/5/8 memory 16KiB L1 cache
      12. /0/5/9 memory 12MiB L2 cache
      13. /0/17 memory 32GiB System Memory
      14. /0/17/0 memory 4GiB DIMM DDR2 Synchronous 667 MHz (1.5 ns)
      15. /0/17/1 memory 4GiB DIMM DDR2 Synchronous 667 MHz (1.5 ns)
      16. /0/17/2 memory DIMM DDR2 Synchronous 667 MHz (1.5 ns) [empty]
      17. /0/17/3 memory DIMM DDR2 Synchronous 667 MHz (1.5 ns) [empty]
      18. /0/17/4 memory 4GiB DIMM DDR2 Synchronous 667 MHz (1.5 ns)
      19. /0/17/5 memory 4GiB DIMM DDR2 Synchronous 667 MHz (1.5 ns)
      20. /0/17/6 memory DIMM DDR2 Synchronous 667 MHz (1.5 ns) [empty]
      21. /0/17/7 memory DIMM DDR2 Synchronous 667 MHz (1.5 ns) [empty]
      22. /0/17/8 memory 4GiB DIMM DDR2 Synchronous 667 MHz (1.5 ns)
      23. /0/17/9 memory 4GiB DIMM DDR2 Synchronous 667 MHz (1.5 ns)
      24. /0/17/a memory DIMM DDR2 Synchronous 667 MHz (1.5 ns) [empty]
      25. /0/17/b memory DIMM DDR2 Synchronous 667 MHz (1.5 ns) [empty]
      26. /0/17/c memory 4GiB DIMM DDR2 Synchronous 667 MHz (1.5 ns)
      27. /0/17/d memory 4GiB DIMM DDR2 Synchronous 667 MHz (1.5 ns)
      28. /0/17/e memory DIMM DDR2 Synchronous 667 MHz (1.5 ns) [empty]
      29. /0/17/f memory DIMM DDR2 Synchronous 667 MHz (1.5 ns) [empty]
      30. /0/100 bridge 5000P Chipset Memory Controller Hub
      31. /0/100/2 bridge 5000 Series Chipset PCI Express x8 Port 2-3
      32. /0/100/2/0 bridge 6311ESB/6321ESB PCI Express Upstream Port
      33. /0/100/2/0/0 bridge 6311ESB/6321ESB PCI Express Downstream Port E1
      34. /0/100/2/0/0/0 bridge 6700PXH PCI Express-to-PCI Bridge A
      35. /0/100/2/0/0/0.2 bridge 6700PXH PCI Express-to-PCI Bridge B
      36. /0/100/2/0/0/0.2/1 storage Serial ATA II RAID 1420SA
      37. /0/100/2/0/2 bridge 6311ESB/6321ESB PCI Express Downstream Port E3
      38. /0/100/2/0/2/0 enp6s0f0 network 80003ES2LAN Gigabit Ethernet Controller (Copper)
      39. /0/100/2/0/2/0.1 enp6s0f1 network 80003ES2LAN Gigabit Ethernet Controller (Copper)
      40. /0/100/2/0.3 bridge 6311ESB/6321ESB PCI Express to PCI-X Bridge
      41. /0/100/2/0.3/1 storage Serial ATA II RAID 1420SA
      42. /0/100/4 bridge 5000 Series Chipset PCI Express x8 Port 4-5
      43. /0/100/6 bridge 5000 Series Chipset PCI Express x8 Port 6-7
      44. /0/100/8 generic 5000 Series Chipset DMA Engine
      45. /0/100/1c bridge 631xESB/632xESB/3100 Chipset PCI Express Root Port 1
      46. /0/100/1d bus 631xESB/632xESB/3100 Chipset UHCI USB Controller #1
      47. /0/100/1d/1 usb2 bus UHCI Host Controller
      48. /0/100/1d.1 bus 631xESB/632xESB/3100 Chipset UHCI USB Controller #2
      49. /0/100/1d.1/1 usb3 bus UHCI Host Controller
      50. /0/100/1d.2 bus 631xESB/632xESB/3100 Chipset UHCI USB Controller #3
      51. /0/100/1d.2/1 usb4 bus UHCI Host Controller
      52. /0/100/1d.7 bus 631xESB/632xESB/3100 Chipset EHCI USB2 Controller
      53. /0/100/1d.7/1 usb1 bus EHCI Host Controller
      54. /0/100/1e bridge 82801 PCI Bridge
      55. /0/100/1e/1 display ES1000
      56. /0/100/1f bridge 631xESB/632xESB/3100 Chipset LPC Interface Controller
      57. /0/100/1f.1 storage 631xESB/632xESB IDE Controller
      58. /0/100/1f.2 storage 631xESB/632xESB SATA AHCI Controller
      59. /0/100/1f.3 bus 631xESB/632xESB/3100 Chipset SMBus Controller
      60. /0/101 bridge 5000 Series Chipset FSB Registers
      61. /0/102 bridge 5000 Series Chipset FSB Registers
      62. /0/103 bridge 5000 Series Chipset FSB Registers
      63. /0/104 bridge 5000 Series Chipset Reserved Registers
      64. /0/105 bridge 5000 Series Chipset Reserved Registers
      65. /0/106 bridge 5000 Series Chipset FBD Registers
      66. /0/107 bridge 5000 Series Chipset FBD Registers
      67. /0/1 scsi2 storage
      68. /0/1/0.0.0 /dev/sda disk 3TB ST33000651AS
      69. /0/1/0.0.0/1 /dev/sda1 volume 2794GiB OS X ZFS partition or Solaris /usr partition
      70. /0/1/0.0.0/9 /dev/sda9 volume 8191KiB reserved partition
      71. /0/2 scsi3 storage
      72. /0/2/0.0.0 /dev/sdb disk 3TB ST3000VX000-9YW1
      73. /0/2/0.0.0/1 /dev/sdb1 volume 2794GiB OS X ZFS partition or Solaris /usr partition
      74. /0/2/0.0.0/9 /dev/sdb9 volume 8191KiB reserved partition
      75. /0/3 scsi13 storage
      76. /0/3/0.0.0 /dev/sdk disk 3TB HUA723030ALA640
      77. /0/3/0.0.0/1 /dev/sdk1 volume 2794GiB OS X ZFS partition or Solaris /usr partition
      78. /0/3/0.0.0/9 /dev/sdk9 volume 8191KiB reserved partition
      79. /0/6 scsi14 storage
      80. /0/6/0.0.0 /dev/sdl disk 3TB ST33000651AS
      81. /0/6/0.0.0/1 /dev/sdl1 volume 2794GiB OS X ZFS partition or Solaris /usr partition
      82. /0/6/0.0.0/9 /dev/sdl9 volume 8191KiB reserved partition
      83. /0/7 scsi15 storage
      84. /0/7/0.0.0 /dev/sdm disk 3TB ST33000651AS
      85. /0/7/0.0.0/1 /dev/sdm1 volume 2794GiB OS X ZFS partition or Solaris /usr partition
      86. /0/7/0.0.0/9 /dev/sdm9 volume 8191KiB reserved partition
      87. /0/8 scsi4 storage
      88. /0/8/0.0.0 /dev/sdc disk 3TB ST33000651AS
      89. /0/8/0.0.0/1 /dev/sdc1 volume 2794GiB OS X ZFS partition or Solaris /usr partition
      90. /0/8/0.0.0/9 /dev/sdc9 volume 8191KiB reserved partition
      91. /0/9 scsi5 storage
      92. /0/9/0.0.0 /dev/sdd disk 3TB ST33000651AS
      93. /0/9/0.0.0/1 /dev/sdd1 volume 2794GiB OS X ZFS partition or Solaris /usr partition
      94. /0/9/0.0.0/9 /dev/sdd9 volume 8191KiB reserved partition
      95. /0/a scsi6 storage
      96. /0/a/0.0.0 /dev/sde disk 3TB ST33000651AS
      97. /0/a/0.0.0/1 /dev/sde1 volume 2794GiB OS X ZFS partition or Solaris /usr partition
      98. /0/a/0.0.0/9 /dev/sde9 volume 8191KiB reserved partition
      99. /0/b scsi7 storage
      100. /0/b/0.0.0 /dev/sdf disk 3TB ST3000VX000-9YW1
      101. /0/b/0.0.0/1 /dev/sdf1 volume 2794GiB OS X ZFS partition or Solaris /usr partition
      102. /0/b/0.0.0/9 /dev/sdf9 volume 8191KiB reserved partition
      103. /0/c scsi8 storage
      104. /0/c/0.0.0 /dev/sdg disk 3TB ST33000651AS
      105. /0/c/0.0.0/1 /dev/sdg1 volume 2794GiB OS X ZFS partition or Solaris /usr partition
      106. /0/c/0.0.0/9 /dev/sdg9 volume 8191KiB reserved partition
      107. /0/d scsi10 storage
      108. /0/d/0.0.0 /dev/sdh disk 3TB ST33000651AS
      109. /0/d/0.0.0/1 /dev/sdh1 volume 2794GiB OS X ZFS partition or Solaris /usr partition
      110. /0/d/0.0.0/9 /dev/sdh9 volume 8191KiB reserved partition
      111. /0/e scsi11 storage
      112. /0/e/0.0.0 /dev/sdi disk 80GB INTEL SSDSA2M080
      113. /0/e/0.0.0/1 /dev/sdi1 volume 42GiB EXT4 volume
      114. /0/e/0.0.0/2 /dev/sdi2 volume 31GiB Extended partition
      115. /0/e/0.0.0/2/5 /dev/sdi5 volume 31GiB Linux swap volume
      116. /0/f scsi12 storage
      117. /0/f/0.0.0 /dev/sdj disk 3TB ST33000651AS
      118. /0/f/0.0.0/1 /dev/sdj1 volume 2794GiB OS X ZFS partition or Solaris /usr partition
      119. /0/f/0.0.0/9 /dev/sdj9 volume 8191KiB reserved partition
      120. /1 system
      Display All
    • Strange. That is old hardware that should work fine with the 4.15 kernel. The plugin's dependencies won't let you use the 4.14 kernel (unless you apt-hold the packages) but it would be a bad idea to use that kernel. Is there a setting in the adaptec bios to enable ahci on the sata card? You could also install the standard 4.9 kernel instead of a backports kernel.
      omv 4.1.22 arrakis | 64 bit | 4.15 proxmox kernel | omvextrasorg 4.1.15
      omv-extras.org plugins source code and issue tracker - github

      Please read this before posting a question and this and this for docker questions.
      Please don't PM for support... Too many PMs!
    • @ryecoaaron, I noticed poking around that there were a ton of commits made to the plugin back in October 2018 that aren't part of the currently used plugin. I was toying with the idea of trying to rework some stuff (in my fumble-fingered way) but I'm not sure if that was stuff that was prep for 5.x, or if it was a partly-completed rewrite, or what. Any idea whether it's a partially complete rewrite or something prepping for 5.x?
    • wolffstarr wrote:

      I noticed poking around that there were a ton of commits made to the plugin back in October 2018 that aren't part of the currently used plugin. I was toying with the idea of trying to rework some stuff (in my fumble-fingered way) but I'm not sure if that was stuff that was prep for 5.x, or if it was a partly-completed rewrite, or what. Any idea whether it's a partially complete rewrite or something prepping for 5.x?
      That was @subzero79 fixing a bunch of stuff. Not sure why we didn't release that. It is not prep for 5.x and I think it should be used. Actually, the only thing that needs to be ported to 5.x is the email notification stuff. The plugin should just work as it is.
      omv 4.1.22 arrakis | 64 bit | 4.15 proxmox kernel | omvextrasorg 4.1.15
      omv-extras.org plugins source code and issue tracker - github

      Please read this before posting a question and this and this for docker questions.
      Please don't PM for support... Too many PMs!
    • ryecoaaron wrote:

      wolffstarr wrote:

      I noticed poking around that there were a ton of commits made to the plugin back in October 2018 that aren't part of the currently used plugin. I was toying with the idea of trying to rework some stuff (in my fumble-fingered way) but I'm not sure if that was stuff that was prep for 5.x, or if it was a partly-completed rewrite, or what. Any idea whether it's a partially complete rewrite or something prepping for 5.x?
      That was @subzero79 fixing a bunch of stuff. Not sure why we didn't release that. It is not prep for 5.x and I think it should be used. Actually, the only thing that needs to be ported to 5.x is the email notification stuff. The plugin should just work as it is.
      Okay. I have no idea whatsoever what I'm doing, so working off his would be best if it's functional, but I didn't want to go down that path without making sure I wasn't basing on the wrong thing.
    • subzero79 wrote:

      The plugin doesn’t support encryption in zfs. Bear in mind that encryption in zol I atm in a rc version, not the current one. If you need encryption only solution I see is to use zvols formatted as luks, then you can use ext4 or xfs on top.
      Other option destroy the pool, format each disk with luks, then create pool with the devices.

      If I were you I would wait.
      @ovizii As a combination of the two above, you can wait for native encryption and during waiting: use a zvol based encrypted container...

      Sent from my phone
    • pro_lamer wrote:

      subzero79 wrote:

      The plugin doesn’t support encryption in zfs. Bear in mind that encryption in zol I atm in a rc version, not the current one. If you need encryption only solution I see is to use zvols formatted as luks, then you can use ext4 or xfs on top.
      Other option destroy the pool, format each disk with luks, then create pool with the devices.

      If I were you I would wait.
      @ovizii As a combination of the two above, you can wait for native encryption and during waiting: use a zvol based encrypted container...
      Sent from my phone
      Thanks. Since I am moving from freenas to OMV I can decrypt my pool and start using OMV while waiting for ZOL 0.8 (with native encryption to hit the Debian 9 repos) - encryption is not "needed" but after having had it for about 5 years I feel naked disabling it :)

      I'm currently trying to figure out the right hardware (upgrades) and I saw some announcement about OMV 5 moving to BTRFS ( github.com/openmediavault/openmediavault/issues/101 ) and a lot of discussions going on there about ZFS so I was wondering whether to start with OMV 4 with the ZOl version supplied by Debian 9 or with OMV 5 with ZOl supplied by Debian 10?
    • ovizii wrote:

      Thanks. Since I am moving from freenas to OMV I can decrypt my pool and start using OMV while waiting for ZOL 0.8 (with native encryption to hit the Debian 9 repos) - encryption is not "needed" but after having had it for about 5 years I feel naked disabling it

      I'm currently trying to figure out the right hardware (upgrades) and I saw some announcement about OMV 5 moving to BTRFS ( github.com/openmediavault/openmediavault/issues/101 ) and a lot of discussions going on there about ZFS so I was wondering whether to start with OMV 4 with the ZOl version supplied by Debian 9 or with OMV 5 with ZOl supplied by Debian 10?
      You really don't want to go to OMV 5 - it's early Beta, most of the plugins (last I checked, a couple of weeks ago) weren't anywhere near ported, and there's still a long way to go.

      That said, the BTRFS change was pushed back to OMV6 at least, so won't be much of a worry.