Compression and Deduplication using OpenSource RedHat VDO - Help

  • RedHat acquired Permabit, which made the very good block layer compression and dedup solution called VDO (already many years of production code). It has opensourced it (it's on github right now). It's solid, tested for years. I have tested it and it seems very good. I think this is an awesome solution for those whose only option was ZFS but just wanted a simple solution to the dedup and compress problem.


    It's realtime and inline, it uses DM and goes under the filesystem and it's very easy to insert between layers. It's easy to use. There are two kernel modules selfcontained, very easy to compile. I compiled for fedora 27/28 without problems.


    Is anyone already working to integrate it into open media vault? Would someone try and post a guide? I guess the best way would it to be distributed natively and have a plugin use it.


    Some links:


    This is the original Permabit product:
    http://permabit.com/products-o…rtual-data-optimizer-vdo/


    This is the announcement of redhat buying it:
    https://www.redhat.com/en/abou…-deduplication-technology


    This is proof that it has been opensourced -> "FINALLY! Open Source Deduplication for Linux!"
    https://www.linbit.com/en/fina…urce-deduplication-linux/


    This is the GitHub repository, with production code:
    https://github.com/dm-vdo


    This is a redhat blog entry about VDO":
    https://rhelblog.redhat.com/20…zer-vdo-in-rhel-7-5-beta/
    https://rhelblog.redhat.com/20…zer-vdo-in-rhel-7-5-beta/


    This is the official documentation on the RHEL7 guide:
    https://access.redhat.com/docu…_administration_guide/vdo


    It's already working on RHEL 7.5 and will be present on Centos 7.5 as well. This is good stuff, I can't wait to see it integrated in OMV and would like to install it myself, just need a little help from someone that can make it work and share the steps.


    I successfully compiled the kernel module on debian 8, but the applications, although they compiled and installed fine, give errors of dependencies (??) at runtime. In debian 9 it won't compile. I'm not sure about the exact environment of the last release of OVM and how to integrate it there.


    Would love to see someone working on this.


    Thank you!!

    • Offizieller Beitrag

    Is anyone already working to integrate it into open media vault? Would someone try and post a guide? I guess the best way would it to be distributed natively and have a plugin use it.

    I've started looking at but haven't had a chance to look at what is needed to "debianize" it. If you can find pre-made debian/ubuntu packages, that would save a lot of time. Creating a plugin based off the luksencryption plugin would save a lot of time as well.

    omv 7.0.5-1 sandworm | 64 bit | 6.8 proxmox kernel

    plugins :: omvextrasorg 7.0 | kvm 7.0.13 | compose 7.1.4 | k8s 7.1.0-3 | cputemp 7.0.1 | mergerfs 7.0.4


    omv-extras.org plugins source code and issue tracker - github - changelogs


    Please try ctrl-shift-R and read this before posting a question.

    Please put your OMV system details in your signature.
    Please don't PM for support... Too many PMs!

    • Offizieller Beitrag

    Why not use deduplication on btrfs?

    You can but vdo works with any filesystem.


    I decided to try vdo on my omv 4 system. It works pretty well. I wish I knew more about dkms.


    Code
    $ sudo vdostats --human-readable
    
    
    Device                    Size      Used Available Use% Space saving%
    /dev/mapper/vdotest     100.0G      6.1G     93.9G   6%           79%

    omv 7.0.5-1 sandworm | 64 bit | 6.8 proxmox kernel

    plugins :: omvextrasorg 7.0 | kvm 7.0.13 | compose 7.1.4 | k8s 7.1.0-3 | cputemp 7.0.1 | mergerfs 7.0.4


    omv-extras.org plugins source code and issue tracker - github - changelogs


    Please try ctrl-shift-R and read this before posting a question.

    Please put your OMV system details in your signature.
    Please don't PM for support... Too many PMs!

Jetzt mitmachen!

Sie haben noch kein Benutzerkonto auf unserer Seite? Registrieren Sie sich kostenlos und nehmen Sie an unserer Community teil!