LXC support for openmediavault-kvm plugin

    • Offizieller Beitrag
    1. LXC should be as secure as docker when inside the container. Outside the container, you can't have loose permissions on the os files since they are now visible on the host. If someone has that much access to your host, your other security has failed.
    2. id mapping for what?
    3. Not sure. You could definitely mount it readonly in the guest OS.
    4. Don't : ) I don't plan to create my own templates since there are so many available. I would rather put effort into a script that will do the setup on a new container.
    5. For now. I need to see what it is needed to get the existing kvm plugin backup stuff to work with a container. I have a feeling there is a lot of work involved but I haven't played with lxc snapshots much.

    omv 7.0.5-1 sandworm | 64 bit | 6.8 proxmox kernel

    plugins :: omvextrasorg 7.0 | kvm 7.0.13 | compose 7.1.6 | k8s 7.1.0-3 | cputemp 7.0.1 | mergerfs 7.0.4


    omv-extras.org plugins source code and issue tracker - github - changelogs


    Please try ctrl-shift-R and read this before posting a question.

    Please put your OMV system details in your signature.
    Please don't PM for support... Too many PMs!

    2. e.g, root in LXC not same id/gid as root on host, etc. Privleged vs. unprivleged container.


    4. I meant once template is downloaded what can be down outside LXC roofs as opposed to just scripting changes to a running LXC.


    5. No problem with as is.

    • Offizieller Beitrag
    Zitat von Krisbee

    root in LXC not same id/gid as root on host, etc. Privleged vs. unprivleged container.

    They are privileged containers. I don't even know if you can specify id/gid. Feel free to research that. I have no plans to add unprivileged containers unless it is very painless codewise.

    Zitat von Krisbee

    I meant once template is downloaded what can be down outside LXC roofs as opposed to just scripting changes to a running LXC.

    You should be able to add, edit, or remove files. It should be the same as changing files in a docker volume path. What did you have in mind? Not that I can change how LXC works.

    omv 7.0.5-1 sandworm | 64 bit | 6.8 proxmox kernel

    plugins :: omvextrasorg 7.0 | kvm 7.0.13 | compose 7.1.6 | k8s 7.1.0-3 | cputemp 7.0.1 | mergerfs 7.0.4


    omv-extras.org plugins source code and issue tracker - github - changelogs


    Please try ctrl-shift-R and read this before posting a question.

    Please put your OMV system details in your signature.
    Please don't PM for support... Too many PMs!

    • Offizieller Beitrag

    id mapping is support but it breaks things when the container boots. Seems like it requires unprivileged containers. https://libvirt.org/formatdomain.html#container-boot

    omv 7.0.5-1 sandworm | 64 bit | 6.8 proxmox kernel

    plugins :: omvextrasorg 7.0 | kvm 7.0.13 | compose 7.1.6 | k8s 7.1.0-3 | cputemp 7.0.1 | mergerfs 7.0.4


    omv-extras.org plugins source code and issue tracker - github - changelogs


    Please try ctrl-shift-R and read this before posting a question.

    Please put your OMV system details in your signature.
    Please don't PM for support... Too many PMs!

    ryecoaaron


    I would not expect to be creating many LXC containers myself, but it would be good to know what is possible and how best to use this new functionality.


    At the moment to get a LXC roofs on OMv6 via the plugin you have to create at least one LXC container, but not necessarily run it. Following that you might want to clone this rootfs (using rsync or zfs send/recv?) before making changes to this LXC rootfs.


    Changes might involve adding/altering files in the LXC roofs before it's run, e.g ssh keys, no ssh password auth, a systemd service definition, etc. You might also want to alter one or more running LXC containers, e.g loop through a list of containers to do a "apt update && apt upgrade -y" or act on a single container.


    IIRC, Proxmox has "pct push" and "pct exec" commands to do this kind of thing from the host. The pct command being a wrapper for LXC commands. I might be wrong, but the the libvrtd LXC driver does not seem to have anything equivalent.


    So as you say, you'd couldbe editing the LXC roofs directly in some cases, but with no equivalent of LXC exec I'm not sure how you'd script something like "apt update && apt upgrade -y" for all running LXC containers.

    Einmal editiert, zuletzt von Krisbee () aus folgendem Grund: typos

    • Offizieller Beitrag
    Zitat von Krisbee

    IIRC, Proxmox has "pct push" and "pct exec" commands to do this kind of thing from the host. The pct command being a wrapper for LXC commands. I might be wrong, but the the libvrtd LXC driver does not seem to have anything equivalent.

    Definitely exists. If proxmox has it, it exists in lxc and libvirt mostly exposes the same.

    virsh -c lxc:///system lxc-enter-namespace LXC_CONTAINER_NAME -- /bin/ls -al /dev

    https://libvirt.org/drvlxc.html

    omv 7.0.5-1 sandworm | 64 bit | 6.8 proxmox kernel

    plugins :: omvextrasorg 7.0 | kvm 7.0.13 | compose 7.1.6 | k8s 7.1.0-3 | cputemp 7.0.1 | mergerfs 7.0.4


    omv-extras.org plugins source code and issue tracker - github - changelogs


    Please try ctrl-shift-R and read this before posting a question.

    Please put your OMV system details in your signature.
    Please don't PM for support... Too many PMs!

    ryecoaaron I had found that, but the command fails when LXC ct is off as expected and when LXC ct is running:


    Code
    root@omv6:~# virsh -c lxc:///system lxc-enter-namespace ublxc  -- /bin/ls -al /root
error: Requested operation is not valid: domain is not running

root@omv6:~# virsh -c lxc:///system lxc-enter-namespace ublxc  -- /bin/ls -al /root
libvirt: Cgroup error : Unable to write to '/sys/fs/cgroup/machine.slice/machine-lxc\x2d10354\x2dublxc.scope/cgroup.procs': Device or resource busy
error: internal error: Child process (10584) unexpected exit status 125

root@omv6:~#



    Also, there's one minor bug I've found. Just navigating to the KVM / VMs web page with an LXC container listed causes the syslog to be spammed with this message:


    Code
    Nov 11 15:38:22 omv6 libvirtd[1512]: this function is not supported by the connection driver: virDomainSnapshotNum
Nov 11 15:38:22 omv6 libvirtd[1512]: this function is not supported by the connection driver: virDomainSnapshotNum
Nov 11 15:38:32 omv6 libvirtd[1512]: this function is not supported by the connection driver: virDomainSnapshotNum
Nov 11 15:38:32 omv6 libvirtd[1512]: this function is not supported by the connection driver: virDomainSnapshotNum
    • Offizieller Beitrag
    Zitat von Krisbee

    I had found that, but the command fails when LXC ct is off as expected and when LXC ct is running:

    No idea. I am new to LXC too.

    Zitat von Krisbee

    Also, there's one minor bug I've found. Just navigating to the KVM / VMs web page with an LXC container listed causes the syslog to be spammed with this message:

    thanks. I will fix that.

    omv 7.0.5-1 sandworm | 64 bit | 6.8 proxmox kernel

    plugins :: omvextrasorg 7.0 | kvm 7.0.13 | compose 7.1.6 | k8s 7.1.0-3 | cputemp 7.0.1 | mergerfs 7.0.4


    omv-extras.org plugins source code and issue tracker - github - changelogs


    Please try ctrl-shift-R and read this before posting a question.

    Please put your OMV system details in your signature.
    Please don't PM for support... Too many PMs!

    ryecoaaron A brief follow up on the “virsh -c lxc:///system lxc-enter-namespace lmslxc -- /bin/ls -al /root” error. I realise livbvrt-lxc and lxc are not the same thing but if the lxc-checkconfig command is still relevant it may point to a problems with cgroups on OMV6.



    I don’t know for sure if the output is significant but on debian 11 you have this for cgroups:


    Code
    root@omv6:~# mount | grep cgroup
cgroup2 on /sys/fs/cgroup type cgroup2 (rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot)
root@omv6:~#


    In thisthread at the debian forum there’s an interchange re: a problem caused by the absence of V1 cgroups. The notes at https://libvirt.org/drvlxc.html#control-groups-requirements imply you need v1 cgroups. Supposedly the answer is to use systemd in some way.


    In contrast to debian, the cgroups on my kubuntu desktop are this:



    Using a virt-manager lxc connection on my desktop shows a command like


    “virsh -c lxc:///system lxc-enter-namespace lmslxc -- /bin/ls -al" works only if you add the option --noseclabel , e.g.:


    Einmal editiert, zuletzt von Krisbee () aus folgendem Grund: incomplete

    • Offizieller Beitrag

    If you find an easy way to enable cgroups v1, maybe the plugin could do that.

    omv 7.0.5-1 sandworm | 64 bit | 6.8 proxmox kernel

    plugins :: omvextrasorg 7.0 | kvm 7.0.13 | compose 7.1.6 | k8s 7.1.0-3 | cputemp 7.0.1 | mergerfs 7.0.4


    omv-extras.org plugins source code and issue tracker - github - changelogs


    Please try ctrl-shift-R and read this before posting a question.

    Please put your OMV system details in your signature.
    Please don't PM for support... Too many PMs!

    ryecoaaron The solution is buried here https://www.debian.org/release…en.html#openstack-cgroups.

    Zitat

    add the parameters
    systemd.unified_cgroup_hierarchy=false and
    systemd.legacy_systemd_cgroup_controller=false
    to the kernel command line in order to override the default and
    restore the old cgroup hierarchy.


    Testing in Omv6 shows the old cgroup behaviour is restored and for a pve kernel:



    Unfortunately starting an LXC container now generates an error from the KVM plugin:


    Code
    Unable to - poweronerror from service: GDBus.Error:org.freedesktop.machine1.NoMachineForPID: PID 3709 does not belong to any known machine

OMV\Exception: Unable to - poweronerror from service: GDBus.Error:org.freedesktop.machine1.NoMachineForPID: PID 3709 does not belong to any known machine in /usr/share/openmediavault/engined/rpc/kvm.inc:2174
Stack trace:
#0 [internal function]: OMVRpcServiceKvm->doCommand(Array, Array)
#1 /usr/share/php/openmediavault/rpc/serviceabstract.inc(123): call_user_func_array(Array, Array)
#2 /usr/share/php/openmediavault/rpc/rpc.inc(86): OMV\Rpc\ServiceAbstract->callMethod('doCommand', Array, Array)
#3 /usr/sbin/omv-engined(537): OMV\Rpc\Rpc::call('Kvm', 'doCommand', Array, Array, 1)
#4 {main}


    I tested this outside of OMV6 in another debian 11 VM which has libvirt installed and after adding those two boot params libvrt-lxc seems to work OK., e.g:



    Of course, I have no idea of the possible unwanted side effects of using those kernel boot params.

    • Offizieller Beitrag
    Zitat von Krisbee

    Unfortunately starting an LXC container now generates an error from the KVM plugin:

    Just for existing LXC containers or new containers as well? Not a huge fan of editing kernel parameters with plugins.

    Zitat von Krisbee

    Of course, I have no idea of the possible unwanted side effects of using those kernel boot params.

    Maybe docker issues?

    omv 7.0.5-1 sandworm | 64 bit | 6.8 proxmox kernel

    plugins :: omvextrasorg 7.0 | kvm 7.0.13 | compose 7.1.6 | k8s 7.1.0-3 | cputemp 7.0.1 | mergerfs 7.0.4


    omv-extras.org plugins source code and issue tracker - github - changelogs


    Please try ctrl-shift-R and read this before posting a question.

    Please put your OMV system details in your signature.
    Please don't PM for support... Too many PMs!

    ryecoaaron The error occurs for both existing and newly added containers. If editing kernel parameters doesn't fit with OMV6, then is this addition to the KVM plugin worth pursuing? Manipulating cgroups looks a tricky thing to do unless there is a systemd way to do it. I doubt if you'd want to develop a separate proper LXC plugin.


    From my casual reading about containers, libvirt LXC seems to have a tiny footprint on the web, often seen as a quick and dirty way of creating a privleged container. Ubuntu may still promote LXD/LXC and Promox persists with their use of LXC containers, but LXC is dwarfed by docker/kubernetes use.


    Do you think many OMV6 users would even make much use of such an option, as opposed to using dockers?

    • Offizieller Beitrag
    Zitat von Krisbee

    If editing kernel parameters doesn't fit with OMV6

    It might fit in omv6 but I don't want to add it to the plugin.


    Zitat von Krisbee

    then is this addition to the KVM plugin worth pursuing?

    Because you can't run commands inside the container from outside the container? If you can't do what you need, I'm sorry. But the containers are working perfect for my needs. I spent a lot of hours adding this and I'm not going to rip it out because some use cases don't work. And if you aren't using lxc, it shouldn't affect your system at all.


    Zitat von Krisbee

    I doubt if you'd want to develop a separate proper LXC plugin.

    Definitely not. I am able to use a lot of code that VMs use and it doesn't make sense to separate them.


    Zitat von Krisbee

    From my casual reading about containers, libvirt LXC seems to have a tiny footprint on the web, often seen as a quick and dirty way of creating a privleged container. Ubuntu may still promote LXD/LXC and Promox persists with their use of LXC containers, but LXC is dwarfed by docker/kubernetes use.


    Do you think many OMV6 users would even make much use of such an option, as opposed to using dockers?

    Probably not but OMV users are not my motivation for adding it. I am my motivation for adding it. No one has to use it. It isn't supposed to replace docker at all. I do a lot of testing where I need a different full distro or version and I can spin up an lxc container faster than I could install and configure a VM.

    omv 7.0.5-1 sandworm | 64 bit | 6.8 proxmox kernel

    plugins :: omvextrasorg 7.0 | kvm 7.0.13 | compose 7.1.6 | k8s 7.1.0-3 | cputemp 7.0.1 | mergerfs 7.0.4


    omv-extras.org plugins source code and issue tracker - github - changelogs


    Please try ctrl-shift-R and read this before posting a question.

    Please put your OMV system details in your signature.
    Please don't PM for support... Too many PMs!

    ryecoaaron It's not about what I need, I was thinking of your development time not knowing how complete your work was.


    Obviously it's not meant to replace docker. I suppose command's over ssh is an alternative way to maintain a container you wanted to keep. I read that those kernel systemd boot params are deprecated from systemd 252 onward and debian testing is already at systemd 251. So changing from unified back to hybrid cgroups looks like a non-starter anyway, or least would have a limited life time .


    Creating a LXC container via libvrit is much faster than a full VM, so it is a useful option to have. Happy to keep testing.


    The templates are dependent on the full LXC package and the default config creates a lxc bridge, but it is ever going to be used? It can be turned on in the relevant LXC config file.

    • Offizieller Beitrag
    Zitat von Krisbee

    I suppose command's over ssh is an alternative way to maintain a container you wanted to keep.

    I guess I still don't know why you are focusing on remote commands. I am using the containers like a VM. I login via ssh or virsh console and do what I need. This is great for when I need something for a very short time (happens often) and now I don't have to do the install.


    Zitat von Krisbee

    The templates are dependent on the full LXC package and the default config creates a lxc bridge, but it is ever going to be used? It can be turned on in the relevant LXC config file.

    I guess I could add it to the list of network types. I hadn't needed it for anything. So, I wasn't worried about it.

    omv 7.0.5-1 sandworm | 64 bit | 6.8 proxmox kernel

    plugins :: omvextrasorg 7.0 | kvm 7.0.13 | compose 7.1.6 | k8s 7.1.0-3 | cputemp 7.0.1 | mergerfs 7.0.4


    omv-extras.org plugins source code and issue tracker - github - changelogs


    Please try ctrl-shift-R and read this before posting a question.

    Please put your OMV system details in your signature.
    Please don't PM for support... Too many PMs!

    I tried this by installing image:"openwrt; 22.03; amd64; default; 20221112 _11:57."


    The following error message was received:

    • Offizieller Beitrag

    It failed to reset the root password. I have no plans to test every image that linuxcontainers offers but I can make that error non-fatal.

    omv 7.0.5-1 sandworm | 64 bit | 6.8 proxmox kernel

    plugins :: omvextrasorg 7.0 | kvm 7.0.13 | compose 7.1.6 | k8s 7.1.0-3 | cputemp 7.0.1 | mergerfs 7.0.4


    omv-extras.org plugins source code and issue tracker - github - changelogs


    Please try ctrl-shift-R and read this before posting a question.

    Please put your OMV system details in your signature.
    Please don't PM for support... Too many PMs!

    • Offizieller Beitrag
    Zitat von albicocca

    wait for better features.

    What better features?

    omv 7.0.5-1 sandworm | 64 bit | 6.8 proxmox kernel

    plugins :: omvextrasorg 7.0 | kvm 7.0.13 | compose 7.1.6 | k8s 7.1.0-3 | cputemp 7.0.1 | mergerfs 7.0.4


    omv-extras.org plugins source code and issue tracker - github - changelogs


    Please try ctrl-shift-R and read this before posting a question.

    Please put your OMV system details in your signature.
    Please don't PM for support... Too many PMs!

Jetzt mitmachen!

Sie haben noch kein Benutzerkonto auf unserer Seite? Registrieren Sie sich kostenlos und nehmen Sie an unserer Community teil!

Benutzerkonto erstellen Anmelden