[HowTo] Nvidia hardware transcoding on OMV 5 in a Plex docker container

It is quite tricky to get Plex hardware transcoding working on OMV in a docker container with nvidia graphics cards.

It took me some weeks to find out all the little details and I want to share it with you.

My configuration is tested for:

- Debian 10 Buster (wich is OMV5)

- OMV 5.6.1-1 Usul

- Kernel Linux 5.10.0-0.bpo.3-amd64

- Docker 5:20.10.5~3-0~debian-buster

- Portainer 2.1.1

- Nvidia driver 460.39

- Cuda Version 11.2

- Nvidia Quadro P1000 graphics card

- Plex 1.22.0.4145

First of all and very important:

Hardware transcoding must work on the intel hardware ! There are several tutorials for that. Mainly it is related to access rights like the "noexex" to "exec" option in the fstab.

Step 1:

if you already tried to install nvidia driver: purge it out !

apt-get purge *nvidia*

apt autoremove

apt autoclean

Step 2:

prepare your header files. Very important !

apt-get install module-assistant

sudo m-a prepare

Step 3: installing Nvidia Driver

follow the instructions given by Nvidia:

https://wiki.debian.org/Nvidia…28via_buster-backports.29

in detail:

Add buster-backports to your /etc/apt/sources.list , for example:

#Busterter-backports
deb http://deb.debian.org/debian buster-backports main contrib non-free

then:

apt update
apt install -t buster-backports nvidia-driver firmware-misc-nonfree

IMPORTANT:

watch the messages during intallation. It there are any error messages something is wrong and you might not have debian buster with backports ! You have to solve this issue first before proceeding !!!

now we have to configure nvidia:

apt install -t buster-backports nvidia-xconfig

sudo nvidia-xconfig

Since Docker 5.20.10.2 (I think) there was a change how docker gets access to hardware via cgroups. You need this workaround in the kernel boot parameters:

echo 'GRUB_CMDLINE_LINUX=systemd.unified_cgroup_hierarchy=false' > /etc/default/grub.d/cgroup.cfg

update-grub

now reboot and the nvidia driver should already work.

Step 4: Install Nvidia container toolkit

Follow the Installation guide by Nvidia:

https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/install-guide.html#setting-up-nvidia-container-toolkit

in detail:

Install curl if you don't have it already:

sudo apt install curl

Setup the stable repository and the GPG key:

distribution=$(. /etc/os-release;echo $ID$VERSION_ID) \
   && curl -s -L https://nvidia.github.io/nvidia-docker/gpgkey | sudo apt-key add - \
   && curl -s -L https://nvidia.github.io/nvidia-docker/$distribution/nvidia-docker.list | sudo tee /etc/apt/sources.list.d/nvidia-docker.list

Install the <span style="font-size: 10pt;">nvidia-docker2</span> package (and dependencies) after updating the package listing:

sudo apt-get update

apt install -t buster-backports nvidia-docker2

Now install Nvidia encode library and nvidia-smi:

apt install -t buster-backports libnvidia-encode1

apt install -t buster-backports nvidia-smi

Step 5: install Nvidia container runtime:

apt install -t buster-backports nvidia-container-runtime

Step 6: some modifications:

Change/edit the Daemon configuration file

/etc/docker/daemon.json :

{
    "runtimes": {
        "nvidia": {
            "path": "/usr/bin/nvidia-container-runtime",
            "runtimeArgs": []
        }
    },
    "default-runtime": "nvidia",
    "data-root": "/var/lib/docker"
}

and the

/etc/nvidia-container-runtime/config.toml

to:

disable-require = false
#swarm-resource = "DOCKER_RESOURCE_GPU"
accept-nvidia-visible-devices-envvar-when-unprivileged = true
#accept-nvidia-visible-devices-as-volume-mounts = false

[nvidia-container-cli]
#root = "/run/nvidia/driver"
#path = "/usr/bin/nvidia-container-cli"
environment = []
#debug = "/var/log/nvidia-container-toolkit.log"
#ldcache = "/etc/ld.so.cache"
load-kmods = true
#no-cgroups = false
#user = "root:video"
#ldconfig = "@/sbin/ldconfig"
#ldconfig = "/sbin/ldconfig"
ldconfig = "/sbin/ldconfig.real"

[nvidia-container-runtime]
#debug = "/var/log/nvidia-container-runtime.log"

Restart docker:

sudo systemctl restart docker

Step 7: starting Plex (in Portainer)

in Portainer you have to add the following parameters in the "Env" tab:

and in the tab "Runtime & Ressources":

change the "Runtime" Value from runc to nvidia !!!

No Privileged mode and no Init set.

Step 8: try and error:

this is how it worked for me. If you want to check operation you can display GPU load with:

watch -d -n 0.5 nvidia-smi

or install:

apt install nvtop

and use:

nvtop

You can also try/use the docker command line interface (cli) to start Plex:

docker run \
-d \
--name=plex-gpu \
--net=host \
--gpus all \
-device /dev/dri:/dev/dri \
-e PUID=xxxxxx \                    #change to your value
-e PGID=yyyyyy \                    #change to your value
-e PLEX_MEDIA_SERVER_USER=plex \
-e VERSION="plexpass" \
-v your volumes \                   #map your volumes
-v your volumes \                   #map your volumes
--restart unless-stopped \
ghcr.io/linuxserver/plex:latest

Step 9: get rid of the session limit:

If you want to disable the session limit (my P1000 had a limit of 3) go ahead with this link. It worked for me also:

https://github.com/keylase/nvidia-patch

in detail:

if you don't have git, install it:

apt git install

then

git clone https://github.com/keylase/nvidia-patch.git nvidia-patch

Patch the nvidia driver:

cd nvidia-patch
bash ./patch.sh

If you want to rollback:

bash ./patch.sh -r

Step 10: Nvidia Power Management:

You need these modifications if you are using autoshutdown as the nvidia driver (or the pci-bus of the card?) is falling of if restarting from hibernate or suspend mode !

You can read more about that in the nvidia documentation:

http://us.download.nvidia.com/…namicpowermanagement.html

http://us.download.nvidia.com/…ADME/powermanagement.html

http://us.download.nvidia.com/…/nvidia-persistenced.html

First of all you need the dedicated nvidia scripts for power management and you have to find them in the nvidia driver install package:

Download the nvidia install package from:

https://www.nvidia.com/de-de/drivers/unix/

or for the 460 driver the direct link:

https://www.nvidia.de/Download/driverResults.aspx/170214/de

Or on the system:

wget http://us.download.nvidia.com/XFree86/Linux-x86_64/460.39/NVIDIA-Linux-x86_64-460.39.run

Now do not install the driver (!) - just extract it:

sh NVIDIA-Linux-x86_64-460.39.run --extract-only

In the next step you have to search for the following files and copy them to the given directories (I used an ssh client for this)

/etc/systemd/system/nvidia-suspend.service
/etc/systemd/system/nvidia-hibernate.service
/etc/systemd/system/nvidia-resume.service
/lib/systemd/system-sleep/nvidia
/usr/bin/nvidia-sleep.sh

then enable the services:

sudo systemctl enable nvidia-suspend.service
sudo systemctl enable nvidia-hibernate.service
sudo systemctl enable nvidia-resume.service

change nvidia kernel config:

/etc/modprobe.d/nvidia-kernel-common.conf

options nvidia NVreg_PreserveVideoMemoryAllocations=1
options nvidia NVreg_DynamicPowerManagementVideoMemoryThreshold=100
options nvidia NVreg_DynamicPowerManagement=0x02
options nvidia NVreg_EnableMSI=0

now make nvidia-sleep.sh executable and update modules:

chmod a+x /usr/bin/nvidia-sleep.sh

sudo update-initramfs -u

Remark:

I still use the script files from the 450 driver version and they still work with 460. So I think they are more general and not driver specific. But if you experience problems

with hardware transcoding on a new driver version after a resume maybe you have to extract them fresh from the new driver package.

And I don't know if this final part is really needed, but if you still have issues with resume from suspend you can try to disable active state power management on pcie:

Change/add kernel parameter pcie_aspm=off in /etc/default/grub to:

GRUB_CMDLINE_LINUX_DEFAULT="quiet pcie_aspm=off"

and after changing grub:

sudo update-grub

reboot

That was a long way

Please let me know if there are still issues with htis guide. I'll try to keep it up to date...

Good luck,

Chris