For OMV 6 proceed to the second post: OMV 6
For OMV 7/8 proceed to OMV 7/8
It is quite tricky to get Plex hardware transcoding working on OMV in a docker container with nvidia graphics cards.
It took me some weeks to find out all the little details and I want to share it with you.
My configuration is tested for:
- Debian 10 Buster (wich is OMV5)
- OMV 5.6.13-1 Usul
- Kernel Linux 5.10.0-0.bpo.8-amd64
- Docker 5:20.10.8~3-0~debian-buster
- Portainer 2.6.0
- Nvidia driver 460.73.01
- Cuda Version 11.2
- Nvidia Quadro P1000 graphics card
- Plex 1.24.0.4897
First of all and very important:
Hardware transcoding must work on the intel hardware ! There are several tutorials for that. Mainly it is related to access rights like the "noexex" to "exec" option in the fstab.
Step 1:
if you already tried to install nvidia driver: purge it out !
apt-get purge *nvidia*
apt autoremove
apt autocleanStep 2:
prepare your header files. Very important !
apt-get install module-assistant
sudo m-a prepareStep 3: installing Nvidia Driver
follow the instructions given by Nvidia:
NvidiaGraphicsDrivers - Debian Wiki
in detail:
Add buster-backports to your /etc/apt/sources.list , for example:
#Busterter-backports
deb http://deb.debian.org/debian buster-backports main contrib non-freethen:
apt update
apt install -t buster-backports nvidia-driver firmware-misc-nonfreeIMPORTANT:
watch the messages during intallation. It there are any error messages something is wrong and you might not have debian buster with backports ! You have to solve this issue first before proceeding !!!
now we have to configure nvidia:
apt install -t buster-backports nvidia-xconfig
sudo nvidia-xconfigSince Docker 5.20.10.2 (I think) there was a change how docker gets access to hardware via cgroups. You need this workaround in the kernel boot parameters:
echo 'GRUB_CMDLINE_LINUX=systemd.unified_cgroup_hierarchy=false' > /etc/default/grub.d/cgroup.cfg
update-grubnow reboot and the nvidia driver should already work.
Step 4: Install Nvidia container toolkit
Follow the Installation guide by Nvidia:
in detail:
Install curl if you don't have it already:
sudo apt install curlSetup the stable repository and the GPG key - all in one command! (use copy button):
if this doesn't work please check for any changes under this link: nvidia container
distribution=$(. /etc/os-release;echo $ID$VERSION_ID) \
&& curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg \
&& curl -s -L https://nvidia.github.io/libnvidia-container/$distribution/libnvidia-container.list | \
sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#g' | \
sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.listInstall the nvidia-docker2 package (and dependencies) after updating the package listing:
sudo apt-get update
apt install -t buster-backports nvidia-docker2Now install Nvidia encode library and nvidia-smi:
apt install -t buster-backports libnvidia-encode1
apt install -t buster-backports nvidia-smiStep 5: install Nvidia container runtime:
apt install -t buster-backports nvidia-container-runtimeStep 6: some modifications:
Change/edit the Daemon configuration file
/etc/docker/daemon.json :
{
"runtimes": {
"nvidia": {
"path": "/usr/bin/nvidia-container-runtime",
"runtimeArgs": []
}
},
"default-runtime": "nvidia",
"data-root": "/var/lib/docker"
}and the
/etc/nvidia-container-runtime/config.toml
to:
disable-require = false
#swarm-resource = "DOCKER_RESOURCE_GPU"
accept-nvidia-visible-devices-envvar-when-unprivileged = true
#accept-nvidia-visible-devices-as-volume-mounts = false
[nvidia-container-cli]
#root = "/run/nvidia/driver"
#path = "/usr/bin/nvidia-container-cli"
environment = []
#debug = "/var/log/nvidia-container-toolkit.log"
#ldcache = "/etc/ld.so.cache"
load-kmods = true
#no-cgroups = false
#user = "root:video"
#ldconfig = "@/sbin/ldconfig"
#ldconfig = "/sbin/ldconfig"
ldconfig = "/sbin/ldconfig.real"
[nvidia-container-runtime]
#debug = "/var/log/nvidia-container-runtime.log"Restart docker:
sudo systemctl restart dockerStep 7: starting Plex (in Portainer)
in Portainer you have to add the following parameters in the "Env" tab:
| name: | NVIDIA_DRIVER_CAPABILITIES | Value | compute,video,utility |
| name | NVIDIA_VISIBLE_DEVICES | Value | all |
and in the tab "Runtime & Ressources":
change the "Runtime" Value from runc to nvidia !!!
No Privileged mode and no Init set.
Step 8: try and error:
this is how it worked for me. If you want to check operation you can display GPU load with:
watch -d -n 0.5 nvidia-smi(If nvidia-smi is not installed, do apt install -t buster-backports nvidia-smi)“
or install:
apt install nvtopand use:
nvtopYou can also try/use the docker command line interface (cli) to start Plex:
docker run \
-d \
--name=plex-gpu \
--net=host \
--gpus all \
-device /dev/dri:/dev/dri \
-e PUID=xxxxxx \ #change to your value
-e PGID=yyyyyy \ #change to your value
-e PLEX_MEDIA_SERVER_USER=plex \
-e VERSION="plexpass" \
-v your volumes \ #map your volumes
-v your volumes \ #map your volumes
--restart unless-stopped \
ghcr.io/linuxserver/plex:latestStep 9: get rid of the session limit:
If you want to disable the session limit (my P1000 had a limit of 3) go ahead with this link. It worked for me also:
in detail:
if you don't have git, install it:
apt install gitthen
git clone https://github.com/keylase/nvidia-patch.git nvidia-patchPatch the nvidia driver:
cd nvidia-patch
bash ./patch.shIf you want to rollback:
bash ./patch.sh -rStep 10: Nvidia Power Management:
You need these modifications if you are using autoshutdown as the nvidia driver (or the pci-bus of the card?) is falling of if restarting from hibernate or suspend mode !
You can read more about that in the nvidia documentation:
Chapter 22. PCI-Express Runtime D3 (RTD3) Power Management
Chapter 21. Configuring Power Management Support
Chapter 29. Using the nvidia-persistenced Utility
First of all you need the dedicated nvidia scripts for power management and you have to find them in the nvidia driver install package:
Download the nvidia install package from:
or for the 460 driver the direct link:
Or on the system:
wget http://us.download.nvidia.com/XFree86/Linux-x86_64/460.39/NVIDIA-Linux-x86_64-460.39.runNow do not install the driver (!) - just extract it:
sh NVIDIA-Linux-x86_64-460.39.run --extract-onlyIn the next step you have to search for the following files and copy them to the given directories (I used an ssh client for this)
/etc/systemd/system/nvidia-suspend.service
/etc/systemd/system/nvidia-hibernate.service
/etc/systemd/system/nvidia-resume.service
/lib/systemd/system-sleep/nvidia
/usr/bin/nvidia-sleep.shthen enable the services:
sudo systemctl enable nvidia-suspend.service
sudo systemctl enable nvidia-hibernate.service
sudo systemctl enable nvidia-resume.servicechange nvidia kernel config:
/etc/modprobe.d/nvidia-kernel-common.conf
options nvidia NVreg_PreserveVideoMemoryAllocations=1
options nvidia NVreg_DynamicPowerManagementVideoMemoryThreshold=100
options nvidia NVreg_DynamicPowerManagement=0x02
options nvidia NVreg_EnableMSI=0now make nvidia-sleep.sh executable and update modules:
chmod a+x /usr/bin/nvidia-sleep.sh
sudo update-initramfs -uRemark:
I still use the script files from the 450 driver version and they still work with 460. So I think they are more general and not driver specific. But if you experience problems
with hardware transcoding on a new driver version after a resume maybe you have to extract them fresh from the new driver package.
And I don't know if this final part is really needed, but if you still have issues with resume from suspend you can try to disable active state power management on pcie:
Change/add kernel parameter pcie_aspm=off in /etc/default/grub to:
GRUB_CMDLINE_LINUX_DEFAULT="quiet pcie_aspm=off"and after changing grub:
sudo update-grub
rebootThat was a long way 
Please let me know if there are still issues with htis guide. I'll try to keep it up to date...
Good luck,
Chris
