THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
This repo contains instructions for setting up plex on an ESXi virtual machine, using hardware transcoding via a passthrough GPU. While these instructions have only been tested on the stack described below, I believe they should work for other setups that may involve GPU passthrough devices other than an Intel GPU.
Please note that these instructions can be used to migrate an existing server with its own media files.
- Intel i7 7700K - Intel HD 630 iGPU
- Asus Q170S1 Motherboard
- ESXi version 6.7.0 Update 3 (Build 15160138)
- ESXi 6.7 virtual machine running Arch Linux.
- Linux media-server 5.7.7-arch1-1 #1 SMP PREEMPT Wed, 01 Jul 2020 14:53:16 +0000 x86_64 GNU/Linux
- Configuration Parameters:
- vhv.allowPassthru=TRUE
- svga.present=TRUE # Most other guides will tell you that you need to set this to false, but this is not the case.
The following instructions will assume you are performing this setup on an Arch Linux VM, with the desired GPU for hardware transcoding set as a passthrough device.
Run lspci | grep VGA, and look for a line in the output that contains the GPU you have passed through.
❯ lspci | grep VGA
00:0f.0 VGA compatible controller: VMware SVGA II Adapter
03:00.0 VGA compatible controller: Intel Corporation HD Graphics 630 (rev 04)
While I believe this will always be card1, it will be worth checking. You can do this by listing the available drivers:
❯ sudo ls /sys/kernel/debug/dri/
0 1 128 129
And then for each of them, listing the output device names:
❯ sudo cat /sys/kernel/debug/dri/128/name
vmwgfx dev=0000:00:0f.0 unique=0000:00:0f.0
❯ sudo cat /sys/kernel/debug/dri/129/name
i915 dev=0000:03:00.0 unique=0000:03:00.0
In my case, the 129 card corresponds to the iGPU card that I want to use, whereas the 128 driver corresponds to the built-in ESXi VGA card that is used for your web console. This number should correlate to a device file in the /dev/dri/ directly as a /dev/dri/renderDXXX file. In my case, I want to use the /dev/dri/renderD129 device.
With the libva-utils package installed, and the correct drivers installed for vaapi (see Arch wiki, we can test that this works with the vainfo command. Replace the --device command with the device file that you have determined is your card. You should also replace the LIBVA_DRIVER_NAME=iHD value to be the value that corresponds to the drivers that you installed for your GPU.
❯ LIBVA_DRIVER_NAME=iHD vainfo --display drm --device /dev/dri/renderD129
vainfo: VA-API version: 1.8 (libva 2.7.1)
vainfo: Driver version: Intel iHD driver for Intel(R) Gen Graphics - 20.1.1 ()
vainfo: Supported profile and entrypoints
VAProfileNone : VAEntrypointVideoProc
VAProfileNone : VAEntrypointStats
VAProfileMPEG2Simple : VAEntrypointVLD
VAProfileMPEG2Simple : VAEntrypointEncSlice
VAProfileMPEG2Main : VAEntrypointVLD
VAProfileMPEG2Main : VAEntrypointEncSlice
VAProfileH264Main : VAEntrypointVLD
VAProfileH264Main : VAEntrypointEncSlice
VAProfileH264Main : VAEntrypointFEI
VAProfileH264Main : VAEntrypointEncSliceLP
VAProfileH264High : VAEntrypointVLD
VAProfileH264High : VAEntrypointEncSlice
VAProfileH264High : VAEntrypointFEI
VAProfileH264High : VAEntrypointEncSliceLP
VAProfileVC1Simple : VAEntrypointVLD
VAProfileVC1Main : VAEntrypointVLD
VAProfileVC1Advanced : VAEntrypointVLD
VAProfileJPEGBaseline : VAEntrypointVLD
VAProfileJPEGBaseline : VAEntrypointEncPicture
VAProfileH264ConstrainedBaseline: VAEntrypointVLD
VAProfileH264ConstrainedBaseline: VAEntrypointEncSlice
VAProfileH264ConstrainedBaseline: VAEntrypointFEI
VAProfileH264ConstrainedBaseline: VAEntrypointEncSliceLP
VAProfileVP8Version0_3 : VAEntrypointVLD
VAProfileVP8Version0_3 : VAEntrypointEncSlice
VAProfileHEVCMain : VAEntrypointVLD
VAProfileHEVCMain : VAEntrypointEncSlice
VAProfileHEVCMain : VAEntrypointFEI
VAProfileHEVCMain10 : VAEntrypointVLD
VAProfileHEVCMain10 : VAEntrypointEncSlice
VAProfileVP9Profile0 : VAEntrypointVLD
VAProfileVP9Profile2 : VAEntrypointVLD
From my experience, Plex does not do a good job of picking the correct display device to use for hardware decoding. Since the ESXi VGA device appears first, I believe Plex attempts to use it for decoding. Other guides that I have seen have gotten around this by setting the VM config svga.present to FALSE. This approach was not acceptable for me due to disk encryption that I use, that requires me to enter the password on boot in the console. Disabling the SVGA device in the VM would prevent me from being able to boot my machine. Instead, we can make use of Docker to run Plex with a subset of the video cards made available to it. Since Docker makes use of Linux namespaces under the hood, I believe this would be possible to achieve without Docker, but I find Docker makes the entire setup much easier.
After installing docker and docker-compose if you don't already have them installed, you should create a directory within your VM with the following file structure. For the purpose of these instructions, I will assume this directory is stored at /home/plex.
/home/plex/config/
/home/plex/docker-compose.yml
We will make use of a pre-built docker image for plex to run our server. For the docker-compose.yml file, you can adapt the following template.
---
version: "3.0"
services:
plex:
image: linuxserver/plex
container_name: plex
network_mode: host
environment:
- PUID=1001
- PGID=1001
- VERSION=public
- UMASK_SET=022
volumes:
- /home/media:/media
- /home/plex/config:/config
devices:
- "/dev/dri/card1:/dev/dri/card0"
- "/dev/dri/renderD129:/dev/dri/renderD128"
restart: unless-stopped
In filling out this template, the following information may be useful:
- The
/configroot in the docker image is the Plex application's root directory. If you already have a plex server running, you can instead map its root directory intoconfig/Library/Application\ Support/Plex\ Media\ Serverin the docker image in order to not lose your existing configuration. For an existing Arch installation, its application files will exist at/var/lib/plex/Plex\ Media\ Server. You could override the config volume mapping in the template with- /var/lib/plex/Plex\ Media\ Server:config/Library/Application\ Support/Plex\ Media\ Server. You may need to create theconfig/Library/Application\ Supportpath before this can be done. - The devices section in the template is used to hide the svga device from Plex. In my case, we map the 129 render device to 128 in the container, and also map the corresponding card as well. I believe card0 will always correspond to device 128, and card1 will correspond to 129. In this case, we also want to map card1 to card0 within the docker image.
- Add as many volume mappings as needed to map your media into the plex docker container. As an example, if all of your media exists under /home/media on your Arch Linux VM, you would add a mapping of /home/media:/media in the template as above.
- The PUID and PGID should be set to a user id and group id that has access to your media files in the Arch VM. If you do not set these correctly, the plex run-as-user will not be able to access your media files within the docker container.
With the config all setup, the only remaining step is to start the container. If everything is setup correctly, we can do this by running docker-compose up -d.
In the folder that contains your docker-compose.yml file, run docker-compose stop.
If everything has gone to plan, with plex up and running, you should be able to setup the plex server by accessing the plex server at http://localhost:32400/web/index.html. Note that plex will need you to access it from localhost when first signing into the server in order to claim ownership of it. You may need to setup some ssh portforwarding in order to do this from your remote machine. You can do this with the following command, and then by going to the link provided:
ssh -L 32400:127.0.0.1:32400 <YOUR_VM_IP>
With plex now setup and running, ensure you have hardware transcoding turned on when available, and then try playing a video with a conversion turned on to a quality lower than the video source. Open a separate tab on plex, and go to the server dashboard. If you expand the video that is being played, you should see Transcoding (HW) in the video transcoding section. If you do not see the (HW) text, hardware transcoding is not working.
While I cannot possibly outline all problems you may face, the easiest way to narrow down your issue is to turn on debug logs in plex, restart the server, and to then check the logs. In this case, this would mean running docker-compose stop && docker-composet up -d, followed by attempting to play a video again with a conversion, and then looking at the log file at config/Library/Application\ Support/Plex\ Media\ Server/Logs/Plex\ Media\ Server.log. Since Plex is using ffmpeg under the hood, look for log lines containing FFMPEG to see if there's any information there.
Plex actually has a built in preference that can be set to override the Libva driver name. Adding VaapiDriver="<DRIVER_NAME>" to your Preferences.xml file will allow you to do this [source].