A Guide to get VTube Studio and VSeeFace working on Linux. Tested on Arch Linux

VTube Studio [Arch Linux Guide].md

VTube Studio/OpenSeeFace/VSeeFace Guide (tested on Arch Linux)

Requirements:

• Latest Proton (GE-Proton, Proton-Experimental are both okay)
• git
• wine64
• lutris (winetricks)
• Install system packages for python (3.11.3 is okay): sudo pacman -S python python-pip python-virtualenv

Download and install OpenSeeFace environment:

• git clone https://github.com/emilianavt/OpenSeeFace && cd OpenSeeFace && virtualenv -p python3 venv && source venv/bin/activate && pip install onnxruntime opencv-python pillow numpy
• You need to always start OpenSeeFace in this folder with active env, so you need always: source venv/bin/active
• Start the tracker with:
• python facetracker.py -c 0 -W 1280 -H 720 --discard-after 0 --scan-every 0 --no-3d-adapt 1 --max-feature-updates 900 --ip 127.0.0.1 --port 11573
• This terminal window needs to be always open while using VTube Studio and it will use a lot of your cpu

VTube Studio

• Start VTube Studio from Steam and make sure that you have SteamPlay enabled and is using the latest Proton version.
• On the first start you will get some errors, just clicking them away should be fine.
• To connect VTube Studio with your running OpenSeeFace tracker, you will need to create a file in the following location:
  • ~/.local/share/Steam/steamapps/common/VTube Studio/VTube Studio_Data/StreamingAssets/ip.txt
  • it's content should be the following:

  # To listen for remote connections, change this to 0.0.0.0 or your actual IP on the desired interface.
  ip=0.0.0.0
  
  # This is the port the server will listen for tracking packets on.
  port=11573
  

• When both the tracker and VTube Studio are running, click on Settings in VTube Studio, select the camera and enable the "Autostart cam wih VTS", then click on "Camera on".
• with this your tracking should be working in VTube Studio
• While facetracking is working with this, many features of VTube Studio aren't working with OpenSeeFace

Install VSeeFace (VTube Studio "alternative" in lutris (gui)

• Download VSeeFace
• unzip the .zip you downloaded
• Open Lutris
  • click on the plus in the top left corner and "Add game"
  • In the Game info tab, you need to give it a name (e.g. VSeeFace) and select wine as the runner
  • In the Game options tab, browse for "VSeeFace.exe" as the Executable and it's directory as working directory
  • If you want, you can set the wine-prefix in the same directory or leave it empty for using the system-wine-prefix (~/.wine)
  • Select 64-bit in the Prefix architecture
  • click save
• make sure you have running the facetracker from above
• Start it once, but you will notice that a font (arial) is missing which we will install now using winetricks
  • Close the game and click on right arrow next to the bottle and select "winetricks"
  • select default prefix and then "install font"
  • search and select arial, when it is installed, select OK and then you can close winetricks
  • Start VSeeFace from Lutris, read or skip the tutorial
  • Set Camera on "Network Tracking" and the microphon input you are using (in my case PulseAudio)
• From here on, just take a look at https://www.vseeface.icu/ for more information :)

Sources:

https://www.reddit.com/r/VirtualYoutubers/comments/j8ebuu/how_to_run_vseeface_on_linux/

https://www.vseeface.icu/#running-on-linux-and-maybe-mac

https://github.com/emilianavt/OpenSeeFace

Thanks to everyone who contributed to this.

Update on a few things:

• As of me writing this (22.05.2022) you need to use Python 3.9 for OpenSeeFace to work due to onnxruntime not being available for newer versions yet
• There are three things I noticed with your install command:
  1. You assume the use of Bash or sh for the activation of the venv. Users of Fish (like me) need to use source env/bin/activate.fish
  2. When setting up the venv, it could be a good idea to force the use of Python 3.9
  3. The pip install has an install command in-front of it which will cause it to fail
• I couldn't get VSeeFace to launch at all. Unity had some problems that I was too lazy to look into
• VTube Studio (The version available on Steam, Version 1.19.0 at the time of writing) works almost out of the box with Proton Experimental. Only problem is the face cam detection on my system (Fedora 36). I solved this by doing the following:
  1. Use OpenSeeFace. I modified the command by changing the camera to use (here you need to just try your way through the different devices) and added the flags -s 1 --port 20202. -s 1 hides the output and --port 20202 needs to be set for VTS to find it
  2. For VTS to pick up OSF you need to add a file called ip.txt inside the VTube_Studio_Data/StreamingAssets folder. Example content
  3. Edit the ip and port inside ip.txt to fit your needs. 0.0.0.0 is fine for local usage and for port you need to set the port specified in the launch command for OSF, 20202 in this case.
  4. The order of launch must always be OSF, then VTS. Otherwise, VTS will just do nothing

Source that led me to the ip.txt file: (https://trello.com/c/kbZWaoKN/26-variable-ip-for-osf-tracker)

Thank you for your input! :D
Honestly I forgot that I had created this gist lmao
It is just as you said, with using the ip file it worked pretty good.
I saw this on youtube from a linux vtuber xD

Thank you for correcting me uwu

Update on a few things:

• As of me writing this (22.05.2022) you need to use Python 3.9 for OpenSeeFace to work due to onnxruntime not being available for newer versions yet

• There are three things I noticed with your install command:

  1. You assume the use of Bash or sh for the activation of the venv. Users of Fish (like me) need to use source env/bin/activate.fish
  2. When setting up the venv, it could be a good idea to force the use of Python 3.9
  3. The pip install has an install command in-front of it which will cause it to fail

• I couldn't get VSeeFace to launch at all. Unity had some problems that I was too lazy to look into

• VTube Studio (The version available on Steam, Version 1.19.0 at the time of writing) works almost out of the box with Proton Experimental. Only problem is the face cam detection on my system (Fedora 36). I solved this by doing the following:

  1. Use OpenSeeFace. I modified the command by changing the camera to use (here you need to just try your way through the different devices) and added the flags -s 1 --port 20202. -s 1 hides the output and --port 20202 needs to be set for VTS to find it
  2. For VTS to pick up OSF you need to add a file called ip.txt inside the VTube_Studio_Data/StreamingAssets folder. Example content
  3. Edit the ip and port inside ip.txt to fit your needs. 0.0.0.0 is fine for local usage and for port you need to set the port specified in the launch command for OSF, 20202 in this case.
  4. The order of launch must always be OSF, then VTS. Otherwise, VTS will just do nothing

Source that led me to the ip.txt file: (https://trello.com/c/kbZWaoKN/26-variable-ip-for-osf-tracker)

OpenSeeFace works with Python 3.10 now, though the required packages are different:

pip install ort-nightly opencv-python pillow numpy

I was also able to shave most parameters off of the facetracker.py command (on Ubuntu, at least), leaving only the camera selection and port workaround that was outlined above. In my case, this resulted in:

python facetracker.py -c 2 --port 11573

Another tip: You can open Steam, right-click on VTube Studio, select Properties and change the launch options to something like this:

bash -c "cd <OpenSeeFace repository path>; source <virtual environment path>/bin/activate; python facetracker.py <command arguments> & %command%; jobs -p | xargs kill"

This will automatically start and stop OpenSeeFace whenever VTube Studio opens or closes.

Now that I am back on Linux full time (Thank you Bungie for forcing me to use Windows if I even want to touch Destiny 2) I came back to this to setup VTube Studio again.
The two problems I encountered were:

• ort-nightly couldn't be found by pip with Python 3.11
• OpenSeeFace didn't update to the latest Numpy version (currently 1.24.x), breaking everything.

My solution: Stay with Python 3.9 and the following packages: pillow, opencv-python onnxruntime, numpy==1.23
Everything else is about the same

For convenience, here's the content for a Pipfile (Github didn't want me to attach the file directly) to use with pipenv that has the proper requirements for the problem I mentioned in my last message.

[[source]]
url = "https://pypi.org/simple"
verify_ssl = true
name = "pypi"

[packages]
numpy = "~=1.23"
pillow = "*"
opencv-python = "*"
onnxruntime = "*"

[dev-packages]

[requires]
python_version = "3.9"

Edit: Using pipenv the command for Steam then changes to bash -c "cd <Path to OpenSeeFace directory>; pipenv run python facetracker.py <Arguments> & %command%; jobs -p | xargs kill"

Another edit: @BenKato151 could you please remove the Outdated/Unmaintained from the main file? This is one of the first results a Google search on the topic of Vtubing on Linux. Having that warning, despite having a working solution, will only demotivate people from looking further. The Steam forums for VTube Studio also link here for the topic of running it on Linux

Thanks a lot for your contributions! :)
I initially created this guide for a some friends and myself, didn't know that it helped outside of my own social circle and got more attention XD

I added the unmaintend and outdated info, because I lost interest and I thought that it wasn't needed anymore, but I understand your reasoning and I agree with it.
Currently I am unable (not at home for a while with crappy laptop) to test, verify and update this guide, but I can edit the gist and include your comment.

I will update it properly when I am able to.
Hope this will be a good way to handle it.

Thank you again. :3

For convenience, here's the content for a Pipfile (Github didn't want me to attach the file directly) to use with pipenv that has the proper requirements for the problem I mentioned in my last message.

[[source]]
url = "https://pypi.org/simple"
verify_ssl = true
name = "pypi"

[packages]
numpy = "~=1.23"
pillow = "*"
opencv-python = "*"
onnxruntime = "*"

[dev-packages]

[requires]
python_version = "3.9"

Edit: Using pipenv the command for Steam then changes to bash -c "cd <Path to OpenSeeFace directory>; pipenv run python facetracker.py <Arguments> & %command%; jobs -p | xargs kill"

Another edit: @BenKato151 could you please remove the Outdated/Unmaintained from the main file? This is one of the first results a Google search on the topic of Vtubing on Linux. Having that warning, despite having a working solution, will only demotivate people from looking further. The Steam forums for VTube Studio also link here for the topic of running it on Linux

Bump!
@BenKato151, Remember to update when able!

Okay, I finally had the time and opportunity to get into this again, sorry for taking so long.

Here some things I've looked into:

VTube Studio and OpenSeeFace:

• Starting VTubeStudio from Steam works out of the box with ProtonExperimental - on fresh install I got some errors but no crashes, clicking on "okay" will make them go away. For now I won't look into it any further
• Installing and starting OpenSeeFace is working with python 3.11.3 - dependencies didn't change (onnxruntime is working, ort-nightly wasn't found by pip), but looking at the github repo no commit has been made since May 2022, so i don't know how uptodate the changes differ from last time I worked with it
• once OpenSeeFace is running, it is working with VTube Studio perfectly after turning Camera in the VTubeStudio Settings on - but the ip.txt file in "VTube Studio/VTube Studio_Data/StreamingAssets/" is still needed
• The fix for the VTube Studio Crash after enabling the camera isn't needed anymore

VSeeFace:

• Everything is working just as I initially wrote. nothing major to change or mention here.

I will update the gist with everything you guys contributed and what I've found myself tomorrow or this sunday, depending on how much time I find for myself :D

Thanks a lot!

This is working great for me thanks!

Note that most of the arguments to facetracker.py are the default settings, you can check what you need on an individual basis with python facetracker.py --help which is quite informative.

Any idea on how to connect iPhone to VTS Linux via USB? It seems that wifi works well, but USB is not working on my side and I don't know how to resolve.

Sorry, but I don't think that'll work over USB. Proton/Wine is very specific regarding USB-passthrough and you'd probably have to do a very specific custom config for it

Hey thank you very much.
I found the following installer for VSeeFace with lutris: https://lutris.net/games/vseeface/
No need to do it manually anymore.

You mean "so you need always: source venv/bin/activate"

But yeah I can confirm this works great.