Requirements:
- Latest Proton (GE-Proton, Proton-Experimental are both okay)
- git
- wine64
- lutris (winetricks)
- Install system packages for python (3.11.3 is okay):
sudo pacman -S python python-pip python-virtualenv
Download and install OpenSeeFace environment:
git clone https://github.com/emilianavt/OpenSeeFace && cd OpenSeeFace && virtualenv -p python3 venv && source venv/bin/activate && pip install onnxruntime opencv-python pillow numpy
- You need to always start OpenSeeFace in this folder with active env, so you need always:
source venv/bin/active
- Start the tracker with:
python facetracker.py -c 0 -W 1280 -H 720 --discard-after 0 --scan-every 0 --no-3d-adapt 1 --max-feature-updates 900 --ip 127.0.0.1 --port 11573
- This terminal window needs to be always open while using VTube Studio and it will use a lot of your cpu
VTube Studio
- Start VTube Studio from Steam and make sure that you have SteamPlay enabled and is using the latest Proton version.
- On the first start you will get some errors, just clicking them away should be fine.
- To connect VTube Studio with your running OpenSeeFace tracker, you will need to create a file in the following location:
~/.local/share/Steam/steamapps/common/VTube Studio/VTube Studio_Data/StreamingAssets/ip.txt
- it's content should be the following:
# To listen for remote connections, change this to 0.0.0.0 or your actual IP on the desired interface. ip=0.0.0.0 # This is the port the server will listen for tracking packets on. port=11573
- When both the tracker and VTube Studio are running, click on Settings in VTube Studio, select the camera and enable the "Autostart cam wih VTS", then click on "Camera on".
- with this your tracking should be working in VTube Studio
- While facetracking is working with this, many features of VTube Studio aren't working with OpenSeeFace
Install VSeeFace (VTube Studio "alternative" in lutris (gui)
- Download VSeeFace
- unzip the .zip you downloaded
- Open Lutris
- click on the plus in the top left corner and "Add game"
- In the Game info tab, you need to give it a name (e.g. VSeeFace) and select wine as the runner
- In the Game options tab, browse for "VSeeFace.exe" as the Executable and it's directory as working directory
- If you want, you can set the wine-prefix in the same directory or leave it empty for using the system-wine-prefix (~/.wine)
- Select 64-bit in the Prefix architecture
- click save
- make sure you have running the facetracker from above
- Start it once, but you will notice that a font (arial) is missing which we will install now using winetricks
- Close the game and click on right arrow next to the bottle and select "winetricks"
- select default prefix and then "install font"
- search and select arial, when it is installed, select OK and then you can close winetricks
- Start VSeeFace from Lutris, read or skip the tutorial
- Set Camera on "Network Tracking" and the microphon input you are using (in my case PulseAudio)
- From here on, just take a look at https://www.vseeface.icu/ for more information :)
Sources:
https://www.reddit.com/r/VirtualYoutubers/comments/j8ebuu/how_to_run_vseeface_on_linux/
https://www.vseeface.icu/#running-on-linux-and-maybe-mac
https://github.com/emilianavt/OpenSeeFace
Thanks to everyone who contributed to this.
OpenSeeFace works with Python 3.10 now, though the required packages are different:
I was also able to shave most parameters off of the
facetracker.py
command (on Ubuntu, at least), leaving only the camera selection and port workaround that was outlined above. In my case, this resulted in:Another tip: You can open Steam, right-click on VTube Studio, select Properties and change the launch options to something like this:
This will automatically start and stop OpenSeeFace whenever VTube Studio opens or closes.