Skip to content

Instantly share code, notes, and snippets.

@emilianavt

emilianavt/BestVTuberSoftware.md

Last active April 27, 2024 19:58
Show Gist options
  • Star 325 You must be signed in to star a gist
  • Fork 9 You must be signed in to fork a gist
  • Save emilianavt/cbf4d6de6f7fb01a42d4cce922795794 to your computer and use it in GitHub Desktop.
Save emilianavt/cbf4d6de6f7fb01a42d4cce922795794 to your computer and use it in GitHub Desktop.
Download ZIP
Best VTuber Software
Raw
BestVTuberSoftware.md

Best VTuber software

This is a list of the most commonly used and relevant vtubing software. The "best" will always be subjective and depend on your specific requirements. Overall, the information in this list is as accurate as I could figure it out, but there might be errors or some details might become out of date. If you find anything that needs to be corrected, please let me know. You can also note it in a comment.

Additional explanations:

  • iPhone means that an iPhone is basically required
  • iFacialMocap support means that tracking data can be received from the iFacialMocap iPhone app
  • VMC protocol means that the application can send and/or receive tracking data from other VMC protocol capable applications, allowing the combination of multiple tracking methods (e.g. VSeeFace receiving VR tracking from Virtual Motion Capture and iPhone/ARKit face tracking from Waidayo)
  • Tobii means that the Tobii eye tracker is supported
  • VR means that VR tracking is used and no camera based tracking is supported
  • Desktop means that mouse and keyboard input is captured and can be used to animate the avatar instead of or in addition to face tracking
  • Leap Motion means that hand and finger tracking through Leap Motion is supported

The order of this list is arbitrary and quite possibly biased, since I develop VSeeFace. Please also note that this list only includes information about tracking, not about other features like transparent background support, model formats and so on.

You can find another overview by Runa Yukiko here.

3D models

Live2D

Other

  • Inochi2D (a free and open source 2D VTuber platform)
  • VPupPr (a free and open source 3D VTuber software with VRM support)

Details for 3D model programs

Application Price Face tracking VR Leap Motion Tobii eye tracker VMC Protocol iFacialMocap Desktop tracking iPhone required Platform Gaze tracking Eyebrow tracking Models Account required
VSeeFace free OpenSeeFace, 66pt no yes no send, receive (from two sources) yes no no Windows yes yes VRM, VSFAvatar (Unity asset bundle) no
Tracking World free - yes no no send, receive no no no Windows no no VRM no
Virtual Motion Capture free/¥300 no yes no yes send (paid only), receive no no no Windows yes (Tobii) no VRM no
VMagicMirror free DlibFaceLandmarkDetector, 17pt no no no no yes yes no Windows no no VRM no
Luppet trial/¥6,000+ ULSee Lite, 30pt no yes no no yes no no Windows no, moves with head no VRM full version requires login
RiBLA Broadcast (β) free MediaPipe no no/MediaPipe no no yes no no Windows/Mac ? ? VRM no
VPupPr/openseeface-gd free OpenSeeFace, 66pt no no no no yes mouse via plugin no Windows/Linux yes yes VRM no
VUP discontinued Faceunity no yes no yes no no no Windows yes maps to expressions VRM yes
SUVA free - yes no yes no no no no Windows yes (Tobii) no Unity asset bundle no
Hitogata free Dlib, custom? no no no no no no no Windows no? yes Character maker, FBX, VRM no
Wakaru free Dlib, 68pt no no no no no no no Windows yes yes VRM no
3tene free/¥2,000 Dlib, 68pt 3tenePRO only yes no no no no no Windows, Mac no no VRM no
VDRAW ¥1,000 Banuba Face AR no no no no no no no Windows ? ? VRM ?
Waidayo free ARKit no no no send no no yes iPhone yes yes VRM no
VKatsu free Dlib, 68pt yes no no no no no no Windows no yes Character maker (restricted use), paid export to Virtual Cast only no
Kalidoface 3D free MediaPipe no no/MediaPipe no no no no no Web ? ? VRM no
VTuber Maker/Live3D free/free trial BaiduFace no no no no no no no Windows ? ? Character maker ($19.99) yes
WebcamMotionCapture trial/$1.99 per month MediaPipe no no/MediaPipe no yes no no no Windows/Mac no? no? VRM ?
Animaze free/various fees VisageTechnolgies, 78pt (was Hyprface) no yes yes yes no (custom app only) yes no Windows yes yes various via auto import (incl. VRM) no
Warudo free MediaPipe/OpenSeeFace no no no yes (multiple senders/receivers) yes keyboard only no Windows yes yes VRM, WarudoMod no

Details for 2D model programs

Application Price Face tracking iPhone required
VTube Studio (Steam) free with watermark/12.49€ OpenSeeFace 66pt, ARKit/ARCore no
PrprLive free/$9.99 DLC IntraFace/GameAnimoji no
FaceRig $14.99+ plus $3.99 (Live2D DLC) ULSee, 66pt no
Animaze free/various fees Hyprface no

Notes

3tene is pronounced "mitene" from Japanese みてね, not "threeteen".

@Leuconoe
Copy link

Leuconoe commented Oct 12, 2023 •
edited

https://meligo.co.kr/index?lang=EN
https://store.steampowered.com/app/2138250/MELIGO_Creator/?beta=0

@DragosStanculescu
Copy link

DragosStanculescu commented Oct 17, 2023 •
edited

Thank you, I haven't really kept track of updates. I've updated the Animaze entry now.

Thank you too! Much appreciated!
Would love to discuss the avatar format with you and its mission. Can we hop on a call sometime?

The main idea with the avatar format is to help artists have a shippable /usable avatar, in a safe format, that they can author themselves, with free tools. But it needs to be a format that cannot be reverse-engineered, ripped/out, and fed into AI art training algorithms easily.
Too many open format model files are being abused by companies and fed into AI training algorithms every day, in some way stealing the artist's future from under their feet, under the disguise of "free formats and tools".

Our mission even back when we started FaceRig in 2014 was to give a customizable face for people who wanted to choose how they appear to the world, and also carve out a safe art market away from large companies' controls and give it "indie feet" for both entertainers and independent artists.
I want to congratulate you on the independent tracker you've created as that definitely further supports this mission.

You probably also know that Animaze, unlike most tools in this space, is on an indie avatar engine that we are developing (for the same reason). We have suspected that a 100% "monopoly" of big company engines like Unity on the avatar world is potentially problematic for the health of the segment, so we insisted on this crazy mission of also developing a totally indie engine for avatars, that:

a. supports as many types of avatars as we humanly can: Live2D, VRM +Vroid, Vrm+PerfectSync, GLB, and they can all sit around a table in multiplayer
b. where all external key tech aspects are secured and commercially licensed, and, once granted/bought/licensed to someone, can't be taken away at the whim of a big company board of directors ( which is the case with many free tech layers currently in the VTubing space- and this year we saw Unity make a move in that direction too).

We are trying to ensure a true safe haven, away from big companies' boards of directors reach.
I feel that your mission is similar, with the main exception that you trust that you can build that safe haven in Unity's courtyard and that they will forever play nice, and there's no risk in giving Unity that much power over it.

Here are some external insights on why that could be problematic, insights were not really visible to everyone back in 2018-2020, but are outlined in this piece by Tom Francis (Gunpoint, Heat Signature, and Tactical Breach Wizards) ) https://www.pentadact.com/2023-09-16-unitys-trap/

The world got a glimpse this year of why we considered a strategical vulnerability, that Unity's John Ricitiello would be granted a metaphorical collar around the neck of every avatar out there. What was eyebrow-raising was both the "free is not really free, was only a gain market share move, and now it is time to monetize" and "AI art is the future" angles.

To be clear I do not fault anyone for not being alarmed by this and using Unity in any way, as overall Unity is a fantastic engine, it has a TON of great features and plugins, and an indie engine can't compete with it across the board on every little thing. Using a big commercial engine is a path that definitely has to be explored, just that while exploring it is good to be aware that the myth of the "free engine" or "free tools" bestowed upon us indie mortals by some large corporations, is often a myth, it often has an asterisk that turns out to be "only free while gaining market share, and/or until a factual monopoly is established".

True avatar independence and freedom, cannot happen in a tech monopoly. Not if the avatars are all "collared" by being dependent on key tech layers from large corporations, that can be yanked away at any time, and where almost everyone has to march to the beat they dictate.

Fortunately, we started the work and were active in the market in this segment super early in 2014, way before most large companies' metaverse/avatar initiatives, and by doing so, did our part in preventing this avatar space from being patent-locked back in 2017-2019, and will do all we can to ensure there is no monopoly that could be closing this space down. Actually, we had been holding this door open for so long and in 2018-2019 we were actually wondering if something else was up, why are only big companies taking a stab at this, when all the indie initiatives will be making a real play for it? The good news is, 2020 changed all that :)

@ahmad88me
Copy link

I created an open source tool that can be found here: https://github.com/ahmad88me/pytoontuber

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment