Skip to content

Instantly share code, notes, and snippets.

@THeK3nger
Last active February 27, 2024 18:42
  • Star 2 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
Star You must be signed in to star a gist
Save THeK3nger/882a31f52bb002dac155ad95529c3680 to your computer and use it in GitHub Desktop.
Unity script for using ElevenLabs TTS service
/**
* An example script on how to use ElevenLabs APIs in a Unity script.
*
* More info at https://www.davideaversa.it/blog/elevenlabs-text-to-speech-unity-script/
*/
using System;
using System.Collections;
using System.Text;
using Newtonsoft.Json;
using UnityEngine;
using UnityEngine.Events;
using UnityEngine.Networking;
public class ElevenlabsAPI : MonoBehaviour {
[SerializeField]
private string _voiceId;
[SerializeField]
private string _apiKey;
[SerializeField]
private string _apiUrl = "https://api.elevenlabs.io";
private AudioClip _audioClip;
// If true, the audio will be streamed instead of downloaded
// Unfortunately, Unity has some problems with streaming audio
// but I left this option here in case you want to try it.
public bool Streaming;
[Range(0, 4)]
public int LatencyOptimization;
// This event is used to broadcast the received AudioClip
public UnityEvent<AudioClip> AudioReceived;
public ElevenlabsAPI(string apiKey, string voiceId) {
_apiKey = apiKey;
_voiceId = voiceId;
}
public void GetAudio(string text) {
StartCoroutine(DoRequest(text));
}
IEnumerator DoRequest(string message) {
var postData = new TextToSpeechRequest {
text = message,
model_id = "eleven_monolingual_v1"
};
// TODO: This could be easily exposed in the Unity inspector,
// but I had no use for it in my work demo.
var voiceSetting = new VoiceSettings {
stability = 0,
similarity_boost = 0,
style = 0.5f,
use_speaker_boost = true
};
postData.voice_settings = voiceSetting;
var json = JsonConvert.SerializeObject(postData);
var uH = new UploadHandlerRaw(Encoding.ASCII.GetBytes(json));
var stream = (Streaming) ? "/stream" : "";
var url = $"{_apiUrl}/v1/text-to-speech/{_voiceId}{stream}?optimize_streaming_latency={LatencyOptimization}";
var request = UnityWebRequest.Post(url, json);
var downloadHandler = new DownloadHandlerAudioClip(url, AudioType.MPEG);
if (Streaming) {
downloadHandler.streamAudio = true;
}
request.uploadHandler = uH;
request.downloadHandler = downloadHandler;
request.SetRequestHeader("Content-Type", "application/json");
request.SetRequestHeader("xi-api-key", _apiKey);
request.SetRequestHeader("Accept", "audio/mpeg");
yield return request.SendWebRequest();
if (request.result != UnityWebRequest.Result.Success) {
Debug.LogError("Error downloading audio: " + request.error);
yield break;
}
AudioClip audioClip = downloadHandler.audioClip;
AudioReceived.Invoke(audioClip);
request.Dispose();
}
[Serializable]
public class TextToSpeechRequest {
public string text;
public string model_id; // eleven_monolingual_v1
public VoiceSettings voice_settings;
}
[Serializable]
public class VoiceSettings {
public int stability; // 0
public int similarity_boost; // 0
public float style; // 0.5
public bool use_speaker_boost; // true
}
}
@firdodev
Copy link

how to use it

@THeK3nger
Copy link
Author

I forgot to link to my original post 😅

In short, though:

  1. You attach this script to an empty game object and fill in the _voiceId and _apiKey fields (the data is in your ElevenLabs account).
  2. Then, from any other script, you can invoke the GetAudio(string) method and receive the result through the AudioReceived event.

For example, this is a minimal client script where I have a text input field and a button. When I click the button, I send the text to ElevenLab and play the result.

public class TestElevenLabsUI : MonoBehaviour
{
    
    public Button sendButton;
    public InputField inputField;
    public ElevenlabsAPI tts;
    
    void Start()
    {
        // Add the PlayClip handler to the ElevenLabsAPI script
        tts.AudioReceived.AddListener(PlayClip);
        
        // Add the Button's onClick handler 
        sendButton.onClick.AddListener( () => {
            tts.GetAudio(inputField.text);
            inputField.text = "";
        });
    }

    public void PlayClip(AudioClip clip)
    {
        AudioSource.PlayClipAtPoint(clip, Camera.main.transform.position);
    }
}

@rafe4
Copy link

rafe4 commented Feb 27, 2024

Hey! Ive been trying to use this but getting 400 bad request, i have another library setup with elevenLabs that is working so i know its not the API key thats wrong, the url seems right too, any clue whats causing the porblem? Many thanks!

@THeK3nger
Copy link
Author

Hi @rafe4! Yes, a 400 Bad Request error makes me think the problem depends on the payload. If you add a Debug.Log(json) after line 60, you should be able to see if the serialized JSON makes sense. On the top of my head, I don't know what may be wrong: maybe the message contains invalid characters?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment