Ivy is a virtual assitant that joins your Amazon IVS (get it? Ivy? IVS?) real-time stage, listens for chat messages that are prefixed with "Hey Ivy", and responds!
- ☒ Build a Virtual Assistant in Unity using a VRoid VRM model
- ☒ Join Amazon IVS Real-Time Stage
- ☒ Join Amazon IVS Chat Room
- ☒ Respond to "Hey Ivy!" chat messages by passing the prompt to Amazon Bedrock
- ☒ Animate model based on response sentiment and display response text in UI
- ☒ Convert response to audio with Amazon Polly
- ☒ Play response audio and sync model's mouth based on audio phonemes
- ☐ Multiple language support?
- ☒ Create and export VR model with VRoid Studio
- ☒ Create Unity project and import VRM (with UniVRM)
- ☒ Broadcast Unity project to real-time stage and connect to Amazon IVS chat room
- ☒ Create architecture (CDK)
- ☒ Stage & Chat room
- ☒ Stage token serverless endpoint
- ☒ Chat token serverless endpoint
- ☒ Invoke Bedrock endpoint
- ☐ Create front-end application
- ☒ Create architecture (CDK)
- ☒ Parse incoming chat messages for prompt
- ☒ Invoke LLM with prompt
- ☒ Animate model "thinking" while model is invoked?
- ☒ Return response, phoneme, sentiment (and image?)
- ☒ Invoke LLM with prompt
- ☒ Display response in UI (typewriter effect)
- ☒ Animate model while "speaking" response and change face based on sentiment
- ☒ Convert response to audio (wav)
- ☒ Play audio in Unity
It always happens, doesn't it... 😆
- ☐ Use stream audio to communicate with Ivy
- ☐ Lofi background music?
☒ Complete all objectives!