What is the Convai Unity SDK

Real-time conversational AI characters for Unity — voice pipeline, opt-in feature modules, supported platforms, and minimum requirements.

The Convai Unity SDK connects Unity characters to Convai so they can speak, listen, reason, and act in real time. A player speaks into a microphone; the SDK captures audio, streams it to Convai for speech recognition and language understanding, generates a response with text-to-speech, and plays it back on the character with synchronized lip sync, facial emotion, and optional in-scene actions. The SDK targets Unity developers building training simulations, interactive experiences, and games.

What it includes

The SDK ships with a complete conversation pipeline and a set of opt-in feature modules.

Conversation pipeline

Always active once connected:

• Real-time voice input — microphone capture, streaming speech recognition

• Language understanding and generation — Convai processes and responds in character

• Text-to-speech — voice generated by Convai, played back through Unity audio

Feature modules

Opt-in, each added as a Unity component:

• Lip sync — real-time blend shape mouth animation; supports ARKit, MetaHuman, and CC4 Extended maps

• Emotion — maps Convai emotion signals to facial blend shapes or Animator parameters

• Actions — character executes structured in-scene commands dispatched by Convai

• Long-term memory — character remembers each player across separate sessions

• Narrative design — trigger-based story section progression tied to conversation flow

• Vision — character sees through a Unity camera, webcam, or Meta Quest passthrough

• Dynamic context — inject runtime state and events into the character's knowledge at any time

Utilities

Optional helpers that run entirely in Unity without Convai communication:

• Dialogue animation — four-layer animator stack driving body and head movement during speech

• Gaze and attention — eye and head gaze blended toward focus targets and conversation partners

Editor tooling

Project Settings API key configuration, scene setup menu, Scene Validator, and custom Inspectors for every SDK component.

Voice → Convai → Character flow

ConvaiRoomManager handles the streaming connection to Convai. ConvaiCharacter receives the response — audio, transcript, emotion signals, and action commands — and routes each to the appropriate module or output.

Requirements

Requirement	Minimum
Unity version	2023.1 or later
Render pipeline	Built-in, URP, or HDRP
Platform	Windows, macOS, Linux, Android, iOS, Meta Quest, WebGL
Network	Internet connection to Convai
API key	Free account at convai.com

The sample scenes use URP. If your project uses the Built-in render pipeline, the samples require minor material reassignment. The SDK itself works with all three pipelines.

The Convai Unity SDK is available on the Unity Asset Store.

For the full platform and Unity version support matrix, see Compatibility and requirements.

Compatibility and requirements

Next steps

Install the SDK and add your first character.

Getting started

To understand the system architecture before setting up, see the architecture page next.

Convai Unity SDK architecture