Convai Unity SDK

Add real-time AI characters to Unity training simulations, interactive experiences, and games — with speech, emotion, actions, and persistent memory.

The Convai Unity SDK connects Unity projects to Convai, bringing conversational AI into training simulations, interactive experiences, and games. Characters process speech in real time, respond with natural language, and synchronize lip movement, facial emotion, and body animation — all configurable per project. Each feature is a self-contained module: add only what your project needs, in the order that fits your workflow.

New to the SDK? Follow Getting Started to install the package, configure your API key, and run your first AI character.

Learn the SDK

Overview What the SDK is, how it is structured, and when to use each module.

Compatibility and requirements Unity versions, render pipelines, platform support, and network requirements.

Core concepts Session lifecycle, turn-taking modes, and the SDK event system.

Features

Each feature is a self-contained module you opt into.

Actions Characters execute structured in-scene commands dispatched by Convai.

Emotion Map Convai emotion signals to facial blendshapes or Animator parameters.

Long-term memory Characters remember each player across separate sessions.

Vision Characters see through a Unity camera, webcam, or Meta Quest passthrough.

Dynamic context Inject runtime state and events into character knowledge at any time.

Narrative design Trigger-based story section progression tied to conversation flow.

Scene metadata Characters automatically read contextual information about scene objects.

Utilities

Helper modules that run entirely within Unity — no Convai communication required.

Dialogue animation Four-layer animator stack driving body and head movement during speech.

Gaze and attention Eye and head gaze blended toward focus targets and conversation partners.

Reference and guides

UI and presentation Transcript UI, subtitle modes, notification system, and settings panel.

Scripting reference Full API reference for session events, character events, and the transcript system.

Platform guides WebGL, Android, iOS, and Meta Quest deployment guides.

Advanced topics Custom providers, performance, and SDK extension points.

Troubleshooting Common failure modes, diagnostic steps, and known issues.

Latest release

v4.2.0 introduces Structured Actions for precise in-scene command dispatch, Meta Quest passthrough vision for AR character awareness, runtime turn-taking mode switching, and expanded dynamic context for richer NPC knowledge.

Next steps

Install the Convai Unity SDK and connect your first AI character using the Getting Started section. If you are evaluating the SDK before installing, review Compatibility and requirements to confirm platform and Unity version support.

Getting started