> For the complete documentation index, see [llms.txt](https://docs.convai.com/api-docs/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.convai.com/api-docs/plugins-and-integrations/convai-unity-sdk/overview/what-is-the-convai-unity-sdk.md).

# What is the Convai Unity SDK

The Convai Unity SDK connects Unity characters to Convai so they can speak, listen, reason, and act in real time. A player speaks into a microphone; the SDK captures audio, streams it to Convai for speech recognition and language understanding, generates a response with text-to-speech, and plays it back on the character with synchronized lip sync, facial emotion, and optional in-scene actions. The SDK targets Unity developers building training simulations, interactive experiences, and games.

### What it includes

The SDK ships with a complete conversation pipeline and a set of opt-in feature modules.

#### Conversation pipeline

Always active once connected:

* **Real-time voice input** — microphone capture, streaming speech recognition
* **Language understanding and generation** — Convai processes and responds in character
* **Text-to-speech** — voice generated by Convai, played back through Unity audio

#### Feature modules

Opt-in, each added as a Unity component:

* **Lip sync** — real-time blend shape mouth animation; supports ARKit, MetaHuman, and CC4 Extended maps
* **Emotion** — maps Convai emotion signals to facial blend shapes or Animator parameters
* **Actions** — character executes structured in-scene commands dispatched by Convai
* **Long-term memory** — character remembers each player across separate sessions
* **Narrative design** — trigger-based story section progression tied to conversation flow
* **Vision** — character sees through a Unity camera, webcam, or Meta Quest passthrough
* **Dynamic context** — inject runtime state and events into the character's knowledge at any time

#### Utilities

Optional helpers that run entirely in Unity without Convai communication:

* **Dialogue animation** — four-layer animator stack driving body and head movement during speech
* **Gaze and attention** — eye and head gaze blended toward focus targets and conversation partners

#### Editor tooling

Project Settings API key configuration, scene setup menu, Scene Validator, and custom Inspectors for every SDK component.

### Voice → Convai → Character flow

```mermaid
graph LR
    MIC[Microphone] --> RM[ConvaiRoomManager]
    RM --> |Voice stream| CLOUD[Convai]
    CLOUD --> |Audio + metadata| CHAR[ConvaiCharacter]
    CHAR --> AUD[Audio playback]
    CHAR --> MOD[Feature modules]
```

`ConvaiRoomManager` handles the streaming connection to Convai. `ConvaiCharacter` receives the response — audio, transcript, emotion signals, and action commands — and routes each to the appropriate module or output.

### Requirements

| Requirement     | Minimum                                                        |
| --------------- | -------------------------------------------------------------- |
| Unity version   | <code class="expression">space.vars.unity\_min\_version</code> |
| Render pipeline | Built-in, URP, or HDRP                                         |
| Platform        | Windows, macOS, Linux, Android, iOS, Meta Quest, WebGL         |
| Network         | Internet connection to Convai                                  |
| API key         | Free account at [convai.com](https://www.convai.com/)          |

{% hint style="info" %}
The sample scenes use URP. If your project uses the Built-in render pipeline, the samples require minor material reassignment. The SDK itself works with all three pipelines.
{% endhint %}

The Convai Unity SDK is available on the [Unity Asset Store](https://assetstore.unity.com/packages/tools/behavior-ai/npc-ai-engine-dialog-actions-voice-and-lipsync-convai-235621).

For the full platform and Unity version support matrix, see Compatibility and requirements.

{% content-ref url="/pages/ijdGXp5AdxHr05KBoBHM" %}
[Compatibility and requirements](/api-docs/plugins-and-integrations/convai-unity-sdk/compatibility-and-requirements.md)
{% endcontent-ref %}

### Next steps

Install the SDK and add your first character.

{% content-ref url="/pages/1d9fa274483952ca377014dd2a4fb49c2f3e80a1" %}
[Getting started](/api-docs/plugins-and-integrations/convai-unity-sdk/getting-started.md)
{% endcontent-ref %}

To understand the system architecture before setting up, see the architecture page next.

{% content-ref url="/pages/E0sHZRYpXvCTPJiLrAH4" %}
[Convai Unity SDK architecture](/api-docs/plugins-and-integrations/convai-unity-sdk/overview/convai-unity-sdk-architecture.md)
{% endcontent-ref %}


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter:

```
GET https://docs.convai.com/api-docs/plugins-and-integrations/convai-unity-sdk/overview/what-is-the-convai-unity-sdk.md?ask=<question>&goal=<endgoal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.