> For the complete documentation index, see [llms.txt](https://docs.convai.com/api-docs/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.convai.com/api-docs/plugins-and-integrations/convai-unreal-engine-plugin/overview/what-is-the-convai-unreal-engine-plugin.md).
# What is the Convai Unreal Engine plugin
The Convai Unreal Engine plugin connects Unreal Engine 5 projects to Convai so in-scene actors can speak, listen, reason, and act in real time. A player speaks into a microphone or sends text; the plugin captures input, streams it to Convai for speech recognition and language understanding, generates a response with text-to-speech, and plays it back on the character with synchronized lip sync, facial emotion, and optional in-scene actions. The plugin targets Unreal developers building training simulations, interactive experiences, and games.
### What it includes
The plugin ships with a complete conversation pipeline and a set of opt-in feature add-ons.
#### Conversation pipeline
Always active once connected:
* **Real-time voice input** — microphone capture through `UConvaiAudioCaptureComponent`, streamed to Convai over WebRTC
* **Language understanding and generation** — Convai processes speech and responds in character
* **Text-to-speech playback** — voice generated by Convai, played back through `UConvaiChatbotComponent`
#### Feature add-ons
Opt-in, each configured through Blueprint or the Details panel:
* **Character actions** — characters execute structured in-scene commands (`Move To`, `Follow`, custom) dispatched from Convai
* **Dynamic context** — push live world state and events into character knowledge at runtime
* **Emotion** — Convai infers emotion from conversation and drives blendshape expressions on the character
* **Long-term memory** — characters remember each player across sessions using an end-user ID
* **Narrative design** — trigger scripted conversation branches and sections by name from Blueprint
* **Scene metadata** — tag level actors so characters are aware of and can act on them
* **Vision** — stream camera frames to Convai so characters can describe what they see
* **Gaze attention** — route the object in the player's gaze direction as context to the active character
* **Lip sync** — Convai-provided face animation data drives blendshape mouth movement in sync with speech audio, with selectable `Viseme Based`, `MetaHuman Blendshapes`, `ARKit Blendshapes`, and `CC4 Extended Blendshapes` modes for different rigs
#### Editor tooling
The `ConvaiEditor` module adds an in-editor configuration window for API key setup, the character dashboard browser, and Blueprint graph utilities such as the **Create Convai Action Handler** right-click entry.
### Relationship to Convai
Convai hosts the language model, voice synthesis, narrative design engine, long-term memory store, and character configuration. The plugin is a client-side integration layer: it captures audio or text from the player, streams it to Convai over WebRTC, and delivers the response back to the character's audio and animation pipeline.
The plugin runs client-side integration logic inside the game process, including session routing, audio playback, action handling, dynamic context, gaze attention, and scene metadata. It does not bundle a local language model or Convai decision engine. Character configuration and conversation decisions remain in Convai, so dashboard-side changes do not require recompiling the Unreal project.
### Voice → Convai → character flow
```mermaid
graph LR
MIC["Microphone / text"] --> PC["UConvaiPlayerComponent"]
PC -->|"session proxy"| PProxy["UConvaiConnectionSessionProxy
(player)"]
Chatbot["UConvaiChatbotComponent"] -->|"session proxy"| CProxy["UConvaiConnectionSessionProxy
(character)"]
PProxy --> SS["UConvaiSubsystem
(Convai client)"]
CProxy --> SS
SS -->|"stream"| CV["Convai"]
CV -->|"audio + face data + action sequence"| SS
SS --> CProxy
CProxy --> Chatbot
Chatbot --> AUD["Audio playback"]
Chatbot --> FS["UConvaiFaceSyncComponent
(lip sync)"]
Chatbot --> BH["Blueprint action handlers"]
```
`UConvaiPlayerComponent` captures microphone audio and sends it through a player session proxy. `UConvaiChatbotComponent` holds a character session proxy. `UConvaiSubsystem` owns the underlying Convai client and routes data between those proxies and Convai. The chatbot receives response data and coordinates audio playback, lip-sync processing, and Blueprint action handlers.
### Blueprint-first design
The main gameplay workflows are exposed to Blueprint graphs. C++ access is available but secondary; most projects can build character logic through Blueprint components, events, and callable nodes, then use C++ only for project-specific extensions.
### Requirements
| Requirement | Minimum |
| ------------- | -------------------------------------------------------------------- |
| Unreal Engine | space.vars.unreal\_min\_version |
| Platforms | `Win64`, `Android` |
| Network | Internet connection to Convai |
| API key | Account at space.vars.dashboard\_url |
{% hint style="info" %}
Android requires microphone permission handling. The plugin bundles the `AndroidPermission` engine plugin as a dependency and requests `RECORD_AUDIO` permission automatically when it connects to Convai.
{% endhint %}
The Convai Unreal Engine plugin is available on [Fab](https://www.fab.com/listings/ba3145af-d2ef-434a-8bc3-f3fa1dfe7d5c). Plugin releases are also published on [GitHub](https://github.com/Conv-AI/Convai-UnrealEngine-SDK-V4/releases).
For the full platform and engine version support matrix, see Compatibility and requirements.
{% content-ref url="/pages/3oblPsurIGf7eseuvZPm" %}
[Compatibility and requirements](/api-docs/plugins-and-integrations/convai-unreal-engine-plugin/compatibility-and-requirements.md)
{% endcontent-ref %}
### Next steps
Install the plugin and add your first AI character.
{% content-ref url="/pages/5oPvRJZZEVuIVMYZLYh8" %}
[Getting started](/api-docs/plugins-and-integrations/convai-unreal-engine-plugin/getting-started.md)
{% endcontent-ref %}
To understand the module and component structure before building, see the architecture page.
{% content-ref url="/pages/avDtqiuctK8vS16Q8s7w" %}
[Plugin architecture](/api-docs/plugins-and-integrations/convai-unreal-engine-plugin/overview/plugin-architecture.md)
{% endcontent-ref %}
For a one-screen overview of every feature and links to the corresponding guides, see the feature map.
{% content-ref url="/pages/HcWIzLmISVXTnaagYQMj" %}
[Feature map](/api-docs/plugins-and-integrations/convai-unreal-engine-plugin/overview/feature-map.md)
{% endcontent-ref %}
---
# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.
## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.
Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter:
```
GET https://docs.convai.com/api-docs/plugins-and-integrations/convai-unreal-engine-plugin/overview/what-is-the-convai-unreal-engine-plugin.md?ask=&goal=
```
`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.
Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.