Ctrlk
PlaygroundExperienceVideosPricingForumStatus
youtube
For the complete documentation index, see llms.txt. This page is also available as Markdown.
RSS feed

Release notes

Version history and notable changes for the Convai Unreal Engine plugin, covering public 4.x beta releases and the full 3.x history.

Track changes to the Convai Unreal Engine plugin across releases. The current release is 4.0.0-beta.27, supporting Unreal Engine 5.0 through 5.8.

This plugin is in beta. Beta releases may include breaking changes between minor versions. The plugin manifest identifies the current package as 4.0.0-beta.27.

4.0.0-beta.27

Breaking Blueprint change

  • Invoke Speech and Invoke Narrative Design Trigger no longer expose the Generate Actions and Replicate On Network pins. Both now follow the chatbot's own configuration. Existing Blueprints that wired the removed pins need a node refresh before they compile.

  • Invoke Speech gained a Delivery option, so a line can wait for a quiet moment instead of interrupting, and its message is now heard exactly once instead of accumulating in the conversation context.

Actions

  • The bundled BP Convai Chatbot Component ships with the Escort action ready to use, so a character can lead a player or another character to a destination without rebuilding the Blueprint handler. The unused Wait for Action entry is disabled.

  • Improved move-task cancellation during level shutdown and garbage collection, so a cancelling task no longer reaches for navigation components that Unreal is already destroying.

Spatial awareness

  • Characters can give directions from the player's point of view. Every described object now also states where it sits relative to the player's camera, so "where is it?" gets an answer that matches what the player sees. Controlled by Describe From Player Perspective under Project Settings > Plugins > Convai, in the Spatial Awareness category, and on by default.

Audio and animation

  • Fixed microphone tests being a no-op while a conversation was live. Recording now works during an open stream, the character does not hear the test recording, and the project's mute setting is restored afterwards.

  • Body and face mesh detection now only considers meshes with a running animation Blueprint, so hair, cloth, and static props can no longer be mistaken for a character's body or face.

Other

  • Added Blueprint-accessible Knowledge Bank operations, and fixed REST requests that use a custom Convai server URL.

  • The debug overlay always shows the full picture: the exact surroundings sentences the character receives, plus the FACTS and EVENTS sections, are now visible in every build.

4.0.0-beta.26

Movement

  • Added the reusable Convai Move To and Convai Escort Blueprint nodes. An escorting character can lead someone to a destination, wait when they fall behind, resume when they catch up, and handle a destination that is already nearby.

  • Improved goal resolution and merged-object navigation, so a movement task picks a reachable destination and distinguishes arriving from already being there.

Actions

  • Added cooperative action-plan cancellation. A character can change course while an action is running, and a custom Blueprint action can clean up before its replacement begins.

  • Improved parameterized actions: named values, spaces, punctuation, localized choices, and object-style responses are handled more reliably, and large action lists and multibyte text are no longer cut off by fixed-size buffers.

Other

  • Improved realtime compatibility by reporting the supported protocol and client version correctly, recognizing current lifecycle messages, and forwarding interaction IDs.

  • Added the authenticated editor route the optional Convai Pak Manager needs, and fixed linking for external editor plugins that resolve their own Convai services.

  • Updated the bundled AgentSkills and action documentation for the new movement, escort, cancellation, and response behavior.

4.0.0-beta.25.1

  • Fixed the plugin failing to load on Unreal Engine 5.8 installed from the Epic Games Launcher, where opening a project reported Plugin 'ConvAI' failed to load because module 'ConvaiToolset' could not be loaded. Projects now open normally. Unreal Engine 5.0 through 5.7 was unaffected and needs no changes.

  • Strengthened release testing: every update is now automatically verified to open cleanly in the Unreal editor on UE 5.6, 5.7, and 5.8 before it ships.

4.0.0-beta.25

Movement and scene awareness

  • Added movement awareness for scene objects, covering direction, stop detection, five sensitivity levels, optional movement states, and configurable start and stop responses.

  • Added Movement Points and named destinations, so a character can choose a reachable approach, recognise that it has already arrived, and react when a path opens or closes.

  • Reduced prompt repetition across duplicate objects and spatial facts, and improved action goal resolution for Blueprint movement.

Dynamic context

  • Added delivery controls for live context updates. A responsive event can wait for a natural pause, send immediately when timing matters, or update the character silently.

  • Improved tracked properties with nested object paths, aliases, Blueprint-rename recovery, clearer Details panels, and more concise change wording.

Actions

  • Added the experimental Remind Self and Watch Property actions, which help a character continue a multi-step task and react to the next meaningful property change.

Tooling

  • Added the Convai Debug Overlay for inspecting surroundings, context states, reachability, tracked properties, and action results during development.

Migration note

  • Goal fields and the Resolve Goal Location pins were renamed in this release. Refresh affected Blueprint nodes after upgrading.

4.0.0-beta.24

AI coding agents

  • Added MCP (Model Context Protocol) support on Unreal Engine 5.8, letting an AI coding agent work directly in the editor.

  • Added a setup workflow for connecting an agent: Claude Code, Cursor, VS Code, Gemini, or Codex.

  • Added the Convai Toolset, a set of AI-callable editor actions so an agent can set up and extend Convai characters instead of hand-editing Blueprints: SetupConvaiCharacter, SetupConvaiPlayer, and SetupConvaiPawnMovement wire a MetaHuman and the player for conversation and navigation; AddNavMeshVolumeForCurrentLevel makes the level navigable; AddConvaiAction and CreateConvaiActionHandler declare actions and generate their handler events; and SetBlueprintPropertyAndPropagate edits a property and pushes the change onto characters already placed in the level.

  • Added Convai AgentSkills, an in-editor knowledge base an agent reads to use Convai correctly, covering project and character setup, conversation, actions, animation, dynamic context, gaze, scene awareness, narrative, player input, vision, and expressiveness.

Compatibility

  • The plugin still compiles across UE 5.0 through 5.8. The new tooling is editor-only, has no runtime dependency, and activates on UE 5.8 and later.

4.0.0-beta.23-hotfix

  • Packaging: the plugin now ships to Fab as a source-only submission, with no Unreal-generated Binaries or Intermediate folders, matching Fab's required plugin file structure.

  • Packaging: dropped the precompiled Installed flag from the plugin descriptor, so Fab compiles the plugin from source for each engine version.

  • Packaging: every module now declares a PlatformAllowList, as Fab requires.

4.0.0-beta.23

  • Added support for Unreal Engine 5.8.

  • Fixed an Android microphone capture issue.

  • Improved facial-animation stability with thread-safe access to blendshape data.

4.0.0-beta.22

Spatial awareness

  • Added a spatial-awareness subsystem that gives each chatbot a continuous sense of its surroundings, delivered to the character as plain-language facts. It describes nearby objects, other characters, and the player by distance band, direction, and facing, gated by line of sight.

  • Added inter-entity relations, such as "on top of the pressure plate" or "to the left of the fountain", anchored to the nearest higher-priority neighbour.

  • Added a master enable plus line-of-sight, distance-band, and relation controls under Project Settings > Plugins > Convai, in the Spatial Awareness category, with per-chatbot surroundings and relations toggles.

  • Retired the per-object proximity mechanism. Spatial awareness replaces it.

Scene objects

  • Added the ability to merge identically named Convai Object components into a single logical object, so a pile of crates reads to the character as one thing. Opt in per component with Merge With Same-Named Objects, and keep separate sets apart with Merge Group Index. A merged object exposes one position, one description, and one collective gaze highlight.

  • Added a global Duplicate Name Suffix Style setting, Numeric or Alphabetical, for disambiguating same-named objects that are not merged.

Dynamic context

  • Added ephemeral context events: Add Context Event gained a bEphemeral option that nudges the character exactly once on the next update without persisting it.

  • SetObjectInAttention now emits a one-flush attention cue by default, so gazing at an object tells the character once without leaving a lingering state. The call also logs a warning explaining why it had no effect, whether actions are disabled on the chatbot, the object is missing from the environment list, or the object is missing from the connect-time action config.

Other

  • Added FConvaiAvatarInfo: the get-characters request now returns full character details rather than only character IDs.

  • Fixed pointing animations so they play and aim correctly from character-issued point actions.

  • Fixed step-onto arrival, so a character standing on a wide object's footprint is treated as having reached it instead of never registering arrival for objects wider than its acceptance radius.

  • Fixed spatial-awareness and duplicate-name settings being ignored at runtime; toggling them in Project Settings now takes effect without an editor restart.

  • Fixed one-flush ephemeral context events being lost while the chatbot was disconnected; the flush is deferred until reconnect.

  • Fixed an audio resampler divide-by-zero that could occur when an invalid sample rate was reported.

  • Fixed the Convai editor update check so it correctly detects V4 releases, and fixed editor theme initialization.

4.0.0-beta.21

Animation and gestures

  • Overhauled the bundled MetaHuman animations.

  • Added gesture and pointing animations triggered automatically from LLM-issued character actions.

  • Added replicated PointAtTarget alongside LookAtTarget on the chatbot component.

Object awareness and gaze

  • Added UConvaiObjectComponent for tagging actors and exposing their live properties as context to the chatbot.

  • Added gaze-driven attention: the actor under the player's gaze becomes the object in attention, with a silhouette highlight and an on-screen cursor.

  • Added proximity-driven state: object components report when they are reachable from a chatbot's pawn so the bot can react.

Actions

  • Added "Create Convai Action Handler" right-click entry in Blueprint graphs for scaffolding typed action handlers.

  • Added "Setup Convai Pawn Movement" content-browser action that seeds navigation defaults on any pawn Blueprint.

  • Added component and socket targeting plus per-entry movement overrides for actions.

  • Improved action completion: handlers can wait for bot speech to finish before the next action, post a dynamic-context event in the same call, and abort an unrecoverable sequence cleanly.

  • Added fuzzy matching for enum and choice action parameters so minor LLM spelling drift no longer drops the action.

  • Switched the action wire format to {name} braces with more reliable response parsing.

  • Enabled actions on by default in the chatbot environment.

Audio and lip sync

  • Added VAD (voice activity detection) parameters in project settings and the connect request.

  • Improved tail-of-utterance audio handling to reduce clipping on soft trailing consonants.

  • Improved lip-sync stability with a starvation fallback when frames briefly stop arriving.

  • Fixed microphone hot-swap so switching device mid-session no longer requires restarting the stream.

Other improvements

  • Added Convai auth caching so subsequent connects don't re-authenticate from scratch.

  • Added GetBodyAndFaceSkeletalMeshComponents helper for resolving MetaHuman and CC5 rigs.

Bug fixes and compatibility

  • Fixed Android crash on app shutdown.

  • Fixed several UE 5.0 / 5.4 / 5.5 / 5.7 compilation and compatibility issues so the plugin builds clean across all supported engine versions.

  • Disabled the Convai editor UI on UE 5.1 and earlier (the property-binding editor feature it relies on isn't available there).

  • Regenerated the gaze-highlight material under UE 5.0 so the asset loads across all supported engine versions.

4.0.0-beta.19

  • Actions integration with WebRTC: chatbot now sends action_config at connect when bEnableActions is on, and parses structured action-response payloads into typed action sequences.

  • Refactored Environment from UConvaiEnvironment UObject to FConvaiEnvironmentData USTRUCT. Granular Add/Remove/Clear methods on the chatbot for Objects, Characters, and Actions; debounced scene-metadata flush; bFlushImmediately advanced opt-in for time-critical updates.

  • Split legacy MainCharacter into ConversationPartner (replicated, scene-aware) and LookAtTarget (animation-only). New bAutoFillConversationPartnerFromPlayer toggle populates the partner from the first registered Convai Player component automatically.

  • New SetObjectInAttention rides the dynamic-context flush and auto-registers the entry in Environment.Objects if missing.

  • Added bEnableActions chatbot toggle. Environment.Actions seeded by default with Move To, Follow, Stop Moving, and Wait For.

  • Actions V2 — typed action templates: FConvaiAction with structured Parameters typed via EConvaiActionParamType. Optional Connector for compound actions, Choices for fixed-list constraints, or EnumType to draw choices from a UENUM.

  • Unified action-result shape: FConvaiResultAction.Parameters as TMap<FString, FConvaiResultParam>. Legacy RelatedObjectOrCharacter and ConvaiExtraParams kept as deprecated mirrors.

  • New Blueprint-pure accessors: Get First Param, Get Param, Get Param Type, Get Param As String/Number/Bool/Ref/Byte, Has Param.

  • HandleActionCompletion gains optional AdditionalNote and ShouldRespond. New AbortActionSequence accepts optional EventText and clears the queue without retrying.

  • Removed the EnableNewActionSystem global toggle — the action queue path is now always on.

  • Backward compatibility: legacy UConvaiEnvironment and UConvaiActionContext resurrected as deprecated shims so older Blueprint graphs keep compiling with deprecation warnings.

4.0.0-beta.18

  • Added Android packaging support.

  • Added dynamic context batching and logging for chatbot interactions to reduce redundant updates.

  • Added starvation blending and playable frame checks for improved lip sync stability.

  • Added command-line overrides for lip sync animation parameters and simulation freeze functionality.

  • Added custom parameter handling and client version retrieval to ConvaiUtils.

  • Added reset idle timer and user idle warning support.

  • Enhanced component resolution in the FaceSync Animation Node to include the parent actor when searching for UConvaiChatbotComponent.

  • Renamed RunLLM parameter to ShouldRespond for improved API clarity in context update methods.

  • Updated lip-sync timing behavior for better audio/animation alignment. The current source default for LipSyncTimeOffset is 0.02 seconds (20 ms).

  • Updated dynamic-context debounce behavior for improved batching consistency. The current source setting is ContextDebounceWindow, with a default of 0.5 seconds.

  • Refactored narrative trigger handling to streamline context processing.

4.0.0-beta.17

  • Optimized lip sync computational performance in MetaHuman Face Animation Blueprint.

  • Improved audio playback time estimation and removed deprecated FSoundSource caching.

  • Enhanced audio finish detection with tolerance to avoid floating-point rounding edge cases.

  • Added performance timing statistics for audio streamer and face sync components in debug mode.

  • Added detailed audio playback debug logging.

  • Improved audio capture cleanup to ensure safe resource release during component destruction and end play.

  • Fixed potential dangling references in async tasks by switching to weak pointers.

  • Fixed re-entrant callbacks during Convai client disconnection.

  • Fixed crash caused by cross-object delegate cleanup in BeginDestroy for chatbot and player components.

  • Fixed player transcriptions not being appended properly.

  • Added emotions provider plumbing.

  • Fixed ConvaiConnectionConfig struct layout mismatch with DLLs.

4.0.0-beta.16

  • Added Android support.

4.0.0-beta.15

  • Fixed lip sync slow receival.

4.0.0-beta.14

  • Added Update Context functionality.

  • Updated OutputFPS to 60 for improved lip sync output.

  • Improved lip sync audio handling and cleaned up unused variables.

  • Improved audio playback time calculation accuracy and simplified play voice data flow.

4.0.0-beta.13

  • Added connection indicator UI widget.

  • Fixed attendee disconnect state notification handling.

  • Fixed editor UI to show changelog only for installed plugin version.

  • Improved connection manager reliability and state handling.

  • Fixed audio not being sent when connection is in orphan state.

  • Improved server event dispatching.

  • Enhanced logging for better debugging.

4.0.0-beta.12

  • Deprecated old ConvaiBaseCharacter and ConvaiBasePlayer Blueprint classes.

  • Added connection indicator widget to show the connection state.

  • Added LLM start/stop events (OnLLMStarted, OnLLMStopped) to ConvaiChatbotComponent.

  • Added emotions support.

  • Added toggle STT functionality.

  • Improved lip sync at the end of response.

  • Fixed lip sync desync corner case due to taking frames from previous response.

  • Fixed PIE crash when recording audio at the same time as starting a new connection.

  • Fixed minimum buffer duration updated to 0.2 seconds.

4.0.0-beta.11

  • Improved compatibility with CC5 Reallusion characters.

4.0.0-beta.10

  • Fixed DLL loading issue.

  • Fixed crash in the Convai Dashboard.

  • Added support for Blueprint-only projects.

4.0.0-beta.9

  • Improved character interrupt handling.

  • Improved lip sync accuracy.

  • Added connection state functions to ConvaiSubsystem and ConvaiChatbotComponent.

4.0.0-beta.7

  • Fixed corruption of some transcription characters.

4.0.0-beta.6

  • Added ARKit blendshapes lip sync support.

  • Updated default LipSyncMode to BS_MHA (MetaHuman) in ConvaiFaceSyncComponent.

  • Added RequiresPrecomputedFaceData method to ConvaiConnectionInterface.

  • Fixed blendshape selection logic to use ARKit names conditionally.

4.0.0-beta.5

  • Updated BlendShapesNames to new naming convention for ARKit.

4.0.0-beta.4

  • Implemented bulk NeuroSync blendshape handling with stats logging.

  • Added audio frame tracking and improved voice handling in ConvaiAudioStreamer and ConvaiChatbotComponent.

  • Added end user ID functionality and device unique ID support.

  • Fixed voice playback overlap by stopping voice when audio starts.

  • Added MetaHuman control names support.

  • Fixed editor freezing issues.

  • Fixed Linux library linking issues.

  • Vision optimization improvements.

4.0.0-beta.3

  • Added UE 5.7 support.

  • Added Blueprint-only project support.

4.0.0-beta.2

  • Added editor login screen and dashboard.

  • Added better echo and noise cancellation.

  • Added initial viseme-based lip sync support using the FaceSync component.

  • Added gamma correction to vision.

  • Fixed the F10 Settings menu.

4.0.0-beta.1

  • Initial WebRTC integration.

Version 3.x releases

3.6.x release history

3.6.7-Beta

  • Added UseSystemCertificates option for SSL configuration on Windows.

  • Fixed Convai logging not showing after packaging.

  • Fixed multiplayer crash due to sending text data over non-game thread.

3.6.6

  • Added EnableSync parameter for audio and lip sync synchronization control.

3.6.5

  • Adjusted MinBufferDuration and AudioLipSyncRatio defaults to reduce audio stutters.

3.6.4

  • Improved data validation in ConvaiAudioStreamer to prevent crashes from invalid input.

  • Fixed MinBufferDuration not working as expected when lip sync is active.

3.6.3

  • Improved memory management in ConvaiGRPC.

3.6.2

  • Potential fix for missing audio for ElevenLabs and some Azure voices.

  • Improved performance when receiving AI audio responses.

3.6.1

  • Fixed headers for Linux build.

3.6.0

  • UE 5.6 support.

  • Optimized internal component registry lookups.

  • Added Convai Logger.

3.5.x release history

3.5.4-hotfix-1

  • Fixed racing condition when playing audio.

  • Updated Android Play Core to 2.0.3.

  • Refactored old unused classes and code.

3.5.4

  • Further improved lip sync accuracy and synchronization.

  • Fixed player speech transcription failing after short character responses.

  • Improved gRPC connection stability to prevent conversation failures.

  • Resolved a rare crash caused by the server sending large volumes of audio in small chunks.

3.5.3-beta

  • Improved lip sync accuracy and synchronization.

  • Fixed voice cutting off for ElevenLabs voices.

  • Fixed Pixel Streaming compatibility.

3.5.2

  • Fixed rare crash when new voice data starts to play while the character is finishing its current sentence.

3.5.1

  • Added Long Term Memory V0.

  • Fixed voice not being sent to clients on multiplayer.

  • Added Dynamic Environment Info.

3.5.0

  • UE 5.5 support.

3.4.x and earlier

3.4.2-beta

  • Fixed player time-out warning message when using text.

  • Increased gRPC connection robustness by adding small delays and read retries.

3.4.1-beta

  • Fixed IsListening() not properly returning the character state.

3.4.0-beta

  • Fixed multiple connection issues.

  • Can now play custom montages over MetaHuman face animation.

3.3.2

  • Added intermediate fix for audio being cut off in multiplayer.

  • Added ConvaiGetAvailableVoices function.

3.3.1

  • Fixed narrative keys not being set properly.

  • Better lip synchronization using timestamps.

  • Added Convai Download Image using RPM link function.

  • Exposed Convai Player component for C++ use.

3.3.0

  • Added Linux support.

  • Fixed issue where an object similar to a character name would be picked up as the related object or character.

3.2.2-Beta

  • Fixed NPC2NPC crash when interrupting the character early on.

3.2.1-Beta

  • Fixed body gestures not animating while talking.

  • Added Narrative Trigger Keys.

  • Improved emotion animations for MetaHumans.

3.2.0

  • Added UE 5.4 support.

3.1.4

  • Added Pixel Streaming support and documentation.

  • Added x86_64 Android support.

  • Added Narrative Design helper functions.

  • Added more Blueprint documentation.

  • Improved action accuracy with objects and characters.

  • Improved MetaHuman facial and body animation logic.

  • Improved lip sync.

  • Updated Reallusion Animation Blueprint.

  • Fixed issue when adding custom MetaHuman animations.

  • Fixed crash with UE 5.3 when there is a connection issue.

  • Fixed multiplayer crash.

  • Fixed rare crash when using Invoke Speech with bad connection.

3.0.1 Hotfix

  • Fixed audio capture on Android sometimes not capturing properly.

  • Fixed lip sync crash on Android and improved performance.

  • Fixed rare crash on Android when a large amount of audio is spoken by the character.

  • Improved overall audio capture.

  • Fixed Update Character node not updating language.

  • Updated Convai ReadyPlayerMe plugin for improved head tracking.

  • Fixed lip sync not working when a player component and a chatbot component are in the same character Blueprint.

  • Added documentation guide for Mac microphone permissions issue.

  • Fixed startup crash for packaged apps on macOS UE 5.3 due to microphone permissions.

3.0.0

Added:

  • New lip sync component FaceSync, which no longer requires the ConvaiOVRLipsync plugin.

  • Mac lip sync support.

  • Emotions.

  • Object in Attention.

  • New Convai Chatbot Component functions: Invoke Speech, Invoke Narrative Design Trigger, Force Set Emotion, Reset Emotion State, Get Emotion Score, Get Talking Time Elapsed, Get Talking Time Remaining, Set Lipsync Component, Clear Action Queue.

  • New Convai Chatbot Component events: On Narrative Section Received, On Emotion State Changed.

  • ConvaiGetLookedAtObjectOrCharacter function.

  • Actor loses focus with player after predefined time.

  • Move To and Follow action failures trigger a character response.

  • Environment replication for multiplayer use cases.

Improved:

  • MetaHuman body animations, facial expressions, and eye look-at.

  • Reallusion Animation Blueprint, lip sync, and eye look-at.

  • Settings widget UI.

  • Actions accuracy.

  • Audio capture for low frame rate.

Fixed:

  • Packaging issue if Convai Player Component was in the scene prior to packaging.

  • Crash when many interactions were happening simultaneously.

  • ConvaiBasePlayerWithVoiceActivation Beta class was only working for the first interaction.

  • Overall plugin stability and warnings.

Deprecated:

  • ConvaiOVRLipsync plugin and component. Delete the plugin from your project and use the new FaceSync component instead.

Next steps

Getting started
PreviousFeature map
NextMigrate to 4.0.0-beta.27

Last updated

Was this helpful?