Here are some Guides to get you started with integrating Convai in Unreal Engine.
Use a premade player blueprint that already contains a Chat and Settings widget.
The goal of this guide is to easily show how to add a premade chat and settings widget to the UI by re-parenting the player blueprint, we will continue on the progress from the Simple Talking Cube guide.
It is recommended to remove the Convai Player component from the player blueprint that was created in the previous guide before proceeding with this guide.
(Optional) If you do not have a player blueprint already, then you can import a first person or third person content into your already existing project.
In the content browser, navigate to your player blueprint, which is by default at FirstPerson/Blueprint/BP_FirstPersonCharacter for First Person or ThirdPerson/Blueprint/BP_ThirdPersonCharacter which is the default for Third Person.
Open the blueprint and click Class Settings then in the Details section Under Class Options change the parent class to ConvaiBasePlayer.
Hit save, compile and hit play to test - use the T key to talk and Enter to text chat.
Hit F10 to open the settings menu where you have various options, like testing your microphone and changing the chat widget layout.
For advanced use cases, you may not want to change the parent class of the player blueprint, in that case we encourage you to use the Convai Base Player Blueprint itself as reference to see how to add the Chat and Settings widget or even create your own widgets from scratch.
We expect developers who are looking to implement everything from scratch or do customization to be advanced enough to navigate the blueprints on their own.
From the Content Browser, click the Add button, then Add feature or content pack to the project then choose either First Person or Third Person then click Add to Project.
Click the Window menu and make sure World Settings is enabled.
On the World Settings tab, Find GameMode Override and set it to BP_FirstPersonGameMode for first person or BP_thirdPersonGameMode if you imported third person content.
Easily install the Convai Plugin in Unreal Engine with our step-by-step guide. Set up your API key to add Convai to your Unreal Engine project.
Download Visual Studio from here.
Ensure having the required C++ toolchains mentioned here. If you already have Visual Studio installed, you may open the installer and select 'Modify' to add the above mentioned toolchains.
Download XCode from the app store
For UE 5.3 and UE 5.0, follow this guide to enable microphone permissions.
There are two methods to install the plugin depending on your requirements
Directly from the Marketplace Link: Recommended for easy installation and tracking updates to the plugin. (UE 5.1 - 5.3)
Building from Github Source: For source UE builds and for unsupported UE versions on the marketplace, but not guranteed as the marketplace approach. (UE 4.26 - 5.0)
From the top toolbar, go to Edit > Plugins.
Find the Convai plugin.
Click the checkbox to enable the plugin.
Restart Unreal Engine.
Go to Edit > Project Settings.
Choose Convai under the Plugins section on the left bar.
Paste the API key into the API Key field.
The following set of guides will help you make a simple project with Convai.
A minimal from scratch example to get you familiar with the plugin main components.
Create a new first person project.
Enable the Convai plugin and add the API key as mentioned here.
Open the created blueprint then search for and add the Convai Chatbot component in the components list.
Note: if you do not find the component then ensure that you have properly installed and enabled the plugin by following the Installation guide.
Select the created component and on the details panel, find the Character ID field and paste your character ID which you can get by creating a new character or using an existing one on the Convai Playground.
Add a box component so that you can see the blueprint when placed in the scene.
Place the blueprint in the scene.
Open the player blueprint which by default in First Person/Blueprints/BP_FirstPersonCharacter for the first person template.
Search and add the Convai Player component in the components list.
Add the following blueprint schematic to allow the player to talk to the AI character via the V key:
Add a keyboard key event to be used as a push to talk button (i.e. the `V` key in this example).
Use Convai Get Looked At Character to get the chatbot component of the character that is currently viewed by the player.
Set Radius to a reasonable distance or zero if you want the player to be talk the character over an infinite distance.
Set Plane View to true to only consider the plane axis (X & Y) and ignore the height axis (Z), this is made to prevent having to look directly at the pivot of the character, instead only looking in the direction of the character is sufficent.
Use the Start Talking node from the Convai Player component initiates the talking session with the character, ensure you have enabled Voice Response to get the character to respond vocally.
On the Released event, use the Finish Talking node on the Convai Player component to let the AI character know that we have finished talking and are now waiting for a response.
Hit play, approach the AI character and push T to talk through the microphone, the character should then respond after releasing the T key.
If the character does not respond then make sure your microphone is set properly as the default microphone in the OS settings.
Use the following Send Text node instead if you want to text chat with the character instead of voice.
Note, over here we use a hard coded string as input to the character, you will need to create the required UI to get the text input from the user and send it to the AI character.
Create a new Actor blueprint that we will be using as the AI character.
