Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Blueprints Reference - Comprehensive guide for Convai Unreal Engine integration.
The Convai Environment class is used to define what actions are available for the character and what are the objects and other characters in the scene.
Convai Environment is used as input to the StartTalking() or SendText() functions in the Convai Player component, and it allows the character to generate actions.
A Convai Environment object must have a Main Character set to be considered valid.
CreateConvaiEnvironment()
UConvaiEnvironment*
Creates a Convai Environment object.
SetMainCharacter(FConvaiObjectEntry InMainCharacter)
void
Assigns the main character initiating the conversation, typically the player character, unless the dialogue involves non-player characters talking to each other.
AddAction(FString Action)
void
Adds an action to the Environment object
AddActions(TArray ActionsToAdd)
void
Adds an array of actions to the Environment object
RemoveAction(FString Action)
void
Remove an action from the environment object.
RemoveActions(TArray ActionsToRemove)
void
Removes an array of actions from the Environment object
ClearAllActions()
void
Remove all actions from the Environment object.
AddObject(FConvaiObjectEntry Object)
void
Adds an object to the Environment object
AddObjects(TArray ObjectsToAdd)
void
Adds an array of objects to the Environment object
RemoveObject(FString ObjectName)
void
Remove an object from the environment object.
RemoveObjects(TArray ObjectNamesToRemove)
void
Removes an array of objects from the Environment object
ClearObjects()
void
Remove all objects from the Environment object.
AddCharacter(FConvaiObjectEntry Character)
void
Adds a character to the Environment object
AddCharacters(TArray CharactersToAdd)
void
Adds an array of characters to the Environment object
RemoveCharacter(FString CharacterName)
void
Remove a character from the environment object.
RemoveCharacters(TArray CharacterNamesToRemove)
void
Removes an array of characters from the Environment object
ClearCharacters()
void
Remove all characters from the Environment object.
Actor component for the player
Convai Player is an Actor component responsible for capturing microphone audio and streaming it to a Convai character.
For voice chat to work in multiplayer, this component must be added to the player's possessed object and not the PlayerController.
Is Recording()
Boolean
Returns True if microphone audio is recorded, False otherwise.
Is Talking()
Boolean
Returns True is microphone audio is been streamed, false otherwise.
Start Recording()
Void
Start recording audio from microphone, use "Finish Recording" function afterwards.
Finish Recording()
USoundWave*
Stops recording from the microphone and outputs the recorded audio from microphone.
Send Text(
UConvaiChatbotComponent* ConvaiChatbotComponent,
FString Text,
UConvaiEnvironment* Environment, bool GenerateActions,
bool VoiceResponse,
bool RunOnServer,
bool UseServerAPI_Key
)
Void
Sends text to character.
Start Talking(
UConvaiChatbotComponent* ConvaiChatbotComponent, UConvaiEnvironment* Environment, bool GenerateActions,
bool VoiceResponse,
bool RunOnServer,
bool StreamPlayerMic,
bool UseServerAPI_Key
)
Void
Starts streaming microphone audio to the character. Use "Finish Talking" afterwards to let the character know that you are doing talking.
Finish Talking()
Void
Stops streaming microphone audio to the character.
Get Available Capture Device Names()
TArray<FString>
Returns all the available capture devices.
Get Microphone Volume Multiplier()
Void
Gets microphone volume multiplier.
Set Capture Device Name()
Boolean
Sets the capture device name.
Set Microphone Volume Multiplier()
Void
Sets the microphone volume multiplier.
Get Active Capture Device()
Void
Gets info about active capture device.
The character to talk to.
Holds all relevant objects and characters in the scene including the (Player), and also all the actions doable by the character. Use the CreateConvaiEnvironment() function to create and then use functions like AddAction(), AddCharacter(), AddMainCharacter() to fill it up.
FString Text
Text to be sent to the character.
bool GenerateActions
Whether or not to generate actions (Environment has to be given and valid)
bool VoiceResponse
If true it will generate a voice response, otherwise, it will only generate a text response.
bool RunOnServer
If true it will run this function on the server, this can be used in multiplayer sessions to allow other players to hear the character's voice response.
bool StreamPlayerMic
If true it will stream the player's voice to other players in the multiplayer session, this is the same effect as voice chat.
* ConvaiChatbotComponent
* Environment
Convai Result Action - Blueprint Reference for Unreal Engine integration.
Actor component for the AI character
Convai Chatbot is an Actor component responsible for processing and getting a response for the voice audio coming from the Convai Player component. It plays the audio response and has a variety of useful events for transcription, actions, text response, and others.
Character Id
FString
The character ID you would like to assign to the component.
Interrupt Voice Fade Out Duration
Float
Time in seconds to gradually fade out voice response when interrupted until it is stopped.
Language Code
FString
Read the value of Language Code.
Ready Player Me Link
FString
Read the value of Ready Player Me Link.
Session Id
FString
To track memory of a previous conversation, set to -1 means no previous conversation, (this property will change as you talk to the character) you can save the session ID for a conversation and then set it back later on to resume a conversation.
Voice Type
FString
Read the value of the Variable VoiceType.
Environment
UConvaiEnvironment
Contains all relevant objects and characters in the scene including the (Player), and also all the actions doable by the character.
Avatar Image Link
FString
Read the value of Avatoar Image Link.
Character Name
FString
Read the value of variable CharacterName.
Backstory
FString
Read the value of variable Backstory.
Is in Conversation()
Boolean
Returns True, if the character is talked to, is talking, or is processing the response.
Is Talking( )
Boolean
Returns True, if the character is currently talking.
Is Thinking()
Boolean
Returns True, if the character is still processing and has not received the full response yet.
Is Listening ()
Boolean
Returns True, if the character is currently listening to the player.
Supports Lip Sync()
Boolean
Returns True, if LipSync component is available and attached to the character.
Get Viseme Names()
Array of Strings
Returns list of viseme names.
Get Visemes()
Array of Float
Returns last predicted viseme scores.
Interrupt Speech()
Void
Interrupts the current speech with a provided fade out duration.
Reset Conversation()
Void
Reset the conversation with the character and remove previous memory. This is the same as setting the session ID property to -1.
On Actions Received
Called when new actions are received from the character.
On Text Received
Called when new text is received from the character.
On Transcription Received
Called when new transcription is available.
On Started Talking
Called when the character starts talking.
On Finished Talking
Called when the character stops talking.
On Failure
Called when there is an error.
Bot Text
The received text
Audio Duration
Duration of spoken received text. Is equal to zero if only text was received and no audio.
Is Final
True if this is the last chunk of text/transcription to be received.
Transcription
The transcription received.
Is Transcription Ready
True if the received transcription is ready and is not going to change anymore, false if the transcription is still in an intermediate state and is going to change.
Utility Functions - Blueprint Reference for Convai Unreal Engine integration.
Description: Create a new character and get the character ID for it.
Inputs:
Char Name: Name of the character.
Voice: Voice name [MALE/FEMALE].
Backstory: Backstory for the new character.
Outputs:
Char ID: character id for the new character.
Nothing is returned, check logs for details on why it failed.
Description: Get a list of character IDs belonging to the user.
Outputs:
Char IDs: list of characters.
Nothing is returned, check logs for details on why it failed.
Description: Fetch all details of a character including backstory, voice, etc.
Inputs:
Char ID: Character ID for which to fetch all the details.
Outputs:
Character Name: Name of your character.
Voice Type: Voice name.
Backstory: Character backstory.
Has Ready Player Me Link: True if the avatar is configured on the website.
Ready Player Me Link: The avatar link to be used to download.
Nothing is returned, check logs for details on why it failed.
Description: Update a particular character.
Inputs:
Char ID: Character ID to be updated.
New Voice: Voice name or [MALE/FEMALE].
New Backstory: Updated backstory.
New Char Name: Name of the character.
For the list of supported voices please refer to the table in Text To Speech API.
To update a subset of properties, such as Voice and Name only, leave the other fields empty and the update will only affect the specified fields with values.
Outputs:
Nothing is returned.
â›” [On Failure]
Nothing is returned, check logs for details on why it failed.
Speech-to-Text - Blueprint Reference for Convai Unreal Engine integration.
Description: Transcribes the provided audio.
Inputs:
Sound Wave: The recorded output from the microphone, please take a look over the tutorial/sample project for an example on how to use Convai voice capture component with the API.
Outputs:
Response: Contains the text output of the audio file if the API was successful, otherwise might contain information on why the API call failed.
Nothing is returned, check logs for details on why it failed.
Description: Transcribes the provided audio file.
Inputs:
Filename: The path to the recorded audio file on your local disk, the file should be in a .wav format.
Outputs:
Response: Contains the text output of the audio file if the API was successful, otherwise might contain information on why the API call failed.
Nothing is returned, check logs for details on why it failed.
The ConvaiObjectEntry structure parses information about a character or an object in the scene, which is then included in the Environment object.
Convai Object Entry structure allows for parsing of information about an object or character in the scene. It takes in the name of the object or character and can also include a reference to the object or character, a description, and a position vector (optional). This information is then added to the Environment object.
For the list of supported voices please refer to the in Text To Speech API.
Ref
FString
A reference to a character or object. (Optional)
Optional Position Vector
FVector
A related position vector. (Optional)
Name
FString
The Name of the character or object.
Description
FString
The bio/description for the character/object. (Optional)
