In a typical application integrating with the Convai API, maintaining a consistent session ID across different sessions is crucial for providing a seamless user experience. This documentation outlines the best practices for storing and retrieving session IDs using Unity's PlayerPrefs, including detailed steps and example scripts.

Importance of Session IDs

A session ID uniquely identifies a session between the client and the Convai server. Storing the session ID locally ensures that the same session ID is used across different sessions, which helps in maintaining context and continuity in interactions.

Storing Session IDs

When initializing a session, if a session ID is not available locally, it should be fetched from the server and then stored locally for future use. Here's how you can achieve this:

1. Fetch and Store Session ID: When initializing a session, check if a session ID is stored locally. If not, fetch a new session ID from the server and store it using PlayerPrefs.

public static async Task<string> InitializeSessionIDAsync(string characterName, ConvaiService.ConvaiServiceClient client, string characterID)
{
    // Retrieve stored session ID if it exists
    string sessionID = PlayerPrefs.GetString(characterID, string.Empty);

    // If no session ID is stored, initialize a new one
    if (string.IsNullOrEmpty(sessionID))
    {
        sessionID = await ConvaiGRPCAPI.InitializeSessionIDAsync(characterName, client, characterID, sessionID);

        // Store the new session ID locally
        if (!string.IsNullOrEmpty(sessionID))
        {
            PlayerPrefs.SetString(characterID, sessionID);
            PlayerPrefs.Save();
        }
    }

    return sessionID;
}

Retrieving Session IDs

When initializing your application, retrieve the stored session ID to ensure continuity in user interactions.

private async void Start()
{
    // Initialize session ID on start
    string characterID = "YourCharacterID"; // Replace with your actual character ID
    string sessionID = await InitializeSessionIDAsync("CharacterName", grpcClient, characterID);

    if (!string.IsNullOrEmpty(sessionID))
    {
        Debug.Log("Session ID initialized and stored: " + sessionID);
    }
    else
    {
        Debug.LogError("Failed to initialize session ID.");
    }
}

Example Class for Session Management

The following example class demonstrates how to manage session IDs using PlayerPrefs in a Unity project:

using System;
using System.Threading.Tasks;
using Convai.Scripts.Utils;
using Google.Protobuf;
using Grpc.Core;
using Service;
using UnityEngine;
using static Service.GetResponseRequest.Types;

public class SessionManager : MonoBehaviour
{
    public ConvaiService.ConvaiServiceClient grpcClient;

    private void Start()
    {
        // Initialize session ID on start
        InitializeSession("CharacterName", grpcClient, "YourCharacterID");
    }

    private async void InitializeSession(string characterName, ConvaiService.ConvaiServiceClient client, string characterID)
    {
        string sessionID = await InitializeSessionIDAsync(characterName, client, characterID);

        if (!string.IsNullOrEmpty(sessionID))
        {
            Debug.Log("Session ID initialized and stored: " + sessionID);
        }
        else
        {
            Debug.LogError("Failed to initialize session ID.");
        }
    }

    public static async Task<string> InitializeSessionIDAsync(string characterName, ConvaiService.ConvaiServiceClient client, string characterID)
    {
        string sessionID = PlayerPrefs.GetString(characterID, string.Empty);

        if (string.IsNullOrEmpty(sessionID))
        {
            sessionID = await ConvaiGRPCAPI.InitializeSessionIDAsync(characterName, client, characterID, sessionID);

            if (!string.IsNullOrEmpty(sessionID))
            {
                PlayerPrefs.SetString(characterID, sessionID);
                PlayerPrefs.Save();
            }
        }

        return sessionID;
    }
}

Detailed Steps for Session Management

1. Initialize Session: Call InitializeSessionIDAsync to check if a session ID is stored. If not, fetch and store it.

2. Store Session ID: Use PlayerPrefs.SetString(characterID, sessionID) to store the session ID locally.

3. Retrieve Session ID: Use PlayerPrefs.GetString(characterID, string.Empty) to retrieve the stored session ID.

4. Use Session ID: Pass the session ID to your Convai API calls to maintain session continuity.

Best Practices

• Error Handling: Ensure proper error handling when fetching and storing session IDs.

• Security: Consider encrypting sensitive information stored in PlayerPrefs.

• Performance: Use asynchronous methods to avoid blocking the main thread when fetching session IDs.

Convai Character emit character emotions when they interact with the player and these emotions help in making the character more human-like, we are starting to implement a system which you as a developer can use to make your game more interactive using the character emotions.

Whenever the character responds to the user, we send back a list of emotions to the SDK, which look something like this

For v0 of this system, we will only be sending the emotions, in future we will apply the facial expressions corresponding to each emotion which will make the character more interactive.

This is a scriptable object which is made automatically after you hit play in the editor with Convai SDK installed and in a Scene where Convai Base Scene Essentials Prefab is present

Default Player Name

You can provide a default name of your players.

Player Name

Current name of your player, out of the box if you use our settings panel, we keep it updated automatically, if you are using some custom logic, it will be your responsibility to keep it updated, as our transcript UI use this name to show it in UI

Speaker ID

Speaker ID for the player. Please note that Speaker ID is directly linked with your API key, so for each API key there should be a unique speaker ID associated with it. We handle the creation of the Speaker ID when it's not found in the Player Prefs if the Boolean is set to true.

Create Speaker ID If Not Found

This Boolean lets the SDK if it should create a unique Speaker ID for that Player Name if it is not found in the Player Prefs.

Buttons

Reset Data

It just makes the Player Name and Speaker ID fields empty.

Copy Data

Copies the data into system buffer so you can paste it anywhere for debugging purpose

Player Pref Settings Button

1. Load: Loads the Player Name and associated Speaker ID from the player Prefs

2. Save: Saves the Player Name and associated Speaker ID from the player Prefs

3. Delete: Deletes the Player Name and associated Speaker ID from the player Prefs

How to maintain the Player Data

Convai provides a pre-made component which you can add to any GameObject to make the PlayerDataContainer work out of the box.

Choose an existing GameObject or create a new GameObject in the scene and add the ConvaiPlayerDataHandler component to your chosen GameObject and it should start working

Optional Step

You can also create the required Scriptable Object by going to Assets > Convai > Resources and right clicking in the project panel and navigating to Create > Convai > Player Data and name it ConvaiPlayerDataSO

Long-Term Memory (LTM) enables the persistent storage of conversational history with NPCs, allowing players to seamlessly continue interactions from where they previously left off, even across multiple sessions. This feature significantly enhances the realism of NPCs, aligning with our goal of creating more immersive and lifelike characters within your game.

Steps to get LTM working

1. Select your Convai Character
2. Add the Long-Term Memory Component onto your character
3. Make sure that Long Term Memory is enabled for that character

Long Term Memory should now be working for your character.

Components of the LTM System

Convai Long Term Memory Component

This component will enable or disable LTM right from the unity editor

Toggling Long Term Memory

1) Click the button provided in the component

2) It will take some time to update, and after that the new status of the LTM should be visible in the inspector.

Troubleshooting

If you encounter this error, ensure that the SpeakerID was created using the same API key currently in use. If you're uncertain about the API key used, you can reset the SpeakerID and PlayerName by accessing the ConvaiPlayerDataSO file located in Assets > Convai > Resources, allowing you to start the process anew.

We provide several UI options to display character and user's transcript out of the box that players can use with the Convai Plugin. You can use and customize these prefabs.

The ConvaiNPC and ConvaiGRPCAPI scripts look for GameObjects with Convai Chat UI Handler as a component, and send any transcripts to the script so that it can be displayed on screen.

Types of UI

ChatBox

Prefab Name: Convai Transcript Canvas - Chat

Both the user's and the character's transcripts are displayed one after other in a scrollable chat box.

Subtitle

Prefab Name: Convai Transcript Canvas - Subtitle

The user and character transcripts are displayed in the bottom like subtitles.

Question-Answer

Prefab Name: Convai Transcript Canvas - QA

The user's transcript is displayed in the top where as the character's transcript is displayed in the bottom.

Mobile Optimised UI Styles

Prefab Name: Convai Transcript Canvas - Mobile Subtitle

Identical to #subtitle UI. Includes a button that can be pressed and held for the user to speak. Ideal for portrait orientation of screen.

Prefab Name: Convai Transcript Canvas - Mobile QA

Prefab Name: Convai Transcript Canvas - Mobile Chat

Convai Chat UI Handler Component

Functions to Know

Settings Panel consists of two main sections.

• Audio Settings

• Interface Settings

Audio Settings

Microphone Settings

The Microphone Settings section is primarily for troubleshooting and testing the microphone when using the Convai plugin.

• In the Input section, you can view the microphones connected to your computer and select the desired one.

• In the Test Input field, you can record your voice using the selected microphone in the Input section. After clicking Stop, you can listen to the recorded voice and observe the sound levels.

Interface Settings

Appearance

The first setting that greets us here is the Appearance setting.

In the Appearance section, you can switch between Transcript UI designs.

There are three Transcript UI options:

• ChatBox

• QuestionAnswer

• Subtitle

Display Name

The second section in Interface Settings is the Display Name section. This section allows you to change how the user's name appears in the Transcript UI.

Notifications Checkmark

The last section in Interface Settings is the Notifications Checkmark.

Convai sometimes displays notifications on the screen to inform the user. If you want to disable these notifications, you can click the checkbox here. ( If the box is green, it's active. If empty, it's inactive )

For more information about notifications, you can refer to this link.

The Convai plugin comes with default notifications, totaling four. Here they are:

Notifications

Not Close Enough to the Character

Appears when you press the talk button but there is no active NPC nearby.

Talk Button Released Early

Appears if you release the talk button in less than 0.5 seconds.

Microphone Issue Detected

Appears when the recorded audio input level is below the threshold.

Connection Problem

Appears when there is no internet connection upon launching the application.

How to Add Your Own Notification?

Adding your custom notification is straightforward.

Let's go through the steps to add a " CharacterStartedListening" notification as an example.

1. Open the script "Convai/Scripts/Notification System/Notification Type.cs." This script stores Notification Types as enums. Give a name to your desired Notification type and add it here.

2. Right-click on "Convai / Scripts / Notification System / Scriptable Objects" and select "Create > Convai > Notification System > Notification" then create a "Notification Scriptable Object".

3. Name the created Notification Scriptable Object. Click on it, and fill in the fields in the Inspector as desired.

4. Add the created Notification Scriptable Object to "Convai/Scripts/Notification System/Scriptable Objects" under "Convai Default Notification Group" (details of Notification Group here****).

5. Your Notification is now ready. The last step is to call this Notification from where you need it. For example, if you created the " CharacterStartedTalking " Notification, find the location where your character listens and write the code.

NotificationSystemHandler.Instance.NotificationRequest(NotificationType.CharacterStartedListening);

6. Replace the parameter with the NotificationType you created. (For our example, NotificationType.CharacterStartedListening)

7. Ensure that the Convai Notification System is present in your scene. (accessible from "Convai/Prefabs/ Notification System")

All steps are complete, and you're ready to test!🙂✅

Notification Scriptable Object

This Scriptable Object stores information about a Notification

• Notification Type

• Notification Icon

• Notification Title

• Notification Message

To create a new Notification Scriptable Object, right-click anywhere in the Project Window and select "Create > Convai > Notification System > Notification"

Notification Group Scriptable Object

This Scriptable Object stores Notification Scriptable Objects as groups. When a Notification is requested, it searches for the Notification using the specified Notification Group in the Convai Notification System prefab's Notification System Handler script.

You can create different Notification groups based on your needs. Note: If your referenced Notification Group does not have the Notification you want, that Notification won't be called.

The Convai Default Notification Group has four Notifications, but you can add more or create a new group with additional notifications.

Overview

The Dynamic UI system is a feature within the Convai Unity SDK that provides developers a robust system for in-game communication. This feature allows for displaying messages from characters and players and supports various UI components for chat, Q&A sessions, subtitles, and custom UI types. This document will guide you through the integration, usage, and creation of custom UI types of the Dynamic UI feature in your Unity project.

Usage

Accessing the Chat UI Handler

To interact with the chat system, you need to reference the ConvaiChatUIHandler in your scripts. You can find the Transcript UI prefab in the Prefabs folder.

Here's an example of how to find and assign the handler:

Sending Messages

Once you have a reference to the ConvaiChatUIHandler, you can send messages using the following methods:

Sending Player Text

To send text as the player:

• input: The string containing the player's message.

Sending Character Text

To send text as a character:

• characterName: The name of the character sending the message.

• currentResponseAudio.AudioTranscript: The transcript of the audio response from the character, trimmed of any leading or trailing whitespace.

Adding Custom UI Types to the Dynamic Chatbox

Creating a Custom UI Class

To create a custom UI type, follow these steps:

Step 1: Define Your Custom Class

Create a new C# script in your Unity project and define your class to inherit from ChatUIBase. For example:

Step 2: Implement Required Methods

Implement the abstract methods from ChatUIBase. You must provide implementations for Initialize, SendCharacterText, and SendPlayerText:

Step 3: Add Custom Functionality

Add any additional functionality or customization options that your custom UI may require.

Step 4: Assign and Use Your Custom UI

To use your custom UI class within the dictionaryConvaiChatUIHandler, you need to add it to the GetUIAppearances dictionary. This involves creating a prefab for your custom UI and assigning it in the ConvaiChatUIHandler.

Here's an example of how to do this:

1. Create a prefab for your custom UI and add your CustomChatUI component to it.

2. Assign the prefab to a public variable in the ConvaiChatUIHandler script.

3. Modify the InitializeUIStrategies method in the ConvaiChatUIHandler script to include your custom UI type.

4. Ensure that your custom UI type is added to the UIType enum:

By following these steps, you can integrate your custom UI type into the Dynamic Chatbox system and switch between different UI types at runtime.

How to Change the Talk Button or Any Input?

1. Double click on the "Controls" asset in your project tab.