LogoLogo
PlaygroundVideosBlogPricing
  • Welcome
  • Convai Playground
    • Playground Walkthrough
    • Get Started
    • Character Creator Tool
      • Create Character
      • Language and Speech
      • Knowledge Bank
      • Personality and Style
      • State of Mind
      • Memory
      • Actions
      • Narrative Design
      • Character Versioning
    • F.A.Q
  • Plugins & Integrations
    • Introduction
    • Unity Plugin
      • Pre-Requisites
      • Compatibility
      • Downloads
        • Limitations of WebGL Plugin
      • Setting Up Unity Plugin
      • Troubleshooting Guide
        • Disable Assembly Validation
        • Missing Newtonsoft Json
        • Microphone Permission Issues
        • Default Animations Incompatibility
        • Animations have Facial Blendshapes
        • Jaw Bone in Avatar is not Free
        • macOS Permission Issues
      • Creating a Convai Powered Scene from Template
      • Importing Ready Player Me (RPM) Characters
      • Importing Custom Characters
      • Adding Actions to your Character
      • Adding Lip-Sync to your Character
      • Adding Narrative Design to your Character
        • Narrative Design Keys
      • Adding NPC to NPC Conversation
      • Adding Scene Reference and Point-At Crosshairs
      • Utilities
        • Character Emotion
        • Player Data Container
        • Long Term Memory
        • Language Support
        • Managing sessionID Locally
        • Transcript UI System
        • Pre-built UI Prefabs
        • Input Management
        • Notification System
        • Settings Panel
        • Dynamic Information Context
      • Building For Supported Platforms
        • Building for iOS/iPadOS
        • Building for WebGL
        • Convai XR
          • Building for VR
            • VR Automatic Installation
            • VR Manual Installation
          • Building for MR
            • MR Automatic Installation
            • MR Manual Installation
          • Building for AR
          • Interacting with XR UI Elements
        • Building for macOS Universal apps
      • Changelogs
      • Tutorials
        • Narrative Design
        • NPC2NPC
    • Unreal Engine
      • Supported Platforms
      • Installation
      • Guides
        • Dynamic Environment Info
        • Change AI Character Movement Speed
        • Integration with Pixel Streaming
        • Adjust Interaction Radius
        • Creating MetaHuman Characters
          • Adding MetaHuman
          • Adding LipSync to MetaHuman (From plugin version 3.0.0 or later )
          • Change the Parent Class for MetaHuman
          • Change the parent class for Player.
          • Adding LipSync to MetaHuman (Deprecated)
        • Creating ReadyPlayerMe Characters
          • Download Plugins
          • Adding ReadyPlayerMe Character
          • Change the parent class for Player.
        • Creating Reallusion Characters
          • Creating a Reallusion Character
          • Importing Reallusion character and setting up the Convai plugin
          • Binding objects to Reallusion Character
        • Event-Aware Convai Characters
        • Operations Based on the Presence of Certain Words
        • Narrative Design
          • Narrative Design Trigger
          • Narrative Design Keys
        • Actions Guide
          • Stage 1: Default Actions
            • Moves To
            • Follows
          • Stage 2: Custom Actions
            • Simple actions
            • Adding Descriptions to Actions
          • Stage 3: Custom Actions with Single Parameter
          • Main Character and Attention Object
        • Customization
          • Push to Talk
          • Alter Character Response Audio Rate
        • Speech To Text Transcription
        • Enable Multiplayer Support
        • 3D Chat Widget
        • Long Term Memory
        • Saving and Loading Session
      • Blueprints Reference
        • Convai Player
        • Convai Chatbot
        • Convai Environment
        • Convai Object Entry
        • Convai Result Action
        • Convai Extra Params
        • Speech To Text
        • Text To Speech
        • Utility Functions
      • Troubleshoot Guide
        • Missing Unreal Engine Tool Set in Microsoft Visual Studio Toolchain
        • Convai Module Not Found
        • MetaHuman Plugin Conflict
        • Failure to Load Character IDs
      • Microphone Settings
        • Set Audio Gain
        • Set Microphone
        • Microphone test
        • List Microphone Devices
      • Mac Microphone Permission: Required for UE 5.0 and 5.3
      • Guides V2 (Under Development)
        • Getting Started
          • Installation
          • Simple Talking Cube
          • Adding Premade Chat and Settings UI
    • Web Plugin
      • PlayCanvas Plugin
        • Adding External Script
        • First Person View
        • Adding characters to scene
        • Character Animations
        • Convai Integration
        • Chat Overlay
      • Convai Web SDK
        • Getting Started
        • Facial Expressions
        • Actions
      • JavaScript Chat UI SDK
        • Getting Started
        • ChatBubble Props
      • Narrative Design Guide
        • Narrative Design Triggers
        • Narrative Design Keys
      • GLB Characters for Convai
      • GLB/FBX animations for Convai
    • Modding Framework
      • Modding Cyberpunk 2077
    • Other Integrations
      • Roblox
        • Sample Game 1
        • Sample Game 2
        • Code Example
          • Character Conversation API
      • Discord
        • Create a Discord Bot
        • Hosting Discord Bot from Personal Server
        • Hosting Discord Bot from Replit
      • Omniverse Extension
      • Third-Party API Integrations
        • ElevenLabs API Integration
    • Convai Pixel Streaming Embed
      • Whitelisting & Publishing an Experience
      • React Typescript
      • ReactJS (JavaScript)
      • Vanilla JavaScript (ES Modules)
      • CDN (UMD Script)
      • On Premise Deployment
      • API Reference
  • Reference
    • Core API Reference
      • Character Base API
      • Interaction API
      • Core AI Setting API
      • Backstory API
      • Chat History API
      • Knowledge Bank API
      • Narrative Design API
      • Action API
      • Language List API
      • Voice List API
      • Character List API
      • Evaluation API
Powered by GitBook
On this page
  • Installation
  • Configuration
  • Interacting with the character
  • Running the Demo
  • Actions
  • Notes

Was this helpful?

  1. Plugins & Integrations
  2. Other Integrations

Omniverse Extension

This guide will show you how to integrate the Convai and NVIDIA Omniverse using the Convai Extension plugin.

PreviousHosting Discord Bot from ReplitNextThird-Party API Integrations

Last updated 5 months ago

Was this helpful?

Installation

To install the Convai Omniverse Extension, follow these steps:

  1. Extract the zip file into your chosen directory.

  2. Open the Omniverse app of your choice (e.g Code) and from the Window menu click Extensions.

  3. Click the green plus icon in the Edit column and add the absolute path to the exts folder found in the repository directory.

  4. Select the Third Party tab and search for Convai in the top left search bar, make sure to check Enabled.

  5. The Convai window should appear, drag it and dock it in any suitable area of the UI.

  6. If the Convai window does not appear, go to the Window menu and select Convai from the list.

Configuration

To add your API Key and Character ID, follow these steps:

  1. On the website click the gear icon in the top-right corner of the playground then copy and paste the API key into the Convai API Key field in the Convai extension window.

  2. Copy the Character ID and paste it in the Character ID field in the extension window.

Interacting with the character

  1. Click Start Talking to initiate the conversation

  2. Click Stop when done talking for the character to respond.

Running the Demo

  1. Open your chosen Omniverse app (e.g., Code).

  2. Go to File->Open and navigate to the repo directory.

  3. Navigate to <repo directory>/ConvaiDemoStage/ConvaiDemo.usd and click `open it.

  4. Click Start Talking in the Convai window to talk to the character then click Stop to send the request.

Actions

Actions can be used to trigger events with the same name as the action in the Action graph. They can be used to run animations based on the action received. To try out actions:

  1. Add a few comma-separated actions to Comma seperated actions field (e.g jump, kick, dance, etc.).

  2. The character will select one of the actions based on the conversation and run any event with the same name as the action in the Action Graph.

Notes

  • The demo stage includes only talk and idle animations. However, it is possible to add more animations and trigger them using the action selected by the character. More on that in the future.

Download the of the extension.

In the Extensions tab, click the gear icon in the top right.

Sign up at .

Go to the and on the left panel and either create a new character or select a sample one.

Click the play button from the Toolbar menu on the left.

latest version
Convai
Dashboard
Extension Overview