LogoLogo
PlaygroundVideosBlogPricing
  • Welcome
  • Convai Playground
    • Playground Walkthrough
    • Get Started
    • Character Creator Tool
      • Create Character
      • Language and Speech
      • Knowledge Bank
      • Personality and Style
      • State of Mind
      • Memory
      • Actions
      • Narrative Design
      • Character Versioning
    • F.A.Q
  • Plugins & Integrations
    • Introduction
    • Unity Plugin
      • Pre-Requisites
      • Compatibility
      • Downloads
        • Limitations of WebGL Plugin
      • Setting Up Unity Plugin
      • Troubleshooting Guide
        • Disable Assembly Validation
        • Missing Newtonsoft Json
        • Microphone Permission Issues
        • Default Animations Incompatibility
        • Animations have Facial Blendshapes
        • Jaw Bone in Avatar is not Free
        • macOS Permission Issues
      • Creating a Convai Powered Scene from Template
      • Importing Ready Player Me (RPM) Characters
      • Importing Custom Characters
      • Adding Actions to your Character
      • Adding Lip-Sync to your Character
      • Adding Narrative Design to your Character
        • Narrative Design Keys
      • Adding NPC to NPC Conversation
      • Adding Scene Reference and Point-At Crosshairs
      • Utilities
        • Character Emotion
        • Player Data Container
        • Long Term Memory
        • Language Support
        • Managing sessionID Locally
        • Transcript UI System
        • Pre-built UI Prefabs
        • Input Management
        • Notification System
        • Settings Panel
        • Dynamic Information Context
      • Building For Supported Platforms
        • Building for iOS/iPadOS
        • Building for WebGL
        • Convai XR
          • Building for VR
            • VR Automatic Installation
            • VR Manual Installation
          • Building for MR
            • MR Automatic Installation
            • MR Manual Installation
          • Building for AR
          • Interacting with XR UI Elements
        • Building for macOS Universal apps
      • Changelogs
      • Tutorials
        • Narrative Design
        • NPC2NPC
    • Unreal Engine
      • Supported Platforms
      • Installation
      • Guides
        • Dynamic Environment Info
        • Change AI Character Movement Speed
        • Integration with Pixel Streaming
        • Adjust Interaction Radius
        • Creating MetaHuman Characters
          • Adding MetaHuman
          • Adding LipSync to MetaHuman (From plugin version 3.0.0 or later )
          • Change the Parent Class for MetaHuman
          • Change the parent class for Player.
          • Adding LipSync to MetaHuman (Deprecated)
        • Creating ReadyPlayerMe Characters
          • Download Plugins
          • Adding ReadyPlayerMe Character
          • Change the parent class for Player.
        • Creating Reallusion Characters
          • Creating a Reallusion Character
          • Importing Reallusion character and setting up the Convai plugin
          • Binding objects to Reallusion Character
        • Event-Aware Convai Characters
        • Operations Based on the Presence of Certain Words
        • Narrative Design
          • Narrative Design Trigger
          • Narrative Design Keys
        • Actions Guide
          • Stage 1: Default Actions
            • Moves To
            • Follows
          • Stage 2: Custom Actions
            • Simple actions
            • Adding Descriptions to Actions
          • Stage 3: Custom Actions with Single Parameter
          • Main Character and Attention Object
        • Customization
          • Push to Talk
          • Alter Character Response Audio Rate
        • Speech To Text Transcription
        • Enable Multiplayer Support
        • 3D Chat Widget
        • Long Term Memory
        • Saving and Loading Session
      • Blueprints Reference
        • Convai Player
        • Convai Chatbot
        • Convai Environment
        • Convai Object Entry
        • Convai Result Action
        • Convai Extra Params
        • Speech To Text
        • Text To Speech
        • Utility Functions
      • Troubleshoot Guide
        • Missing Unreal Engine Tool Set in Microsoft Visual Studio Toolchain
        • Convai Module Not Found
        • MetaHuman Plugin Conflict
        • Failure to Load Character IDs
      • Microphone Settings
        • Set Audio Gain
        • Set Microphone
        • Microphone test
        • List Microphone Devices
      • Mac Microphone Permission: Required for UE 5.0 and 5.3
      • Guides V2 (Under Development)
        • Getting Started
          • Installation
          • Simple Talking Cube
          • Adding Premade Chat and Settings UI
    • Web Plugin
      • PlayCanvas Plugin
        • Adding External Script
        • First Person View
        • Adding characters to scene
        • Character Animations
        • Convai Integration
        • Chat Overlay
      • Convai Web SDK
        • Getting Started
        • Facial Expressions
        • Actions
      • JavaScript Chat UI SDK
        • Getting Started
        • ChatBubble Props
      • Narrative Design Guide
        • Narrative Design Triggers
        • Narrative Design Keys
      • GLB Characters for Convai
      • GLB/FBX animations for Convai
    • Modding Framework
      • Modding Cyberpunk 2077
    • Other Integrations
      • Roblox
        • Sample Game 1
        • Sample Game 2
        • Code Example
          • Character Conversation API
      • Discord
        • Create a Discord Bot
        • Hosting Discord Bot from Personal Server
        • Hosting Discord Bot from Replit
      • Omniverse Extension
      • Third-Party API Integrations
        • ElevenLabs API Integration
  • Reference
    • Core API Reference
      • Character Base API
      • Interaction API
      • Core AI Setting API
      • Backstory API
      • Chat History API
      • Knowledge Bank API
      • Narrative Design API
      • Action API
      • Language List API
      • Voice List API
      • Character List API
      • Evaluation API
Powered by GitBook
On this page
  • Installation
  • Mod Creation
  • Helpful Tools, Examples and Guides from the Community
  • Special Thanks

Was this helpful?

  1. Plugins & Integrations
  2. Modding Framework

Modding Cyberpunk 2077

This guide will show you how to make Mods that add Convai's features to Cyberpunk 2077

PreviousModding FrameworkNextOther Integrations

Last updated 1 year ago

Was this helpful?

video link

This guide will show you how to make Mods that add Convai's features to any pre-existing or already released game, regardless of which engine it is using as long as it has a modding framework capable of creating logic based mods and is able to call scripts or executables or at least write to text files. We will use Cyberpunk 2077 and Cyber Engine Tweaks it's modding framework as an example. The mod we'll create will let players talk to the characters through their microphone or keyboard and the character can even select and perform actions. Lipsync is a little tricky and depends on how the game implements it. If you're working on a mod for an Unreal Engine game, check out our guide for the Hogwarts Legacy mod on Nexus Mods, where we explain lip sync in more detail. We'll be use the following tools: : This repository provides a set of libraries to access the Convai Character GetResponse API via gRPC streaming. : Cyber Engine Tweaks is a framework giving modders a way to script mods using Lua with access to all the internal scripting features, it also comes with a UI to configure the different mods you are using or using the console directly.

Installation

Before you begin, you'll need to set up the Convai SDK. If you haven't done so already, please follow the steps outlined . Cyber Engine Tweaks can found at nexusmods.

Mod Creation

To grasp the Mod creation process, let's walk through the steps I took for the Cyberpunk 2077 Mod. You can find the code I used in the Source Code folder of the mod. We'll begin with "main.cc," which holds the logic for "main.exe." Essentially, "main.cc" captures either the player's audio from microphone or text input from keyboard depending on the value in user_method_of_input.txt which is set by the lua mod, main.cc looks in the "get_response_config_example.txt" file for details like character ID, API key, available actions, and more which is again set by the lua mod. It then sends this information to Convai. After that, it plays the audio of the resulting dialogue of the character the player is interacting with and saves both the dialogue and action to a text file. This text file is later used to display subtitles and choose the appropriate in-game action. Additionally you can even get lipsync viseme data from the convai sdk for accurate lipsync. We use Cyber Engine Tweaks to create a Lua mod, which waits for the player to press F6 or F7 or whichever key the player selected as the hotkey for the mod. When it is pressed, the Lua mod finds the character the player is looking at, extracts information like the character's name, and uses it to find the character ID from a dictionary defined in the Lua file. This information is then written to the "get_response_config_example.txt" file. Next, we would use os.execute to run "main.exe" but since Cyber Engine Tweaks does not support os.execute so we need to look at a text file called control.txt and waits for it to have "start" and when it does run the main logic, the player will need to double click main.exe before launching the game. You can take a look at main.cc in source code folder to get a better understanding of what's going on. The execution completes after playing the dialogue. We parse the dialogue text and action value to display as subtitles. Using string matching we check which action was selected and execute the correct if block for that action. Reviewing the code in "init.lua", "main.cc" will provide a clearer understanding.

Helpful Tools, Examples and Guides from the Community

Special Thanks

A special shout-out goes to the modding community at Cyberpunk 2077. Without their invaluable assistance, tools, guides, and documentation, none of this would have been achievable. Much appreciation for their indispensable contributions!

Convai SDK(C++)
Cyber Engine Tweaks
here
https://www.nexusmods.com/cyberpunk2077/articles/945
https://wiki.redmodding.org/home/
https://wiki.redmodding.org/cyber-engine-tweaks/
https://github.com/psiberx/cp2077-cet-kit/tree/main
https://github.com/WolvenKit/cet-examples/tree/main/ai-components