Vision quick start

Add Vision to an existing Convai scene so a character can see and respond to live camera input, using the SDK's automatic component setup.

Add Vision to an existing Convai scene in three steps. The SDK adds and wires the required components automatically when you set ConvaiRoomManager to Video mode.

Prerequisites

Before starting, verify:

• A ConvaiCharacter is in the scene and responds to speech in Play Mode

Turn on vision for a character

1

Set Connection Type to Video

Select the ConvaiRoomManager GameObject in the Hierarchy. In the Inspector, set Connection Type to Video.

Connection Type set to Video on ConvaiRoomManager.

A dialog appears immediately:

Convai Vision Setup — Dynamic vision context requires a video publisher and frame source. Add ConvaiVisionPublisher and CameraVisionFrameSource under this ConvaiRoomManager?

Click Add Components.

Convai Vision Setup dialog prompt.

The SDK creates a child GameObject named ConvaiVisionRoot under ConvaiRoomManager and adds both ConvaiVisionPublisher and CameraVisionFrameSource to it.

2

Assign a camera

Select the ConvaiVisionRoot GameObject (child of ConvaiRoomManager). On the CameraVisionFrameSource component, locate the Target Camera field.

• If your scene has a Camera tagged MainCamera, leave the field blank — the component resolves it automatically at runtime.
• To capture a specific camera (an overhead view, a security camera), drag that camera into the Target Camera field.

The default Capture Preset is Balanced (1280 × 720 at 15 fps), which suits most scenarios.

If Target Camera is blank and no camera in the scene is tagged MainCamera, the frame source enters Failed state at runtime. Always assign a camera explicitly or ensure one camera has the MainCamera tag.

Target Camera assigned on CameraVisionFrameSource.

3

Verify with Debug Preview

On any scene GameObject, click Add Component → Convai/Vision/Vision Debug Preview (Editor Only).

Press Play. An overlay appears in the Game view showing the live camera feed and a statistics panel. Once the room connects, the FPS counter increments and the frame count increases.

Vision Debug Preview component added.

Success: The Debug Preview overlay shows a live image and a non-zero FPS counter. Reading ConvaiVisionPublisher.IsPublishing from any script returns true. The character now receives the scene camera feed alongside the audio conversation.

If the overlay stays blank or the FPS counter reads zero, verify that ConvaiRoomManager.Connection Type is set to Video and that the room has fully connected. See Troubleshoot vision for a step-by-step diagnosis.

Manual component setup

If you clicked Later on the dialog, or want to place the components on a specific GameObject, add them manually:

1. Select the target GameObject (any persistent scene object — typically on or near your NPC).
2. Add Component → search for Convai Vision Publisher.
3. On the same GameObject (or a child), Add Component → Convai/Vision/Camera Vision Frame Source.
4. Assign the Target Camera if not using Camera.main.
5. Leave the Source field on ConvaiVisionPublisher blank — the publisher auto-discovers CameraVisionFrameSource on the same GameObject at runtime. Assign it explicitly only if you have multiple frame sources in the scene.

Manual vision setup — publisher and frame source components.

Next steps

How vision worksVision frame sourcesPublish policiesTroubleshoot vision