Publishing & Policies

Control how Vision frames are delivered to the backend. Covers publish policies, frame rate and bitrate configuration, and manual publishing control.

Controlling How Vision Frames Are Published

ConvaiVisionPublisher is the component that bridges the frame source and the Convai backend. It manages the WebRTC video track lifecycle — starting, maintaining, and stopping the stream in response to room connection events — and exposes a set of policies and overrides for tuning the transport characteristics. This page explains every Inspector field, describes each publish policy, and shows how to take manual control of publishing via script.

ConvaiVisionPublisher Inspector Reference

Frame Source

Field	Type	Default	Description
Frame Source Component	MonoBehaviour	(auto-resolved)	The IVisionFrameSource implementation to publish. Leave blank to auto-discover the first available frame source in the scene. If multiple frame sources exist, assign this explicitly.

Publish Policy

Field	Type	Default	Description
Publish Policy	VisionPublishPolicy	AutoCompatible	Controls the client-side frame rate and bitrate. See the policy table below.

Video Settings

Field	Type	Default	Description
Video Track Name	string	"unity-scene"	The name of the WebRTC video track published to the room. The Convai backend uses this name to associate the video stream with the session. Change it only if you have a specific multi-track configuration.
Publish Frame Rate Override	int (0–30)	0	Overrides the frame rate specified by the publish policy. 0 means "use the policy default".
Publish Bitrate Override	int	0	Overrides the maximum bitrate (bits per second) specified by the publish policy. 0 means "use the policy default".

---

Publish Policy Comparison

VisionPublishPolicy is a client-side transport setting. It controls how aggressively frames are sampled and how much bandwidth is allocated. It does not determine which AI model or vision provider processes the stream on the backend.

Policy	Value	Frame rate	Max bitrate	Recommended for
AutoCompatible	0	10 fps	750 000 bps	General use — a balanced default that works across most backend configurations
HighResponsiveness	1	15 fps	1 000 000 bps	Live multimodal interactions where the character must react quickly to visual changes
LowOverhead	2	5 fps	350 000 bps	High-volume deployments, cost-sensitive sessions, or scenarios where snapshots are more important than motion
Manual	3	—	—	Explicit control over when publishing starts and stops; uses AutoCompatible rates when enabled

VisionPublishPolicy describes only the client-side transport — frame rate and bitrate sent over the network. It does not expose or control the backend AI model, vision provider, or processing latency. Do not use it as a proxy for backend behaviour.

---

Frame Rate and Bitrate Overrides

When you need a specific value that falls between policies, use the override fields:

• Publish Frame Rate Override accepts values from 1 to 30. Setting 0 restores the policy default.

• Publish Bitrate Override accepts any positive integer in bits per second. Setting 0 restores the policy default.

Example: AutoCompatible policy with a 12 fps override produces 12 fps at 750 000 bps.

On WebGL, the frame rate is clamped to a maximum of 15 fps regardless of policy or override. This is a platform constraint imposed by canvas.captureStream().

---

Auto-Publish Lifecycle

By default (any policy except Manual) publishing follows the room connection:

1. Room connects with Connection Type = Video → publishing starts automatically.

2. Room disconnects or the module stops → publishing stops and the video track is unpublished.

3. The component is disabled → publishing stops.

4. The component is re-enabled while the room is connected → publishing resumes.

You do not need to call any method to trigger this behaviour.

---

Manual Policy

Set Publish Policy to Manual when you want explicit control — for example, to start publishing only when a user action occurs, or to pause the stream between training modules.

Starting and stopping manually

using Convai.Modules.Vision;

ConvaiVisionPublisher publisher = GetComponent<ConvaiVisionPublisher>();

// Begin publishing (uses AutoCompatible rates)
publisher.EnablePublishing(true);

// Stop publishing without disconnecting the room
publisher.EnablePublishing(false);

Switching policies at runtime

// Switch to HighResponsiveness mid-session
publisher.SetPublishPolicy(VisionPublishPolicy.HighResponsiveness);

SetPublishPolicy takes effect immediately. If publishing is active, the current track is unpublished and a new one is started with the updated profile.

---

WebGL Platform Notes

On WebGL the ConvaiVisionPublisher captures the visible Unity browser canvas via canvas.captureStream() instead of reading from a frame source component. You do not need to add a frame source component to your scene for WebGL builds.

Publish policy selection still applies — use LowOverhead for performance-sensitive WebGL deployments. Frame rate is capped at 15 fps on all WebGL builds regardless of the policy or override value.

---

Conclusion

ConvaiVisionPublisher manages the full video track lifecycle — it starts publishing when the room connects, stops when it disconnects, and exposes a policy system for controlling frame rate and bandwidth. With publishing configured, add Debug Preview to verify the stream visually before deploying. For the full scripting API, see Advanced Topics.