Publishing & Policies
Control how Vision frames are delivered to the backend. Covers publish policies, frame rate and bitrate configuration, and manual publishing control.
Controlling How Vision Frames Are Published
ConvaiVisionPublisher is the component that bridges the frame source and the Convai backend. It manages the WebRTC video track lifecycle — starting, maintaining, and stopping the stream in response to room connection events — and exposes a set of policies and overrides for tuning the transport characteristics. This page explains every Inspector field, describes each publish policy, and shows how to take manual control of publishing via script.
ConvaiVisionPublisher Inspector Reference
Frame Source
Frame Source Component
MonoBehaviour
(auto-resolved)
The IVisionFrameSource implementation to publish. Leave blank to auto-discover the first available frame source in the scene. If multiple frame sources exist, assign this explicitly.
Publish Policy
Publish Policy
VisionPublishPolicy
AutoCompatible
Controls the client-side frame rate and bitrate. See the policy table below.
Video Settings
Video Track Name
string
"unity-scene"
The name of the WebRTC video track published to the room. The Convai backend uses this name to associate the video stream with the session. Change it only if you have a specific multi-track configuration.
Publish Frame Rate Override
int (0–30)
0
Overrides the frame rate specified by the publish policy. 0 means "use the policy default".
Publish Bitrate Override
int
0
Overrides the maximum bitrate (bits per second) specified by the publish policy. 0 means "use the policy default".
Publish Policy Comparison
VisionPublishPolicy is a client-side transport setting. It controls how aggressively frames are sampled and how much bandwidth is allocated. It does not determine which AI model or vision provider processes the stream on the backend.
AutoCompatible
0
10 fps
750 000 bps
General use — a balanced default that works across most backend configurations
HighResponsiveness
1
15 fps
1 000 000 bps
Live multimodal interactions where the character must react quickly to visual changes
LowOverhead
2
5 fps
350 000 bps
High-volume deployments, cost-sensitive sessions, or scenarios where snapshots are more important than motion
Manual
3
—
—
Explicit control over when publishing starts and stops; uses AutoCompatible rates when enabled
VisionPublishPolicy describes only the client-side transport — frame rate and bitrate sent over the network. It does not expose or control the backend AI model, vision provider, or processing latency. Do not use it as a proxy for backend behaviour.
Frame Rate and Bitrate Overrides
When you need a specific value that falls between policies, use the override fields:
Publish Frame Rate Override accepts values from
1to30. Setting0restores the policy default.Publish Bitrate Override accepts any positive integer in bits per second. Setting
0restores the policy default.
Example: AutoCompatible policy with a 12 fps override produces 12 fps at 750 000 bps.
On WebGL, the frame rate is clamped to a maximum of 15 fps regardless of policy or override. This is a platform constraint imposed by canvas.captureStream().
Auto-Publish Lifecycle
By default (any policy except Manual) publishing follows the room connection:
Room connects with
Connection Type = Video→ publishing starts automatically.Room disconnects or the module stops → publishing stops and the video track is unpublished.
The component is disabled → publishing stops.
The component is re-enabled while the room is connected → publishing resumes.
You do not need to call any method to trigger this behaviour.
Manual Policy
Set Publish Policy to Manual when you want explicit control — for example, to start publishing only when a user action occurs, or to pause the stream between training modules.
Starting and stopping manually
Switching policies at runtime
SetPublishPolicy takes effect immediately. If publishing is active, the current track is unpublished and a new one is started with the updated profile.
WebGL Platform Notes
On WebGL the ConvaiVisionPublisher captures the visible Unity browser canvas via canvas.captureStream() instead of reading from a frame source component. You do not need to add a frame source component to your scene for WebGL builds.
Publish policy selection still applies — use LowOverhead for performance-sensitive WebGL deployments. Frame rate is capped at 15 fps on all WebGL builds regardless of the policy or override value.
Conclusion
ConvaiVisionPublisher manages the full video track lifecycle — it starts publishing when the room connects, stops when it disconnects, and exposes a policy system for controlling frame rate and bandwidth. With publishing configured, add Debug Preview to verify the stream visually before deploying. For the full scripting API, see Advanced Topics.
Last updated
Was this helpful?