> For the complete documentation index, see [llms.txt](https://docs.convai.com/api-docs/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.convai.com/api-docs/plugins-and-integrations/convai-unreal-engine-plugin/features/vision/vision-blueprint-reference.md). # Vision Blueprint reference Complete reference for the Blueprint-accessible vision surface in the Convai Unreal Engine plugin. Items here are verified against `VisionInterface.h`, `ConvaiWebcamBase.h`, `EnvironmentWebcame.h`, `EnvironmentWebcame.cpp`, `ConvaiAudioStreamer.h`, and `ConvaiAudioStreamer.cpp`. The shipped frame source is **Environment Webcam** (`UEnvironmentWebcam`). There is no separate physical webcam component in the current plugin. ### Enums #### EVisionState Declared in `VisionInterface.h`. Describes the lifecycle state of a vision component. | Value | Display name | Description | | ----------- | ------------ | ----------------------------------------------------------- | | `Stopped` | Stopped | The vision system is inactive. No frames are captured. | | `Starting` | Starting | Defined in the enum. Not set by the shipped implementation. | | `Capturing` | Capturing | The vision system is active and capturing frames. | | `Stopping` | Stopping | Defined in the enum. Not set by the shipped implementation. | | `Paused` | Paused | Defined in the enum. Not set by the shipped implementation. | In the current shipped path, **Environment Webcam** sets only `Stopped` and `Capturing`. #### ETextureSourceType Declared in `VisionInterface.h`. Describes the texture type returned by **Get Image Texture**. | Value | Display name | Description | | ---------------- | -------------- | ------------------------------------------ | | `Texture2D` | Texture2D | The texture is a `UTexture2D`. | | `RenderTarget2D` | RenderTarget2D | The texture is a `UTextureRenderTarget2D`. | ### UConvaiWebcamBase Blueprint nodes `UConvaiWebcamBase` is the abstract base class for Convai vision components. All nodes below are in category **Convai | Vision**. #### Start Starts capture on the component. In `UEnvironmentWebcam`, this validates the capture component and render target, then sets state to `Capturing`. No input or output parameters. #### Stop Stops capture on the component. In `UEnvironmentWebcam`, this sets state to `Stopped` and disables `CaptureComponent->bCaptureEveryFrame`. No input or output parameters. #### Get State Returns the current `EVisionState` of the component. `BlueprintPure`. | Return | Type | Description | | ------------ | -------------- | ----------------------------------- | | Return Value | `EVisionState` | The current vision lifecycle state. | #### Set Max FPS Sets the maximum capture rate in frames per second. Values `<= 0` are rejected and recorded as an error. | Parameter | Type | Description | | --------- | ----- | ------------------------- | | Max FPS | `int` | The maximum FPS to apply. | The chatbot upload path clamps the effective send rate to `1`–`60` when reading `GetMaxFPS()`. #### Get Max FPS Returns the current maximum FPS setting. `BlueprintPure`. | Return | Type | Description | | ------------ | ----- | ----------------------------------------- | | Return Value | `int` | The current maximum FPS. Default is `15`. | #### Get Last Error Message Returns the last error message recorded by the component. `BlueprintPure`. | Return | Type | Description | | ------------ | --------- | --------------------------------------------- | | Return Value | `FString` | Human-readable description of the last error. | #### Get Last Error Code Returns the last error code recorded by the component. `BlueprintPure`. The initial value is `-1`. | Return | Type | Description | | ------------ | ----- | ------------------- | | Return Value | `int` | Numeric error code. | #### Get Image Texture Returns the current frame texture and its source type. | Parameter | Type | Direction | Description | | ------------------- | -------------------- | --------- | ---------------------------------------------------------------- | | Texture Source Type | `ETextureSourceType` | Output | Whether the returned texture is `Texture2D` or `RenderTarget2D`. | | Return Value | `UTexture*` | Return | The current frame texture. | `CaptureRaw`, `CaptureCompressed`, `GetCompressedData`, and `IsCompressedDataAvailable` are part of `IConvaiVisionInterface` but are not Blueprint nodes on the shipped classes. The chatbot send path uses `CaptureRaw()` from C++. ### UConvaiWebcamBase events #### On Frame Ready A `BlueprintAssignable` delegate on `UConvaiWebcamBase`. For `UEnvironmentWebcam`, it broadcasts during `TickComponent()` while the component is in `Capturing` and the event is bound. To run logic only on the first frame, use a boolean guard inside the bound event. `FOnFirstFrameCaptured` on `IConvaiVisionInterface` is C++ only. ### UEnvironmentWebcam properties `UEnvironmentWebcam` extends `UConvaiWebcamBase`. All properties are in category **Convai | Vision**. Own properties declared on `UEnvironmentWebcam`: | Property | Type | Default | Description | | ---------------------------- | --------------------------- | ------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------- | | `ConvaiRenderTarget` | `UTextureRenderTarget2D*` | `null` | The render target the component renders into. Required before calling **Start**. | | `CaptureComponent` | `USceneCaptureComponent2D*` | Auto-created as `EnvironmentSceneCapture2D` | Internal scene capture component. Constructor sets `bCaptureEveryFrame = false`, `bCaptureOnMovement = false`, and `CaptureSource = SCS_FinalToneCurveHDR`. | | `bCopyPostProcessProperties` | `bool` | `false` | When enabled, copies settings from the first `APostProcessVolume` found in the world during `BeginPlay`. | | `bAutoStartVision` | `bool` | `false` | When `true`, calls **Start** automatically during `BeginPlay`. | Inherited from `UConvaiWebcamBase`: | Property | Type | Default | Description | | ---------------- | --------- | ------- | --------------------------------------------------------------- | | `Identifier` | `FString` | `""` | Optional label. Not used by the shipped runtime path. | | `m_MaxFPS` | `int` | `15` | Maximum capture FPS. Set via **Set Max FPS**. | | `bUpdateOnFetch` | `bool` | `true` | Stored on the base class. Not used by the shipped runtime path. | ### UEnvironmentWebcam runtime behavior | Function | Behavior | | --------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | `BeginPlay()` | Copies post-process settings if `bCopyPostProcessProperties` is `true`, then calls **Start** if `bAutoStartVision` is `true`. | | `Start()` | Returns without changing state if `CanStart()` fails. Otherwise sets state to `Capturing`, assigns `ConvaiRenderTarget` to `CaptureComponent->TextureTarget` if needed, enables `bCaptureEveryFrame`, and fires `OnFirstFrameCaptured` if bound. | | `Stop()` | Returns without changing state if `CanStop()` fails. Otherwise sets state to `Stopped`, disables `bCaptureEveryFrame`, and fires `OnFramesStopped` if bound. | | `CaptureRaw()` | Reads raw RGBA bytes from `ConvaiRenderTarget` and returns width and height from the render target. | | `CaptureCompressed()` | Converts `ConvaiRenderTarget` to JPEG bytes. The chatbot send path uses `CaptureRaw()`. | ### Chatbot vision nodes The following nodes are declared on `UConvaiAudioStreamer` and appear on `UConvaiChatbotComponent` through inheritance. Category **Convai | Vision**. They are also inherited by `UConvaiPlayerComponent`, but return `false` on the player because only the chatbot overrides `CanUseVision()`. #### Set Vision Component Registers a vision component with the chatbot, replacing any previously registered one. | Parameter | Type | Description | | ---------------- | ------------------ | ---------------------------------------------------- | | Vision Component | `UActorComponent*` | Any component implementing `UConvaiVisionInterface`. | | Return | Type | Description | | ------------ | ------ | --------------------------------------------------------------------------------------- | | Return Value | `bool` | `true` if the component implements the interface and was registered; `false` otherwise. | #### Supports Vision Returns `true` if a valid vision component is registered. If none is registered, searches the owning `Actor` for the first component that implements `UConvaiVisionInterface`. | Return | Type | Description | | ------------ | ------ | ------------------------------------------------------------------ | | Return Value | `bool` | `true` when a valid `IConvaiVisionInterface` component is present. | ### Project setting #### AlwaysAllowVision `AlwaysAllowVision` is a `Config` property on `UConvaiSettings` in category **Convai** with `AdvancedDisplay`. Open **Edit > Project Settings > Plugins > Convai** to change it. When enabled, initial connection setup uses connection type `video`, which allows projects to register vision components after `BeginPlay`. Blueprint users can also read the setting with **Is Always Allow Vision Enabled** on `UConvaiUtils` (category **Convai | Settings**). ### Next steps {% content-ref url="/pages/jR121aXRSrtaarm2aad5" %} [Vision usage examples](/api-docs/plugins-and-integrations/convai-unreal-engine-plugin/features/vision/vision-usage-examples.md) {% endcontent-ref %} {% content-ref url="/pages/1zs9olIHfeqwXONkvpVT" %} [Vision frame sources](/api-docs/plugins-and-integrations/convai-unreal-engine-plugin/features/vision/vision-frame-sources.md) {% endcontent-ref %} {% content-ref url="/pages/LiRqiZh8lqH02mdwtjJn" %} [Custom vision components](/api-docs/plugins-and-integrations/convai-unreal-engine-plugin/features/vision/custom-vision-components.md) {% endcontent-ref %} --- # Agent Instructions This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com. ## Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter: ``` GET https://docs.convai.com/api-docs/plugins-and-integrations/convai-unreal-engine-plugin/features/vision/vision-blueprint-reference.md?ask=&goal= ``` `ask` is the immediate question: it should be specific, self-contained, and written in natural language. `goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.