> For the complete documentation index, see [llms.txt](https://docs.convai.com/api-docs/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.convai.com/api-docs/zh/cha-jian-yu-ji-cheng/convai-unity-sdk/features/vision.md).

# 视觉

视觉使 Convai 角色能够看到 Unity 场景内部正在发生的事情。帧源会从 Unity 摄像机、物理摄像头或 Meta Quest 透视传输流中捕获图像，并且 SDK 会通过 WebRTC 连同音频对话一起将它们流式传输到 Convai。

### 平台支持

| 平台                | 支持的帧源                                                | 备注                                                              |
| ----------------- | ---------------------------------------------------- | --------------------------------------------------------------- |
| PC / Mac / 主机     | `CameraVisionFrameSource`, `WebcamVisionFrameSource` | 完整 RenderTexture 管线；最高 30 fps                                   |
| Android / iOS     | `CameraVisionFrameSource`, `WebcamVisionFrameSource` | WebCam 源在启动时请求摄像头权限                                             |
| WebGL             | *（Canvas，自动）*                                        | `canvas.captureStream()` 路径——不需要帧源组件；帧率上限为 15 fps； **需要 HTTPS** |
| Meta Quest 3 / 3S | `QuestVisionFrameSource`                             | 需要 Meta XR SDK 和 `horizonos.permission.HEADSET_CAMERA`          |

{% hint style="info" %}
视觉需要 `ConvaiRoomManager.Connection Type` 设置为 **视频**。如果将其设置为 `音频`, `ConvaiVisionPublisher` 无论其他组件如何配置，都会保持空闲。
{% endhint %}

{% hint style="danger" %}
**WebGL：需要 HTTPS。** 该 `canvas.captureStream()` 浏览器会阻止非 HTTPS 来源上的 API。 `http://localhost` 是唯一的例外。在生产环境测试视觉之前，请将你的 WebGL 构建部署到 HTTPS 主机上。
{% endhint %}

<table data-view="cards"><thead><tr><th></th><th data-hidden data-card-target data-type="content-ref"></th></tr></thead><tbody><tr><td><strong>视觉如何工作</strong><br>了解管线架构、关键概念、组件放置和启动顺序。</td><td><a href="/pages/6ebd5ccda4df347e0fb3db3967023f01c8054427">/pages/6ebd5ccda4df347e0fb3db3967023f01c8054427</a></td></tr><tr><td><strong>动态视觉上下文</strong><br>开启帧采样，让角色基于缓冲帧进行推理，而不是对每一帧到来的输入都作出反应，并使用与动态上下文相同的响应模式词汇。</td><td><a href="/pages/4290d3b5bc8fff8787b36689dbdf110d6ebc4c11">/pages/4290d3b5bc8fff8787b36689dbdf110d6ebc4c11</a></td></tr><tr><td><strong>视觉快速入门</strong><br>通过逐步的 Inspector 演示，让角色接收实时摄像头画面——无需编写代码。</td><td><a href="/pages/8688e51bd7c568104d68e6f3c42a8d7ef8b881e8">/pages/8688e51bd7c568104d68e6f3c42a8d7ef8b881e8</a></td></tr><tr><td><strong>视觉帧源</strong><br>为各种平台和使用场景配置 CameraVisionFrameSource、WebcamVisionFrameSource 和 QuestVisionFrameSource。</td><td><a href="/pages/3c41ddceb11fc2af6b830a08a48c437b590c8882">/pages/3c41ddceb11fc2af6b830a08a48c437b590c8882</a></td></tr><tr><td><strong>发布策略</strong><br>选择发布策略，调整帧率和比特率，并了解包括 WebGL 在内的平台特定行为。</td><td><a href="/pages/b15c237537e3521d8e360ca90de5e9b15be340ec">/pages/b15c237537e3521d8e360ca90de5e9b15be340ec</a></td></tr><tr><td><strong>视觉调试预览</strong><br>将当前活动的帧源可视化为屏幕叠加层，并在编辑器中监控捕获状态。</td><td><a href="/pages/885a0e6676256e7738207107695532701ef9b1ec">/pages/885a0e6676256e7738207107695532701ef9b1ec</a></td></tr><tr><td><strong>自定义帧源</strong><br>实现 IVisionFrameSource，以发布任何自定义视频管线——接口约定、Y 轴翻转要求以及最小实现。</td><td><a href="/pages/cb7758289bc48a93210bce51933fd819fd05e245">/pages/cb7758289bc48a93210bce51933fd819fd05e245</a></td></tr><tr><td><strong>Vision 脚本 API</strong><br>ConvaiVisionPublisher 的属性和方法、运行时状态监控，以及用于分析集成的领域事件。</td><td><a href="/pages/9ee175b12da238182d707abd778abdc3a8706c80">/pages/9ee175b12da238182d707abd778abdc3a8706c80</a></td></tr><tr><td><strong>视觉使用示例</strong><br>面向安全培训、设备上岗培训、VR 演示导览和手动触发会话的端到端示例。</td><td><a href="/pages/c296dab9e1d439fdd895aa80e239915e6c1d0943">/pages/c296dab9e1d439fdd895aa80e239915e6c1d0943</a></td></tr><tr><td><strong>视觉故障排除</strong><br>使用结构化检查清单和决策树，诊断发布失败、空白画面、权限错误以及平台特定问题。</td><td><a href="/pages/86612bc613c5a299a468b71b3fb8a40e625ee1fe">/pages/86612bc613c5a299a468b71b3fb8a40e625ee1fe</a></td></tr></tbody></table>

### 动态视觉上下文

动态视觉上下文使 Convai 能够持续对已发布的视频轨进行采样，将其放入滚动帧缓冲区，并把最新帧附加到角色的轮次中，而不是在每一帧到达时都作出反应。它与 [动态上下文](/api-docs/zh/cha-jian-yu-ji-cheng/convai-unity-sdk/features/dynamic-context.md): `ConvaiRespondMode` 值 `静默`, `自动`，以及 `MustRespond` 决定是由新输入发言、由模型决定，还是保持沉默。参见 [动态上下文脚本 API](/api-docs/zh/cha-jian-yu-ji-cheng/convai-unity-sdk/features/dynamic-context/dynamic-context-scripting-api.md) 以获取完整的 `ConvaiRespondMode` 参考资料。

### 下一步

从……开始 [视觉快速入门](/api-docs/zh/cha-jian-yu-ji-cheng/convai-unity-sdk/features/vision/quick-start.md) 以从场景摄像机获取可用的视频流，然后使用 [视觉帧源](/api-docs/zh/cha-jian-yu-ji-cheng/convai-unity-sdk/features/vision/frame-sources.md) 来为你的平台选择正确的捕获方法。有关管线详情，请参见 [视觉如何工作](/api-docs/zh/cha-jian-yu-ji-cheng/convai-unity-sdk/features/vision/how-vision-works.md).


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter:

```
GET https://docs.convai.com/api-docs/zh/cha-jian-yu-ji-cheng/convai-unity-sdk/features/vision.md?ask=<question>&goal=<endgoal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.