> For the complete documentation index, see [llms.txt](https://docs.convai.com/api-docs/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.convai.com/api-docs/zh/cha-jian-yu-ji-cheng/convai-unity-sdk/features/vision.md).

# 视觉

查找在 Unity 中为 Convai 角色添加实时场景视觉的指南，包括帧源、发布策略、脚本编写和故障排查。

视觉功能让 Convai 角色能够看到 Unity 场景中发生的事情。帧源会从 Unity 摄像机、实体网络摄像头或 Meta Quest 透视画面中捕获图像，SDK 则通过 WebRTC 将这些图像与音频对话一同流式传输到 Convai。

### 平台支持

| 平台                | 支持的帧源                                                | 备注                                                              |
| ----------------- | ---------------------------------------------------- | --------------------------------------------------------------- |
| PC / Mac / 主机     | `CameraVisionFrameSource`, `WebcamVisionFrameSource` | 完整的 RenderTexture 流程；最高 30 fps                                  |
| Android / iOS     | `CameraVisionFrameSource`, `WebcamVisionFrameSource` | 网络摄像头源会在启动时请求摄像头权限                                              |
| WebGL             | *（Canvas，自动）*                                        | `canvas.captureStream()` 路径 — 无需帧源组件；帧率上限为 15 fps； **需要 HTTPS** |
| Meta Quest 3 / 3S | `QuestVisionFrameSource`                             | 需要 Meta XR SDK 和 `horizonos.permission.HEADSET_CAMERA`          |

{% hint style="info" %}
视觉功能要求将 `ConvaiRoomManager.Connection Type` 设置为 **视频**。如果设置为 `Audio`, `ConvaiVisionPublisher` 将保持空闲状态，无论其他组件如何配置。
{% endhint %}

{% hint style="danger" %}
**WebGL：需要 HTTPS。** canvas.captureStream() `API` 会被浏览器在非 HTTPS 来源上屏蔽。 `http://localhost` 是唯一的例外。在生产环境中测试视觉功能之前，请先将 WebGL 构建部署到 HTTPS 主机。
{% endhint %}

<table data-view="cards"><thead><tr><th></th><th data-hidden data-card-target data-type="content-ref"></th></tr></thead><tbody><tr><td><strong>视觉功能的工作原理</strong><br>了解管线架构、关键概念、组件放置方式和启动顺序。</td><td><a href="https://docs.convai.com/api-docs/zh/cha-jian-yu-ji-cheng/convai-unity-sdk/features/vision/how-vision-works.md">视觉的工作原理</a></td></tr><tr><td><strong>动态视觉上下文</strong><br>启用帧采样，让角色基于缓冲帧进行推理，而不是处理每一帧传入画面；其响应模式词汇与动态上下文相同。</td><td><a href="https://docs.convai.com/api-docs/zh/cha-jian-yu-ji-cheng/convai-unity-sdk/features/vision/dynamic-vision-context.md">动态视觉上下文</a></td></tr><tr><td><strong>视觉功能快速入门</strong><br>通过分步的 Inspector 操作指南，让角色接收实时摄像头画面 — 无需编写代码。</td><td><a href="https://docs.convai.com/api-docs/zh/cha-jian-yu-ji-cheng/convai-unity-sdk/features/vision/quick-start.md">视觉快速开始</a></td></tr><tr><td><strong>视觉帧源</strong><br>针对各种平台和使用场景配置 CameraVisionFrameSource、WebcamVisionFrameSource 和 QuestVisionFrameSource。</td><td><a href="https://docs.convai.com/api-docs/zh/cha-jian-yu-ji-cheng/convai-unity-sdk/features/vision/frame-sources.md">视觉帧源</a></td></tr><tr><td><strong>发布策略</strong><br>选择发布策略、调整帧率和比特率，并了解包括 WebGL 在内的平台特定行为。</td><td><a href="https://docs.convai.com/api-docs/zh/cha-jian-yu-ji-cheng/convai-unity-sdk/features/vision/publishing-and-policies.md">发布策略</a></td></tr><tr><td><strong>视觉调试预览</strong><br>将活动帧源显示为屏幕叠加层，并在 Editor 中监控捕获状态。</td><td><a href="https://docs.convai.com/api-docs/zh/cha-jian-yu-ji-cheng/convai-unity-sdk/features/vision/debug-preview.md">视觉调试预览</a></td></tr><tr><td><strong>自定义帧源</strong><br>实现 IVisionFrameSource，以发布任意自定义视频管线 — 包括接口约定、Y 轴翻转要求和最简实现。</td><td><a href="https://docs.convai.com/api-docs/zh/cha-jian-yu-ji-cheng/convai-unity-sdk/features/vision/custom-frame-sources.md">自定义帧来源</a></td></tr><tr><td><strong>视觉脚本 API</strong><br>ConvaiVisionPublisher 的属性和方法、运行时状态监控，以及用于分析集成的领域事件。</td><td><a href="https://docs.convai.com/api-docs/zh/cha-jian-yu-ji-cheng/convai-unity-sdk/features/vision/scripting-api.md">Vision 脚本 API</a></td></tr><tr><td><strong>视觉功能使用示例</strong><br>涵盖安全培训、设备入门、VR 导览和手动触发会话的端到端示例。</td><td><a href="https://docs.convai.com/api-docs/zh/cha-jian-yu-ji-cheng/convai-unity-sdk/features/vision/usage-examples.md">视觉使用示例</a></td></tr><tr><td><strong>视觉功能故障排查</strong><br>使用结构化检查清单和决策树，诊断发布失败、画面空白、权限错误和平台特定问题。</td><td><a href="https://docs.convai.com/api-docs/zh/cha-jian-yu-ji-cheng/convai-unity-sdk/features/vision/troubleshooting-and-diagnostics.md">排查视觉问题</a></td></tr></tbody></table>

### 动态视觉上下文

动态视觉上下文会持续从已发布的视频轨道采样，并将帧存入循环缓冲区；它会把最新帧附加到角色的每轮对话中，而不是对每一帧的到达都做出响应。它与以下功能共用响应模式词汇： [动态上下文](https://docs.convai.com/api-docs/zh/cha-jian-yu-ji-cheng/convai-unity-sdk/features/dynamic-context.md): `ConvaiRespondMode` 值 `Silent`, `Auto`，以及 `MustRespond` 用于决定新输入是否会触发语音、由模型自行决定，还是保持静默。完整 [动态上下文脚本 API](https://docs.convai.com/api-docs/zh/cha-jian-yu-ji-cheng/convai-unity-sdk/features/dynamic-context/dynamic-context-scripting-api.md) 请参阅 `ConvaiRespondMode` 参考文档。

### 后续步骤

请从 [视觉快速入门](https://docs.convai.com/api-docs/zh/cha-jian-yu-ji-cheng/convai-unity-sdk/features/vision/quick-start.md) 开始，先从场景摄像机获取可用的视频流，然后使用 [视觉帧源](https://docs.convai.com/api-docs/zh/cha-jian-yu-ji-cheng/convai-unity-sdk/features/vision/frame-sources.md) 为你的平台选择合适的捕获方法。有关管线的详细信息，请参阅 [视觉工作原理](https://docs.convai.com/api-docs/zh/cha-jian-yu-ji-cheng/convai-unity-sdk/features/vision/how-vision-works.md).


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation by asking a question.

Perform an HTTP GET request on the following URL with the `ask` and `goal` query parameters:

```
GET https://docs.convai.com/api-docs/zh/cha-jian-yu-ji-cheng/convai-unity-sdk/features/vision.md?ask=<question>&goal=<user_goal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is what the user is ultimately trying to achieve, the reason they need the answer. Sharing it helps GitBook give you a better, more relevant answer. A goal is most helpful when it describes the outcome the user wants rather than restating the question. For example, with `ask=how do I create an API token`, a goal like `automate deployments from our CI pipeline` lets GitBook tailor the answer to that use case.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.
