> For the complete documentation index, see [llms.txt](https://docs.convai.com/api-docs/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.convai.com/api-docs/plugins-and-integrations/convai-unity-sdk/platform-guides/xr-headsets.md).

# XR headsets

Find guides for deploying the Convai Unity SDK on XR headsets, including Meta Quest Vision passthrough setup and custom XR frame source implementation.

The Convai Unity SDK runs on Android-based XR headsets (Meta Quest, Horizon OS, Android XR) and Windows XR headsets without extra configuration for core features. Voice conversation, lip sync, actions, emotion, and long-term memory work the same as on any other supported platform. Vision is the only feature that requires XR-specific integration work — and only when you want the AI character to see the real world through the headset's cameras.

{% embed url="<https://youtu.be/CGPAG_CWGJw>" %}
Convai Unity VR setup for Meta Quest
{% endembed %}

### Core feature support

| Feature              | Android XR (Meta Quest, Horizon OS)            | Windows XR                              |
| -------------------- | ---------------------------------------------- | --------------------------------------- |
| Voice conversation   | ✅ Full                                         | ✅ Full                                  |
| Lip sync             | ✅ Full                                         | ✅ Full                                  |
| Actions              | ✅ Full                                         | ✅ Full                                  |
| Emotion              | ✅ Full                                         | ✅ Full                                  |
| Long-Term Memory     | ✅ Full                                         | ✅ Full                                  |
| Narrative Design     | ✅ Full                                         | ✅ Full                                  |
| Vision (passthrough) | ✅ `QuestVisionFrameSource` (Quest 3 / 3S only) | ⚠️ Custom `IVisionFrameSource` required |
| Spatial audio        | ✅ Full                                         | ✅ Full                                  |

For Android-based XR headsets, all core setup follows the Android platform guide — including the `RECORD_AUDIO` manifest declaration and runtime permission flow.

{% content-ref url="/pages/32708cb5ab33fa2d1d0f91e747cc06da20220bc1" %}
[iOS and Android](/api-docs/plugins-and-integrations/convai-unity-sdk/platform-guides/ios-and-android.md)
{% endcontent-ref %}

### Push-to-talk on Meta Quest

On Meta Quest, push-to-talk reads the controller's A, B, X, and Y buttons through Unity's XR input API instead of a keyboard key. If the active XR controller cannot be read, or disconnects mid-press, the SDK fails closed and stops microphone capture rather than leaving it open. Keyboard and non-XR joystick input are unaffected. See [Configure conversation input mode](/api-docs/plugins-and-integrations/convai-unity-sdk/getting-started/configure-conversation-input-mode.md) for the full button mapping.

Vision is the only feature that varies between XR platforms. The decision tree below shows when extra setup is needed:

```mermaid
graph TD
    A[XR Deployment] --> B{Vision needed?}
    B -->|No| C[Follow Android or Windows guide for permissions]
    C --> D[Done — all core features work]
    B -->|Yes| E{Meta Quest 3 or 3S?}
    E -->|Yes| F[Use QuestVisionFrameSource]
    F --> G[Import Meta XR SDK + declare manifest permissions]
    E -->|No — other XR headset| H[Implement custom IVisionFrameSource]
    H --> I[See Vision on other XR platforms below]
```

### Vision on Meta Quest

`QuestVisionFrameSource` streams the passthrough camera feed from a Meta Quest 3 or 3S headset to Convai, enabling the AI character to see and respond to the real world the learner is looking at. The component uses reflection to bind to the Meta XR SDK's `PassthroughCameraAccess` at runtime — the Convai SDK has no hard compile-time dependency on any Meta SDK package.

For the complete setup guide, Inspector reference, and troubleshooting steps, see the Meta Quest Vision setup page.

<table data-view="cards"><thead><tr><th></th><th data-hidden data-card-target data-type="content-ref"></th></tr></thead><tbody><tr><td><strong>Meta Quest Vision setup</strong><br>Configure QuestVisionFrameSource, declare required manifest permissions, and validate passthrough Vision on Quest 3 and 3S.</td><td><a href="/api-docs/plugins-and-integrations/convai-unity-sdk/platform-guides/meta-quest-vision.md">Meta Quest Vision setup</a></td></tr></tbody></table>

### Vision on other XR platforms

No built-in frame source exists for non-Meta XR headsets — OpenXR-only devices, Windows Mixed Reality, HoloLens, or Android XR platforms that do not expose a passthrough camera through the Meta XR SDK.

To enable Vision on these devices, implement `IVisionFrameSource` and supply frames from your XR SDK's camera API:

{% code title="CustomXRVisionFrameSource.cs" %}

```csharp
using System;
using Convai.Runtime.Vision.Sources;
using UnityEngine;

public class CustomXRVisionFrameSource : MonoBehaviour, IVisionFrameSource
{
    [SerializeField] private float _targetFrameRate = 15f;
    [SerializeField] private string _sourceId = "custom-xr";

    private RenderTexture _renderTexture;

    public bool IsCapturing { get; private set; }
    public long FrameCount { get; private set; }
    public (int Width, int Height) FrameDimensions => (_renderTexture ? _renderTexture.width : 0,
                                                       _renderTexture ? _renderTexture.height : 0);
    public float TargetFrameRate => _targetFrameRate;
    public string SourceId => _sourceId;
    public RenderTexture CurrentRenderTexture => _renderTexture;
    public bool IsFrameReady { get; private set; }

    public event Action FrameReady;

    public void StartCapture()
    {
        // Initialize your XR SDK camera and RenderTexture here.
        // Each time a new frame is available, blit it into _renderTexture (top-down / Y-flipped),
        // then increment FrameCount, set IsFrameReady = true, and raise FrameReady?.Invoke().
        IsCapturing = true;
    }

    public void StopCapture()
    {
        IsCapturing = false;
        IsFrameReady = false;
    }
}
```

{% endcode %}

Assign your custom source to `ConvaiVisionPublisher` via its Inspector field. See the Vision feature documentation for the full `IVisionFrameSource` contract, publishing policies, and debug preview setup.

### Next steps

{% content-ref url="/pages/iRseCYB4vBfleWvWqnEO" %}
[Meta Quest Vision setup](/api-docs/plugins-and-integrations/convai-unity-sdk/platform-guides/meta-quest-vision.md)
{% endcontent-ref %}

{% content-ref url="/pages/32708cb5ab33fa2d1d0f91e747cc06da20220bc1" %}
[iOS and Android](/api-docs/plugins-and-integrations/convai-unity-sdk/platform-guides/ios-and-android.md)
{% endcontent-ref %}


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation by asking a question.

Perform an HTTP GET request on the following URL with the `ask` and `goal` query parameters:

```
GET https://docs.convai.com/api-docs/plugins-and-integrations/convai-unity-sdk/platform-guides/xr-headsets.md?ask=<question>&goal=<user_goal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is what the user is ultimately trying to achieve, the reason they need the answer. Sharing it helps GitBook give you a better, more relevant answer. A goal is most helpful when it describes the outcome the user wants rather than restating the question. For example, with `ask=how do I create an API token`, a goal like `automate deployments from our CI pipeline` lets GitBook tailor the answer to that use case.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.