> For the complete documentation index, see [llms.txt](https://docs.convai.com/api-docs/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.convai.com/api-docs/zh/cha-jian-yu-ji-cheng/web-plugins/convai-web-sdk/react/custom-ui-and-advanced-usage.md).

# 在 React 中构建自定义 UI

用你自己的 React 消息和控制组件替换预构建小组件，同时由 Convai Web SDK 继续处理音频和状态。

基于 `useConvaiClient` 构建自己的 React 对话界面，而不是渲染 `ConvaiWidget`。当 widget 的布局不适合你的应用时，请参考本页面。完成后，你的组件将能够显示对话、发送文本，并控制麦克风、摄像头和屏幕共享。

### 前提条件

* 已安装 Convai Web SDK 的 React 应用。请参阅 [React 入门](https://docs.convai.com/api-docs/zh/cha-jian-yu-ji-cheng/web-plugins/convai-web-sdk/react.md).
* 一次 `useConvaiClient` 调用来处理对话。应在组件树靠上层的位置创建一次客户端，并将其向下传递；每次调用此 Hook 都会创建一个独立的客户端及其连接。

### 播放角色的音频

不使用 widget 时，没有任何组件会通过默认的 LiveKit 传输播放角色语音。请在 `AudioRenderer` 中渲染 `AudioContext.Provider` ，并将客户端的 room 传入其中。

{% code title="src/App.tsx" %}

```tsx
import { useConvaiClient } from '@convai/web-sdk/react';
import { AudioRenderer, AudioContext } from '@convai/web-widget/react';
import { Conversation } from './Conversation';

export default function App() {
  const convaiClient = useConvaiClient({
    apiKey: 'YOUR_API_KEY',
    characterId: 'YOUR_CHARACTER_ID',
  });

  return (
    <AudioContext.Provider value={convaiClient.room}>
      <AudioRenderer />
      <Conversation convaiClient={convaiClient} />
    </AudioContext.Provider>
  );
}
```

{% endcode %}

`AudioRenderer` 和 `AudioContext` 是 LiveKit 的 `@convai/web-widget/react` 组件，使用 SDK 中的名称。 `RoomAudioRenderer` 和 `RoomContext` 分别是其原始名称。 `AudioContext` 这里的 AudioContext 并非浏览器的 Web Audio `AudioContext`；如果同一个文件也使用 Web Audio，请为它使用另一个名称导入。

浏览器会阻止播放声音，直到访客在页面上单击、轻触或按下按键。如果应用在页面加载时连接，角色在此之前可能会静音，而且此 `AudioRenderer` 本身不会启动播放。请在首次交互时恢复播放：

```tsx
useEffect(() => {
  const start = () => void client.room.startAudio();
  addEventListener('pointerdown', start, { once: true });
  addEventListener('keydown', start, { once: true });
  return () => {
    removeEventListener('pointerdown', start);
    removeEventListener('keydown', start);
  };
}, [client]);
```

直接通过单击操作来连接（例如单击 **开始** 按钮），即可彻底避免此问题。

使用 `transport: "websocket"`时，Pipecat 传输会通过 Web Audio 自行播放语音，而 `AudioRenderer` 不需要。

### 渲染消息

`chatMessages` 包含 SDK 记录的所有消息类型。对于聊天视图，请保留用户输入的文本、语音转录文本以及角色文本。

{% code title="src/Conversation.tsx" %}

```tsx
import { useState } from 'react';
import type { useConvaiClient } from '@convai/web-sdk/react';

type Client = ReturnType<typeof useConvaiClient>;

const SHOWN = new Set(['user-llm-text', 'user-transcription', 'bot-llm-text', 'bot-output']);

export function Conversation({ convaiClient }: { convaiClient: Client }) {
  const [text, setText] = useState('');
  const canSend = convaiClient.state.isConnected && convaiClient.isBotReady;

  const send = () => {
    if (!text.trim() || !canSend) return;
    convaiClient.sendUserTextMessage(text);
    setText('');
  };

  return (
    <div>
      {!convaiClient.state.isConnected && (
        <button onClick={() => convaiClient.connect()}>连接</button>
      )}

      <ul>
        {convaiClient.chatMessages
          .filter((m) => SHOWN.has(m.type) && m.content.trim())
          .map((m) => (
            <li key={m.id} className={m.type.startsWith('user') ? 'user' : 'character'}>
              {m.content}
            </li>
          ))}
      </ul>

      <p>{convaiClient.userTranscription}</p>

      <input value={text} onChange={(e) => setText(e.target.value)} disabled={!canSend} />
      <button onClick={send} disabled={!canSend}>发送</button>

      <Controls convaiClient={convaiClient} />
    </div>
  );
}
```

{% endcode %}

| 类型              | 包含                                                |
| --------------- | ------------------------------------------------- |
| `user-llm-text` | 你通过 `sendUserTextMessage`发送的文本，会在发送时添加到本地。        |
| `发送的文本消息`       | 你语音的最终转录文本。实时的部分转录文本位于 `userTranscription`.       |
| `bot-llm-text`  | 角色以流式方式返回的回复。 `isStreaming` 在 `true` 回复完成之前一直为此值。 |
| `bot-output`    | 角色说出的文本；当没有 `bot-llm-text` 消息携带回复时，此处会记录该文本。      |

消息类型的完整列表请参阅 [配置和类型参考](https://docs.convai.com/api-docs/zh/cha-jian-yu-ji-cheng/web-plugins/convai-web-sdk/vanilla-typescript/best-practices-and-type-definitions.md).

### 添加媒体控件

控件对象不是 React 状态，因此其字段（例如 `audioControls.isAudioMuted`）发生变化时，不会触发组件重新渲染。请改为读取 hook 中具有响应性的 `isAudioMuted`, `isVideoEnabled`和 `isScreenShareActive` ，并调用控件方法来更改这些值。

{% code title="src/Controls.tsx" %}

```tsx
import type { useConvaiClient } from '@convai/web-sdk/react';

type Client = ReturnType<typeof useConvaiClient>;

export function Controls({ convaiClient }: { convaiClient: Client }) {
  const { audioControls, videoControls, screenShareControls } = convaiClient;
  const hasVideo = convaiClient.connectionType === 'video';

  return (
    <>
      <button onClick={() => audioControls.toggleAudio()}>
        {convaiClient.isAudioMuted ? 'Unmute' : 'Mute'}
      </button>

      {hasVideo && (
        <button onClick={() => videoControls.toggleVideo()}>
          {convaiClient.isVideoEnabled ? 'Stop camera' : 'Start camera'}
        </button>
      )}

      {hasVideo && (
        <button onClick={() => screenShareControls.toggleScreenShare()}>
          {convaiClient.isScreenShareActive ? 'Stop sharing' : 'Share screen'}
        </button>
      )}
    </>
  );
}
```

{% endcode %}

这些控制方法会返回 promise，并且可能被拒绝，例如浏览器拒绝访问麦克风或摄像头时。摄像头和屏幕共享要求使用以下选项建立连接： `enableVideo: true`.

### 验证自定义界面

选择 **连接**，等待输入框启用，然后发送一条消息。

{% hint style="success" %}
你的消息会显示在列表中，角色的回复会在下方逐步显示，同时你会听到角色的声音。
{% endhint %}

### 后续步骤

{% content-ref url="/pages/68959b8b127b7dad992c41de6f7e87cd945ce3e3" %}
[useConvaiClient Hook 参考](https://docs.convai.com/api-docs/zh/cha-jian-yu-ji-cheng/web-plugins/convai-web-sdk/react/useconvaiclient-hook.md)
{% endcontent-ref %}

{% content-ref url="/pages/8c6e3631e7e7da5b3c9acf5a0019d7737dc79e19" %}
[Web SDK 事件参考](https://docs.convai.com/api-docs/zh/cha-jian-yu-ji-cheng/web-plugins/convai-web-sdk/event-reference.md)
{% endcontent-ref %}

{% content-ref url="/pages/1ef8009eea712d4c41fd7a2a07e473a277395698" %}
[处理错误和断开连接](https://docs.convai.com/api-docs/zh/cha-jian-yu-ji-cheng/web-plugins/convai-web-sdk/error-handling.md)
{% endcontent-ref %}

{% content-ref url="/pages/40347340c726b64e9b42d20d133807f8d73d6014" %}
[在 React 中添加音频可视化器](https://docs.convai.com/api-docs/zh/cha-jian-yu-ji-cheng/web-plugins/convai-web-sdk/react/audio-visualizer-optional.md)
{% endcontent-ref %}


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation by asking a question.

Perform an HTTP GET request on the following URL with the `ask` and `goal` query parameters:

```
GET https://docs.convai.com/api-docs/zh/cha-jian-yu-ji-cheng/web-plugins/convai-web-sdk/react/custom-ui-and-advanced-usage.md?ask=<question>&goal=<user_goal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is what the user is ultimately trying to achieve, the reason they need the answer. Sharing it helps GitBook give you a better, more relevant answer. A goal is most helpful when it describes the outcome the user wants rather than restating the question. For example, with `ask=how do I create an API token`, a goal like `automate deployments from our CI pipeline` lets GitBook tailor the answer to that use case.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.
