> For the complete documentation index, see [llms.txt](https://docs.convai.com/api-docs/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.convai.com/api-docs/zh/cha-jian-yu-ji-cheng/web-plugins/convai-web-sdk/vanilla-typescript/building-a-custom-ui-typescript.md).

# 在 TypeScript 中构建自定义 UI

在原生 TypeScript 中基于 Convai Web SDK 构建你自己的聊天界面，包含连接状态、消息、角色语音和媒体控制。

基于 `ConvaiClient` 构建自己的聊天界面，而不是使用预构建的小组件。当小组件不符合你的设计需求时，可以参考本页面。完成后，你的页面将能够建立连接、显示对话、播放角色语音并切换麦克风。

### 前提条件

* `@convai/web-sdk` 已安装。请参阅 [使用原生 TypeScript 入门](https://docs.convai.com/api-docs/zh/cha-jian-yu-ji-cheng/web-plugins/convai-web-sdk/vanilla-typescript.md).
* Convai API 密钥或身份验证令牌，以及角色 ID。

### 添加标记

{% code title="index.html" %}

```html
<div id="status">disconnected</div>
<button id="connect">Connect</button>
<button id="mic" disabled>Unmute</button>

<div id="chat"></div>

<input id="input" type="text" placeholder="Type a message" disabled />
<button id="send" disabled>Send</button>

<script type="module" src="/src/main.ts"></script>
```

{% endcode %}

### 接入客户端

下面的脚本会在点击后建立连接，并根据 `stateChange`同步控件状态，根据 `messagesChange`呈现对话，并通过 `AudioRenderer`.

{% code title="src/main.ts" %}

```typescript
import {
  AudioRenderer,
  ConvaiClient,
  type ChatMessage,
  type ConvaiClientState,
} from '@convai/web-sdk/vanilla';

const client = new ConvaiClient({
  apiKey: 'YOUR_API_KEY',
  characterId: 'YOUR_CHARACTER_ID',
});

const statusEl = document.getElementById('status') as HTMLDivElement;
const connectBtn = document.getElementById('connect') as HTMLButtonElement;
const micBtn = document.getElementById('mic') as HTMLButtonElement;
const chatEl = document.getElementById('chat') as HTMLDivElement;
const inputEl = document.getElementById('input') as HTMLInputElement;
const sendBtn = document.getElementById('send') as HTMLButtonElement;

let audioRenderer: AudioRenderer | null = null;

client.on('stateChange', (state: ConvaiClientState) => {
  statusEl.textContent = state.isConnected ? state.agentState : 'disconnected';
  connectBtn.disabled = state.isConnected || state.isConnecting;
  micBtn.disabled = !state.isConnected;
});

client.on('botReady', () => {
  inputEl.disabled = false;
  sendBtn.disabled = false;
});

client.on('messagesChange', (messages: ChatMessage[]) => {
  const shown = messages.filter(
    (m) =>
      (m.type === 'user-llm-text' ||
        m.type === 'user-transcription' ||
        m.type === 'bot-llm-text' ||
        m.type === 'bot-output') &&
      m.content.trim(),
  );
  chatEl.replaceChildren(
    ...shown.map((m) => {
      const div = document.createElement('div');
      div.className = m.type.startsWith('user') ? 'user' : 'bot';
      div.textContent = m.content;
      return div;
    }),
  );
  chatEl.scrollTop = chatEl.scrollHeight;
});

// 在默认 WebRTC 传输上播放角色语音。
client.on('connect', () => {
  audioRenderer ??= new AudioRenderer(client.room);
});

client.on('disconnect', () => {
  audioRenderer?.destroy();
  audioRenderer = null;
  inputEl.disabled = true;
  sendBtn.disabled = true;
});

connectBtn.addEventListener('click', async () => {
  try {
    await client.connect();
  } catch (error) {
    statusEl.textContent = error instanceof Error ? error.message : 'connect failed';
  }
});

function sendMessage() {
  const text = inputEl.value.trim();
  if (!text) return;
  client.sendUserTextMessage(text);
  inputEl.value = '';
}

sendBtn.addEventListener('click', sendMessage);
inputEl.addEventListener('keydown', (e) => {
  if (e.key === 'Enter') sendMessage();
});

micBtn.addEventListener('click', async () => {
  try {
    await client.audioControls.toggleAudio();
    micBtn.textContent = client.audioControls.isAudioMuted ? 'Unmute' : 'Mute';
  } catch (error) {
    console.error('Microphone toggle failed:', error);
  }
});
```

{% endcode %}

在默认 WebRTC 传输中，麦克风默认处于关闭状态，除非你设置了 `startWithAudioOn: true`，因此首次点击会请求浏览器授予麦克风权限。媒体控制方法都是异步的；如果浏览器拒绝访问设备，它们会被拒绝，因此请将它们放在 `try`.

{% hint style="warning" %}
如果没有 `AudioRenderer`，基于 WebRTC 传输的自定义 UI 虽然可以连接并接收文本，但不会播放角色语音。聊天小组件会为你创建一个；自定义 UI 则必须自行创建。
{% endhint %}

### 在 WebSocket 传输上播放音频

使用 `transport: 'websocket'`时，传输层会播放语音，无需进行额外配置。Pipecat 的 WebSocket 传输通过自己的 Web Audio 播放器播放角色音频，因此自定义 UI 无需 `AudioRenderer` ，也无需添加 `botAudioTrack` 监听器：

```typescript
import '@convai/web-sdk/vanilla/websocket';
import { ConvaiClient } from '@convai/web-sdk/vanilla';

const client = new ConvaiClient({
  apiKey: 'YOUR_API_KEY',
  characterId: 'YOUR_CHARACTER_ID',
  transport: 'websocket',
});

await client.connect(); // 无需额外代码即可播放角色语音
```

该 `botAudioTrack` 事件不会在此传输方式下触发，因此 `AudioRenderer` 为其创建的实例会保持静音，不会造成任何影响。

请参阅 [使用 WebSocket 传输方式](https://docs.convai.com/api-docs/zh/cha-jian-yu-ji-cheng/web-plugins/convai-web-sdk/websocket-transport-layer.md) 了解其余设置步骤。

### 添加摄像头和屏幕共享控件

摄像头和屏幕共享需要视频连接。在客户端配置中设置 `enableVideo: true` ，然后调用以下控件：

```typescript
await client.videoControls.toggleVideo();
await client.screenShareControls.toggleScreenShare();
```

读取 `client.videoControls.isVideoEnabled` 和 `client.screenShareControls.isScreenShareActive` 以设置按钮标签，或者订阅同一控件对象上的 `videoStateChange` 和 `screenShareStateChange` 事件。

### 验证界面

点击 **连接**。状态会显示 `connected`；角色准备就绪后，输入框会启用。输入消息后，系统会在聊天中回复并朗读回复。点击麦克风按钮并说话：你的话语会以转录文本的形式显示，角色也会作出回复。

### 后续步骤

{% content-ref url="/pages/ffc5fb138ba8c77242a24832c33e0e453dfb2b0f" %}
[在 TypeScript 中处理事件和消息](https://docs.convai.com/api-docs/zh/cha-jian-yu-ji-cheng/web-plugins/convai-web-sdk/vanilla-typescript/events-and-message-handling.md)
{% endcontent-ref %}

{% content-ref url="/pages/f1fc7386520b3ebeeafa719a8d9efb554e1d7024" %}
[ConvaiClient 参考](https://docs.convai.com/api-docs/zh/cha-jian-yu-ji-cheng/web-plugins/convai-web-sdk/vanilla-typescript/convaiclient-core-api.md)
{% endcontent-ref %}

{% content-ref url="/pages/e842433c402c6a1f2b7938d74b7fd4adf7da7152" %}
[在原生 TypeScript 中添加口型同步](https://docs.convai.com/api-docs/zh/cha-jian-yu-ji-cheng/web-plugins/convai-web-sdk/vanilla-typescript/real-time-lipsync.md)
{% endcontent-ref %}


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation by asking a question.

Perform an HTTP GET request on the following URL with the `ask` and `goal` query parameters:

```
GET https://docs.convai.com/api-docs/zh/cha-jian-yu-ji-cheng/web-plugins/convai-web-sdk/vanilla-typescript/building-a-custom-ui-typescript.md?ask=<question>&goal=<user_goal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is what the user is ultimately trying to achieve, the reason they need the answer. Sharing it helps GitBook give you a better, more relevant answer. A goal is most helpful when it describes the outcome the user wants rather than restating the question. For example, with `ask=how do I create an API token`, a goal like `automate deployments from our CI pipeline` lets GitBook tailor the answer to that use case.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.
