> For the complete documentation index, see [llms.txt](https://docs.convai.com/api-docs/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.convai.com/api-docs/zh/cha-jian-yu-ji-cheng/web-plugins/convai-web-sdk-1/facial-expressions.md).

# 面部表情

{% hint style="info" %}
:no\_entry: 此插件已被弃用，不再受支持。

\
如果你仍在使用旧插件，我们建议切换到新的 Convai Web SDK： [链接](https://docs.convai.com/api-docs/plugins-and-integrations/web-plugins/convai-web-sdk)

\
仅供历史参考——这些文档已不再维护。
{% endhint %}

### 初始化

要启动面部表情功能，请初始化 `ConvaiClient` 并传入必要参数。 `enableFacialData` 标志必须设置为 `true` 以启用面部表情数据。

```javascript
convaiClient.current = new ConvaiClient({
  apiKey: '<apiKey>',
  characterId: '<characterId>',
  enableAudio: true,
  enableFacialData: true,
  faceModel: 3, // OVR 唇形同步
});
```

{% hint style="info" %}
faceModel：3 为标准且持续维护的版本。
{% endhint %}

### 接收视觉素数据

通过加入提供的回调来获取视觉素数据。示例代码演示了如何处理并更新面部数据。

```javascript
const [facialData, setFacialData] = useState([]);
const facialRef = useRef([]);

convaiClient.current.setResponseCallback((response) => {
if (response.hasAudioResponse()) {
    let audioResponse = response?.getAudioResponse();
      if (audioResponse?.getVisemesData()?.array[0]) {
      // 视觉素数据
        let faceData = audioResponse?.getVisemesData().array[0];
        // faceData[0] 表示 sil 值。当收到新的音频片段时，它为 -2。
          if (faceData[0] !== -2) {
            facialRef.current.push(faceData);
            setFacialData(facialRef.current);
          }
      }
}
```

### 调制形态目标

使用 `useFrame` 钩子，来自 `react-three-fiber` 根据接收到的面部数据调制形态目标。

<pre class="language-javascript"><code class="lang-javascript"><strong>import {OvrToMorph} from 'convai-web-sdk'
</strong><strong>const blendShapeRef = useRef([]);
</strong>const currentBlendFrame = useRef(0);

useFrame((state, _delta) => {
 // 初始化混合形状
    if(client?.facialData.length > 0){
// 映射 reallusion 变形需要 OvrToMorph
    OvrToMorph(client?.facialData[currentBlendFrame.current],blendShapeRef);
    }
  if (currentBlendFrame.current &#x3C;= blendShapeRef?.current?.length) {
    // 根据面部数据调整形态目标的逻辑
    currentBlendFrame.current += 1;
  }
});
</code></pre>

除了面部表情之外， `convai-web-sdk` 还允许开发者为特定面部特征调制骨骼调整。接收“Open\_Jaw”、“Tongue”和“V\_Tongue\_Out”的骨骼调整，并如下所示将其应用到你的角色上：

```javascript

// 如果当前视觉素值是 Open_Jaw
characterRef.current.getObjectByName("CC_Base_JawRoot").setRotationFromEuler(jawRotation);
// 颌部旋转值可以使用 THREE.lerp() 进行调制。
// 示例
const jawRotation = new THREE.Euler(0,0,1.57);
// 这里 1.57 是闭合下颌的基础值。
jawRotation.z = THREE.MathUtils.lerp(jawRotation.z,1.57 + blendShapeRef.current[currentBlendFrame.current-1][blend]*0.2,0.8);

```

{% hint style="info" %}
这些代码示例是针对 Reallusion 角色的。
{% endhint %}

### 处理 100fps 动画

实现 [节流 ](https://www.geeksforgeeks.org/lodash-_-throttle-method/)使用 lodash 以确保在 100fps 下动画平滑。所提供的示例演示了如何保持一致的动画帧率。

```javascript
const throttledUpdate = _.throttle(updateAnimation, 10);
const [tick, setTick] = useState(true);

const updateAnimation = () => {
  setTick((tick) => {
    if (tick) {
      return tick;
    }
    return true;
  });
  requestAnimationFrame(throttledUpdate);
};

// 在组件挂载时启动动画循环
useEffect(() => {
  requestAnimationFrame(throttledUpdate);
  // 在组件卸载时清理动画循环
  return () => {
    cancelAnimationFrame(throttledUpdate);
  };
}, []);

// 使用帧钩子更新动画
useFrame((state, _delta) => {
  if (tick) {
    // 获取视觉素数据并相应地更改形态目标的逻辑
    setTick(false);
  }
});
```

{% hint style="info" %}
注意：节流函数并非 100% 精确
{% endhint %}

### 处理 100fps 边缘情况

节流精度可能会导致边缘情况。实现时钟设置，并使用经过时间处理高于和低于 100fps 的场景。

```javascript
const [startClock, setStartClock] = useState(false);

useFrame((state, _delta) => {
  if (!startClock || !client?.isTalking) {
    state.clock.elapsedTime = 0;
    if (startClock) setStartClock(false);
  }
  if (client?.isTalking) {
    setStartClock(true);
  }
});

// 处理高于和低于 100fps 的场景
if (startClock) {
  if (Math.abs(Math.floor(state.clock.elapsedTime * 100) - currentBlendFrame.current) > 15) {
    if (Math.floor(state.clock.elapsedTime * 100) - currentBlendFrame.current > 0) {
      // 低于 100fps
           for(let i=0;i<15;i++){
              blendShapeRef.current.push(0)
            }
            currentBlendFrame.current += 15;
    } else {
      // 高于 100fps
       if(blendShapeRef.current.length > 15){
            blendShapeRef.current.splice(-15);
            currentBlendFrame.current -= 15;
       }
    }
  }
}


```


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://docs.convai.com/api-docs/zh/cha-jian-yu-ji-cheng/web-plugins/convai-web-sdk-1/facial-expressions.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.