> For the complete documentation index, see [llms.txt](https://docs.convai.com/api-docs/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.convai.com/api-docs/plugins-and-integrations/convai-unity-sdk/embodiment/body-animation/how-body-animation-works.md). # How body animation works Understand the layered animation graph that plays idle, talk, locomotion, action, and pointing motion, and how dialogue state drives it. `ConvaiBodyAnimationController` builds a layered `PlayableGraph` directly against a Humanoid `Animator` — no Animator Controller asset, no states, no transitions authored in the Animator window. This page describes what that graph looks like, what drives each layer, and how the layers resolve when more than one wants the arms at once. *** ### The layer stack The graph runs six layers, from the base pose up through the most specific overlay. | Layer | Mask | Driven by | | ----------------- | ------------------------------------------------- | -------------------------------------------------------------------------------------------------------------- | | Locomotion (base) | Full body | The idle variant pool, plus the NavMesh-synced state machine for starts, walk-jog blending, stops, and turns. | | Talk | Upper body by default, full body per entry | `DialogueState.Speaking`, scaled by live speech energy, released as the response nears its end. | | Action | Full body, upper body, or a custom mask per entry | `PlayAction` calls and Convai actions. A full-body action suspends locomotion and ducks the overlays above it. | | Pointing | Upper body | `PointAt`, with an apex hold and re-aiming while the target moves. | | Moving Talk | Arms and hands | An additive or softened-override walk-and-talk overlay, so a talk gesture does not freeze the arms mid-stride. | | Talk Beat | Arms and hands, additive | Short speech-onset and referential gesture accents. | Every layer reports the weight it wants for a given tick; a single arbiter resolves the final port weights afterward, so no two layers can write conflicting weight to the same port in the same frame. *** ### Content and behavior are two separate assets A `ConvaiBodyAnimationSet` supplies the clips: idle and talk variants, locomotion clips, named actions, and pointing directions. A `ConvaiBodyAnimationConfig` supplies the tuning: fade timings, speech-energy scaling, and the roughly one hundred behavior fields covered in [Body animation config reference](/api-docs/plugins-and-integrations/convai-unity-sdk/embodiment/body-animation/config-reference.md). Assign both directly on the controller, or bundle them together in a `ConvaiBodyAnimationProfile` for preset-based routing across characters. See [Build an animation set](/api-docs/plugins-and-integrations/convai-unity-sdk/embodiment/body-animation/build-an-animation-set.md) for how to author a set's content. *** ### How dialogue state drives the talk layer The talk layer reads [dialogue state](/api-docs/plugins-and-integrations/convai-unity-sdk/core-concepts/dialogue-state.md) to decide which pool of clips to play, not only whether to play at all: | Dialogue state | What plays | | ---------------------------------------------- | ----------------------------------------------------------------------------------- | | `Speaking` | The set's Talk pool, scaled by live speech energy. | | `Listening` / `Attending` | The set's Listen pool, if the set authors one. | | `Thinking`, sustained past a short entry delay | The set's Think pool, if the set authors one. | | `Interrupted` | The current talk pose freezes briefly, then releases faster than a normal fade-out. | Listen and Think are optional pools. A set that does not author them, including the SDK's shipped default set, releases to idle for those states instead of playing a pose — the character never freezes or stands in an incorrect posture for want of content. *** ### How the talk layer releases The talk layer does not wait for `DialogueState` to leave `Speaking` before it starts winding a gesture down. When the character has Lip Sync, the talk layer also reads how much speech is left to play and whether that end is known. Once the remaining time drops to or below **Talk Release Lead Seconds** (`0.6` s by default), the layer begins its normal release early — the gesture decelerates and its weight fades out as the response's last words play, instead of continuing at full weight until `Speaking` ends and then freezing mid-motion. Without a Lip Sync reading, the talk layer releases only once `DialogueState` leaves `Speaking`. See [Body animation config reference](/api-docs/plugins-and-integrations/convai-unity-sdk/embodiment/body-animation/config-reference.md#talking) for **Talk Release Lead Seconds** and the other fields that shape the release — **Talk Release Delay Seconds** and **Talk Release Playback Speed**. *** ### Layer arbitration Action ownership outranks pointing and talk: a running full-body action ducks every other overlay so the two never compete for the same bones. Pointing and talk can play together — a character can point at something while continuing to gesture with the other arm. The arbiter resolves this every tick from each layer's reported desired weight, not from layer order in the hierarchy or the order components were added. *** ### Locomotion is optional `ConvaiNavMeshLocomotion` drives layer 0's walk and jog states, but a character with no locomotion component is a complete, valid setup: idle, talk, actions, gestures, and pointing all work in place. Add locomotion only when the character needs to move — see [Configure locomotion](/api-docs/plugins-and-integrations/convai-unity-sdk/embodiment/body-animation/configure-locomotion.md). *** ### Next steps {% content-ref url="/pages/maeZ3PL41ne89ucE7uOm" %} [Body animation quick start](/api-docs/plugins-and-integrations/convai-unity-sdk/embodiment/body-animation/quick-start.md) {% endcontent-ref %} {% content-ref url="/pages/6GpgVAc2W3BarmbKMCYK" %} [Build an animation set](/api-docs/plugins-and-integrations/convai-unity-sdk/embodiment/body-animation/build-an-animation-set.md) {% endcontent-ref %} {% content-ref url="/pages/8JDFZt8KJ6rwnIUtT66t" %} [Conversation flow](/api-docs/plugins-and-integrations/convai-unity-sdk/embodiment/conversation-flow.md) {% endcontent-ref %} --- # Agent Instructions This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com. ## Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter: ``` GET https://docs.convai.com/api-docs/plugins-and-integrations/convai-unity-sdk/embodiment/body-animation/how-body-animation-works.md?ask=&goal= ``` `ask` is the immediate question: it should be specific, self-contained, and written in natural language. `goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.