> For the complete documentation index, see [llms.txt](https://docs.convai.com/api-docs/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.convai.com/api-docs/plugins-and-integrations/convai-unreal-engine-plugin/features/character-actions/attention-and-reference-grounding.md).

# Attention and reference grounding

Most character actions work without any attention setup — reference parameters resolve automatically from the registered object list. Read this page when you want the character to understand pronouns like "it", "that", and "there", or when you need to control which object a character is focused on.

When a player says "Go to it" or "Pick that up," the word "it" or "that" must resolve to a specific object. The plugin solves this through two connected mechanisms: reference grounding (resolving object names from Convai's output against the registered environment) and attention (tracking which object the character or player is currently focused on so pronoun references resolve predictably).

### Reference grounding

When Convai returns an action with a `Reference`-typed parameter, the raw value is a name string chosen from the registered object and character names. The plugin resolves that string against `EnvironmentData.Objects` and `EnvironmentData.Characters` by exact registered name.

The result is stored in `FConvaiResultParam.RefValue` as a populated `FConvaiObjectEntry` (including the live `Ref` Actor pointer). An unresolvable value produces an empty `FConvaiObjectEntry` with a null `Ref`.

Reference grounding works against the live `FConvaiEnvironmentData` on the chatbot component. Objects added via `AddObject` at runtime update the local environment immediately, so later action parsing can resolve against the current local object and character lists. The action contract fixes which action templates the character can perform until the next reconnect, and mid-session scene-context updates refresh scene context rather than expanding the connect-time action target set.

### Attention source priority

The attention slot has an ownership model that determines which source can write to it:

| Source     | Priority | Set by                                       | Can be overwritten by          |
| ---------- | -------- | -------------------------------------------- | ------------------------------ |
| `None`     | —        | Default / cleared state                      | Any source                     |
| `Gaze`     | Low      | Gaze pipeline via `UConvaiObjectComponent`   | Another gaze event or Explicit |
| `Explicit` | High     | Blueprint/C++ call to `SetObjectInAttention` | Only another Explicit call     |

**Explicit wins.** Once `SetObjectInAttention` sets the slot, gaze events cannot overwrite it until an explicit clear is issued.

### The attention slot

The attention slot is a single `FConvaiObjectEntry` stored as `CurrentAttentionObject` in `FConvaiEnvironmentData`. The plugin includes this slot in the action configuration and scene context so Convai has a current focus object when interpreting ambiguous references such as `"this"`, `"that"`, `"it"`, and `"there"`.

The attention slot has an associated source flag on the chatbot component:

| `EConvaiAttentionSource` value | Meaning                                                                                     |
| ------------------------------ | ------------------------------------------------------------------------------------------- |
| `None`                         | No object is in attention.                                                                  |
| `Explicit`                     | A Blueprint or C++ call to `SetObjectInAttention` set the slot. Gaze cannot overwrite this. |
| `Gaze`                         | The gaze pipeline set the slot. Another gaze event can overwrite it.                        |

#### Setting attention from Blueprint

Call `SetObjectInAttention` on the `Convai Chatbot` component to explicitly set the attention object:

```
// Blueprint pseudocode
SetObjectInAttention(
    AttentionObject: FConvaiObjectEntry,  // the object to focus on
    Text: FString,                         // optional context event text
    ShouldRespond: EC_RunLLMOption,        // how Convai should react
    bFlushImmediately: bool                // bypass debounce if urgent
)
```

Pass an empty `FConvaiObjectEntry` (default-constructed, with an empty `Name`) to clear the slot.

`SetObjectInAttention` has no effect when `EnvironmentData.bEnableActions` is `false` on the chatbot — Convai only resolves attention when `action_config` was sent at `/connect`.

#### Setting attention from gaze

The gaze pipeline uses `TrySetObjectInAttentionFromGaze` and `TryClearObjectInAttentionFromGaze`. These are gated by the ownership rule:

* `TrySetObjectInAttentionFromGaze` succeeds when `AttentionSource` is `None` or `Gaze`. It returns `false` when the slot is `Explicit`, leaving the explicit owner intact.
* `TryClearObjectInAttentionFromGaze` clears the slot only when the chatbot still considers gaze the owner **and** the expected object matches the currently-attended object. This prevents stale "gaze lost" events from wiping a newer target.

`UConvaiObjectComponent` drives these calls automatically via `NotifyGazeAttentionBegin` and `NotifyGazeAttentionEnd`. You typically do not need to call the gaze setters directly unless you are building a custom focus system. For the full trace pipeline, highlight behavior, and component-scoped targeting rules, see [How gaze attention works](/api-docs/plugins-and-integrations/convai-unreal-engine-plugin/features/gaze-attention/how-gaze-attention-works.md).

#### AttentionSource property

`AttentionSource` is a `Transient`, Blueprint-readable `EConvaiAttentionSource` property on `UConvaiChatbotComponent`. Read it to decide whether your Blueprint logic should call the Explicit or gaze-gated setter:

```
// Blueprint pseudocode
// Before setting explicit attention, check the current source
Source = ChatbotComponent.AttentionSource
// Source == None or Gaze → safe to call SetObjectInAttention
// Source == Explicit → check whether your call should override the existing explicit owner
```

### How gaze promotes an object to attention

The gaze pipeline runs automatically through `UConvaiObjectComponent`. You typically only need to read `AttentionSource` or call `SetObjectInAttention` directly; the steps below document how the automatic path works.

When `UConvaiObjectComponent` is added to a scene Actor and the player looks at it:

1. `UConvaiPlayerComponent`'s gaze pipeline fires `NotifyGazeBegin` on the object component, triggering `OnGazedIn`.
2. When the player's gaze dwell time exceeds the configured threshold, `NotifyGazeAttentionBegin` fires.
3. `NotifyGazeAttentionBegin` fans out to every registered chatbot in the subsystem and calls `TrySetObjectInAttentionFromGaze`.
4. Each chatbot independently accepts or rejects the update based on its own `AttentionSource`.

When the player looks away, `NotifyGazeAttentionEnd` fans out `TryClearObjectInAttentionFromGaze` to every chatbot.

### Relationship to action parameter resolution

When Convai returns an action with a `Reference`-typed parameter, the value is grounded against the full `Objects` and `Characters` lists — not only the current attention object. The attention object only influences which object Convai selects when the player uses an ambiguous pronoun. Once selected, the normal reference-grounding path resolves it to a live `Ref` pointer.

### LookAtTarget and PointAtTarget

`UConvaiChatbotComponent` exposes two replicated Actor references for animation:

* `LookAtTarget` — the Actor the character's AnimBP should drive gaze/IK toward.
* `PointAtTarget` — the Actor the character's arm IK should gesture toward.

These are independent of the attention slot and have no effect on the conversation or action pipeline. Set them from Blueprint to drive animation state without affecting how Convai resolves object references.

### Next steps

{% content-ref url="/pages/qIWRz61DFTTZdII0U2XA" %}
[Configuring actions](/api-docs/plugins-and-integrations/convai-unreal-engine-plugin/features/character-actions/configuring-actions.md)
{% endcontent-ref %}

{% content-ref url="/pages/lEX58Owo3bAVjCGO0ezK" %}
[Actions Blueprint reference](/api-docs/plugins-and-integrations/convai-unreal-engine-plugin/features/character-actions/actions-blueprint-reference.md)
{% endcontent-ref %}


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter:

```
GET https://docs.convai.com/api-docs/plugins-and-integrations/convai-unreal-engine-plugin/features/character-actions/attention-and-reference-grounding.md?ask=<question>&goal=<endgoal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.