> For the complete documentation index, see [llms.txt](https://docs.convai.com/api-docs/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.convai.com/api-docs/plugins-and-integrations/convai-unreal-engine-plugin/features/scene-metadata/managing-the-environment-at-runtime.md).

# Managing the environment at runtime

Use these methods to update a chatbot's world knowledge while the game is running — for example, when a new room loads, a prop spawns, or a new NPC enters the scene. `UConvaiChatbotComponent` exposes mutation methods for objects, characters, the active conversation partner, the in-attention object, and the connect-time environment extras. Runtime mutation methods are `BlueprintCallable` in the `Convai|Actions` category; `GatherEnvironmentExtras` is `BlueprintCallable` in the `Convai|Session` category.

### Method groups at a glance

| Group                | Methods                                                                                        | Purpose                                            |
| -------------------- | ---------------------------------------------------------------------------------------------- | -------------------------------------------------- |
| Objects              | `AddObject`, `AddObjects`, `RemoveObject`, `RemoveObjects`, `ClearObjects`                     | Add or remove world objects mid-session            |
| Characters           | `AddCharacter`, `AddCharacters`, `RemoveCharacter`, `RemoveCharacters`, `ClearCharacters`      | Add or remove other NPCs the chatbot can reference |
| Conversation partner | `SetConversationPartner`                                                                       | Tell the chatbot who it is speaking with           |
| Attention            | `SetObjectInAttention`, `TrySetObjectInAttentionFromGaze`, `TryClearObjectInAttentionFromGaze` | Set, gaze-gate, or clear the in-attention object   |
| Session start        | `GatherEnvironmentExtras`                                                                      | Populate extras before `/connect`                  |
| Utility              | `EnsureObjectComponentsForEnvironmentObjects`                                                  | Spawn missing `UConvaiObjectComponent` instances   |

Most methods that push network updates accept a `bFlushImmediately` parameter. `TryClearObjectInAttentionFromGaze` clears through the non-immediate path. See [Debounce and flush](#debounce-and-flush) for details.

### The `FConvaiEnvironmentData` struct

`EnvironmentData` (`FConvaiEnvironmentData`) is the static configuration that `UConvaiChatbotComponent` sends to Convai at `/connect` time. It holds:

* `bEnableActions` (`bool`, default `true`) — connect-time action configuration toggle. When `false`, the `action_config` block is not sent at `/connect`, and attention resolution through `SetObjectInAttention` has no effect.
* `Actions` — the default action list (`Move To`, `Follow`, `Stop Moving`, `Wait For`).
* `Objects` — the objects exposed to the chatbot at connect time.
* `Characters` — the characters exposed to the chatbot at connect time.
* `CurrentAttentionObject` — the object currently used for attention and reference resolution.

Mutate the environment through the methods below rather than writing to `EnvironmentData` fields directly from Blueprint.

### Adding and removing objects

| Method                                          | Description                                                                                                                       |
| ----------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------- |
| `AddObject(Object, bFlushImmediately)`          | Adds one `FConvaiObjectEntry` and schedules an `update-scene-metadata` push when the object was not in the connect-time snapshot. |
| `AddObjects(Objects, bFlushImmediately)`        | Adds multiple entries in one call.                                                                                                |
| `RemoveObject(ObjectName, bFlushImmediately)`   | Removes the entry matching the given name and schedules a sync.                                                                   |
| `RemoveObjects(ObjectNames, bFlushImmediately)` | Removes multiple entries by name.                                                                                                 |
| `ClearObjects(bFlushImmediately)`               | Removes all objects from the local list.                                                                                          |

{% hint style="warning" %}
If an object was included in `action_config` at `/connect`, calling `AddObject` with the same name mid-session does not update the frozen `action_config` snapshot. Only objects that are new to the session travel through the live `update-scene-metadata` lane. To propagate a description change to an existing object, call `StopSession` then `StartSession` to reconnect.
{% endhint %}

### Adding and removing characters

| Method                                                  | Description                                           |
| ------------------------------------------------------- | ----------------------------------------------------- |
| `AddCharacter(Character, bFlushImmediately)`            | Adds one `FConvaiObjectEntry` to the characters list. |
| `AddCharacters(Characters, bFlushImmediately)`          | Adds multiple entries.                                |
| `RemoveCharacter(InCharacterName, bFlushImmediately)`   | Removes the matching entry by name.                   |
| `RemoveCharacters(InCharacterNames, bFlushImmediately)` | Removes multiple entries by name.                     |
| `ClearCharacters(bFlushImmediately)`                    | Removes all characters from the local list.           |

### Setting the conversation partner

`SetConversationPartner(Partner, bFlushImmediately)` tells the chatbot which other character it is currently speaking with. If `Partner.Name` is not already in `EnvironmentData.Characters`, the method adds it automatically.

To clear the conversation partner without removing the character from the list, pass an `FConvaiObjectEntry` with an empty `Name`.

### Controlling attention

`SetObjectInAttention(Object, Text, ShouldRespond, bFlushImmediately)` sets the object the chatbot is focused on and optionally triggers a context event. The call has no effect when `bEnableActions` is `false`. If the object is not already in the objects list, the method adds it automatically.

`AttentionSource` (`EConvaiAttentionSource`, `Transient`, `BlueprintReadOnly`, category `Convai|Actions`) reflects who last set the attention:

* `None` — no attention target is set.
* `Explicit` (`Explicit (Blueprint/C++)`) — attention was set by Blueprint or C++ code.
* `Gaze` — attention was set by the gaze pipeline.

When `AttentionSource` is `Explicit`, calls to `TrySetObjectInAttentionFromGaze` are blocked and return `false`. This prevents the gaze system from overriding explicit programmatic attention.

`TryClearObjectInAttentionFromGaze(ExpectedObject)` clears a gaze-owned attention slot only when `AttentionSource` is `Gaze` and the current attention object still matches `ExpectedObject.Name`. This protects a newer attention target from being cleared by a late gaze-end event from an older target.

### Populating the environment at session start

Use `GatherEnvironmentExtras` when your level is procedurally generated or when you need to build the object list from a runtime world query rather than placing components manually in the Details panel.

`GatherEnvironmentExtras` is a `BlueprintNativeEvent` (display name `"Gather Environment Extras"`) called once inside `StartSession()` before the `/connect` handshake. Override it in a Blueprint subclass of `UConvaiChatbotComponent` to append objects, characters, or actions that depend on runtime world state.

The override receives three output arrays:

* `OutExtraActions` — appended to the default action list.
* `OutExtraObjects` — appended to `EnvironmentData.Objects`.
* `OutExtraCharacters` — appended to `EnvironmentData.Characters`.

The override adds to the configured defaults; it does not replace them. This is the correct place to populate the environment from a world query rather than from the Details panel.

See [Scene metadata usage examples](/api-docs/plugins-and-integrations/convai-unreal-engine-plugin/features/scene-metadata/scene-metadata-usage-examples.md) for a worked pseudocode example of this pattern.

### Ensuring object components

`EnsureObjectComponentsForEnvironmentObjects()` iterates over `EnvironmentData.Objects`. For each entry with a valid `Ref` actor, it spawns a matching `UConvaiObjectComponent` when one does not already cover the same target. It returns the count of newly spawned components and is safe to call multiple times (idempotent).

Auto-spawned components register with the subsystem-wide object pool, so every chatbot in the level can see them through the same proximity, tracked-property, and gaze pipeline as manually placed object components.

Call it at `BeginPlay` after populating `EnvironmentData.Objects` from code, or after calling `AddObject` / `AddObjects` when you want the proximity and gaze systems to track dynamically added actors.

### Debounce and flush

All mutation methods batch updates into a debounce window, coalescing rapid calls into a single WebRTC message. Pass `bFlushImmediately = true` to bypass the window and send immediately. Use `bFlushImmediately` sparingly — high-frequency calls produce many small messages and increase network overhead.

### Next steps

{% content-ref url="/pages/4fgSEKKPL6fKko07SAoV" %}
[Scene metadata usage examples](/api-docs/plugins-and-integrations/convai-unreal-engine-plugin/features/scene-metadata/scene-metadata-usage-examples.md)
{% endcontent-ref %}

{% content-ref url="/pages/vDCzfp22bc2x6dppbHkD" %}
[Troubleshoot scene metadata](/api-docs/plugins-and-integrations/convai-unreal-engine-plugin/features/scene-metadata/troubleshoot-scene-metadata.md)
{% endcontent-ref %}


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter:

```
GET https://docs.convai.com/api-docs/plugins-and-integrations/convai-unreal-engine-plugin/features/scene-metadata/managing-the-environment-at-runtime.md?ask=<question>&goal=<endgoal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.