Client Messages

Client messages are JSON commands sent to the session control WebSocket. The current protocol keeps this page separate from Server events because the two directions have different responsibilities: the client requests an action, while the server reports what was accepted, scheduled, started, completed, or failed.

Common rules

Send one JSON object per WebSocket message.
Keep client_request_id stable when retrying the same logical request.
assets.add replays the original result when the same id and payload are repeated; a different payload with that id returns request_conflict.
state.update ignores a repeated request id after it has been accepted. instruction.create is not a deduplicated command, so do not resend it unless you intend to create another instruction.
Ignore fields that are not listed for the message type you are sending.
Server events have a separate numeric id; use that id for reconnecting.

`instruction.create`

Creates a new piece of story direction. It affects content that has not been published yet.

{
  "type": "instruction.create",
  "client_request_id": "choice_001",
  "text": "The detective opens the hidden drawer, finds a torn photograph, and asks the witness who is missing from it.",
  "duration_seconds": 10,
  "state_ops": [
    {
      "op": "set",
      "path": ["clue_count"],
      "value": 1,
      "when": {"type": "instruction_end"}
    }
  ]
}

Field	Required	Description
`type`	Yes	`instruction.create`.
`client_request_id`	Recommended	Your correlation id. Reuse it when retrying the same request.
`text`	Yes	Natural-language story direction.
`duration_seconds`	No	Requested video duration. The service uses its default when omitted.
`state_ops`	No	State changes attached to this instruction.

`assets.add`

Adds image or audio references to the live session. The batch becomes available only after assets.added.

{
  "type": "assets.add",
  "client_request_id": "asset_002",
  "assets": [
    {
      "name": "hidden_photo",
      "type": "image",
      "file_id": "file_img_hidden_photo",
      "description": "A torn photograph found in the drawer"
    }
  ]
}

Each asset must contain exactly one source: file_id or url. Names are unique within the session; use a new name when the image or sound should be replaced.

`state.update`

Applies state changes immediately or schedules them for an existing instruction.

{
  "type": "state.update",
  "client_request_id": "state_001",
  "ops": [
    {
      "op": "set",
      "path": ["characters", "detective", "hp"],
      "value": 85,
      "when": {"type": "immediate"}
    }
  ]
}

ops is accepted as a batch. Supported operations are:

`op`	Behavior
`set`	Create or replace the value at `path`.
`delete`	Remove the value at `path`.
`append`	Append `value` to a list at `path`.
`remove`	Remove a matching value from a list at `path`.

when.type can be immediate, instruction_start, or instruction_end. The latter two require instruction_id when the target is not the instruction currently being created.

`state.get`

Requests a state snapshot for the current WebSocket connection:

{"type":"state.get","client_request_id":"state_read_001"}

The server replies with state.snapshot. This response is private to the requesting connection and is not added to the session event history.

`session.stop`

Stops the session immediately or requests a closing instruction.

{
  "type": "session.stop",
  "closing": {
    "text": "The detective stores the photograph and leaves before the storm clears.",
    "duration_seconds": 8
  }
}

For an immediate stop:

{"type":"session.stop","immediate":true}

Message validation

An invalid message receives an error event. The server rejects the whole message or whole operation batch; it does not partially apply an invalid state_ops or ops list.

Situation	Public code
Missing or invalid field	`invalid_request`
Session is ending	`session_ending`
Session has ended	`session_ended`
Too many queued instructions	`queue_full`
Asset cannot be prepared	`asset_preparation_failed`
Same request id conflicts with a different request	`request_conflict`

See Server events for response shapes and lifecycle events. The page path remains client-events for compatibility with existing links.

Overview

A1: Streaming Avatar

W1: Streaming World

Blazing Fast Video Generation