Server events

Server events are JSON messages emitted by the service on the Streaming Avatar WSS connection (control.url, which points to /v1/realtime-avatar/control). Each event includes an event_id and a type. Error events may also include the client event_id that caused the failure. Forward compatibility. Vivix may add new event types and fields over time. Clients must ignore any event type or field they do not recognize and must not disconnect on unknown data.

1. Which event should I listen to?

Know a response was accepted and has an id. · Listen to: response.created Show live text or captions. · Listen to: response.output_text.delta Show final text for a streamed text item. · Listen to: response.output_text.done Receive the final output item, including completed function calls. · Listen to: response.output_item.done Check final response status. · Listen to: response.done Track when avatar media starts or stops rendering over RTC. · Listen to: response.render.started and response.render.stopped Track user speech activity and transcription. · Listen to: input_audio.speech_started, input_audio.speech_stopped, and conversation.item.input_audio_transcription.*

2. Error Events

error

Emitted when client or server processing fails. Most errors are recoverable and do not close the session. error: object Error details. error.message: string Human-readable error message. error.type: string The type of error, such as invalid_request_error or server_error. error.code: optional string Machine-readable error code, when available. error.event_id: optional string Client event id that caused the error, when applicable. error.param: optional string Request parameter related to the error, when applicable. event_id: string Server-generated unique event id. type: “error” Event type. Must be error. error

{
  "event_id": "evt_error_001",
  "type": "error",
  "error": {
    "message": "The 'type' field is missing.",
    "type": "invalid_request_error",
    "code": "invalid_event",
    "event_id": "evt_session_update_001"
  }
}

3. Session Events

`session.closed`

When a session closes, the service attempts to send this event before disconnecting the remaining control connections. When received, stop sending controls and clean up local media. Do not rely on this event alone: it may be missed if the connection has already closed. It means realtime interaction has ended, but asynchronous resource cleanup may still be in progress. Use Get Session for authoritative lifecycle state. event_id: string type: “session.closed” session_id: string reason: string session.closed

{
  "type": "session.closed",
  "event_id": "evt_123",
  "session_id": "d411368c-714e-4951-9a28-c11c083d07d8",
  "reason": "interaction_idle_timeout"
}

`session.updated`

media_transport · optional string. The current video transport, when returned. media · optional object. Use the updated connection details when present. See Sessions API for the transport-specific shape. Emitted after a requested session.update is applied. Returns the effective mutable session state. event_id: string The effective update request ID: the client event ID when supplied, otherwise generated by the server. type: “session.updated” Event type. Must be session.updated. session: object Effective mutable session state after the update is applied. session.mode: string Effective session mode, one of text_chat or video_avatar. session.active_avatar_id: string Avatar whose instructions apply by default for later responses. session.source_image_id: nullable string Effective source image for avatar video. Returns null in text_chat mode. session.updated

{
  "event_id": "evt_session_update_0001",
  "type": "session.updated",
  "session": {
    "mode": "video_avatar",
    "active_avatar_id": "host_a",
    "source_image_id": "side"
  }
}

`session.update.done`

The terminal result for an accepted session.update. Correlate it with request_event_id. session.updated acknowledges the update; it does not replace the completion result. event_id · string, server event ID.
type · always session.update.done.
request_event_id · string, original request ID.
status · string. One of completed, cancelled, or failed.
reason · optional string for cancelled or failed results, such as superseded or session_shutdown.

{
  "event_id": "evt_update_done_1",
  "type": "session.update.done",
  "request_event_id": "evt_image_side_1",
  "status": "completed"
}

A request that fails validation returns error; do not keep waiting for completion. session.update.done is not retained or replayed.

`session.bootstrap.opening.done`

The bootstrap terminal event may replay to a new control connection. session.update.done is not retained or replayed. There is no guaranteed global order between these event types. Handle unfamiliar reasons without failing. The terminal result for the opening. event_id · string, server-generated event ID. type · always session.bootstrap.opening.done. status · completed, cancelled, or failed. reason · optional string, present only for cancelled or failed results, such as superseded or session_shutdown.

{
  "event_id": "evt_opening_1",
  "type": "session.bootstrap.opening.done",
  "status": "completed"
}

These events describe server-side processing. Use the player state to determine whether the browser is displaying the result.

4. Registry Events

`asset.added`

Confirms that an asset.add event was accepted and that the registered asset ids are available for later response.script calls. event_id: string Unique id for this server event. type: “asset.added” Event type. Must be asset.added. assets: object Assets registered by the accepted event. assets.images: optional array Registered image assets. assets.audio: optional array Registered audio assets. assets.speech_text: optional array Registered speech text assets. assets.visual_prompts: optional array Registered visual prompt assets. asset.added

{
  "event_id": "evt_asset_added_001",
  "type": "asset.added",
  "assets": {
    "audio": [
      {
        "asset_id": "promo_sting",
        "url": "https://cdn.example.com/audio/promo-sting.wav",
        "mime_type": "audio/wav"
      }
    ],
    "speech_text": [
      {
        "asset_id": "line_discount",
        "text": "This deal is available for the next ten minutes."
      }
    ],
    "visual_prompts": [
      {
        "asset_id": "gesture_point",
        "text": "point toward the featured product"
      }
    ]
  }
}

`avatar.added`

Confirms that an avatar.add event was accepted and that the new avatars are available to select with session.update. event_id: string Unique id for this server event. type: “avatar.added” Event type. Must be avatar.added. avatars: array Avatars registered by the accepted event. avatars[].avatar_id: string Registered avatar id. avatars[].instructions, voice, visual: optional fields Effective avatar definition accepted by the service. avatar.added

{
  "event_id": "evt_avatar_added_001",
  "type": "avatar.added",
  "avatars": [
    {
      "avatar_id": "host_c",
      "instructions": "You are Host C, a practical product specialist.",
      "visual": {
        "source_images": [
          {
            "source_image_id": "front",
            "url": "https://api.vivix.ai/host-c-front.png",
            "description": "A man facing the camera in a brightly lit studio, front view.",
            "media_type": "image/png"
          }
        ],
        "default_source_image_id": "front"
      }
    }
  ]
}

5. Conversation Events

`conversation.item.created`

Reports a new conversation item created by the client or by response generation. event_id: string Unique id for this server event. type: “conversation.item.created” Event type. Must be conversation.item.created. previous_item_id: nullable string Conversation item that precedes this item. Returns null when the item has no predecessor. item: object The created Conversation Item. Uses the same item shape documented in conversation.item.create. id: string Server-assigned or client-provided conversation item identifier. type: string enum Item type, such as message, function_call, or function_call_output. role, content, call_id, name, arguments, output: conditional fields Fields depend on the item type. See the item schema in conversation.item.create. status: optional string enum Optional item status, when returned by the service. conversation.item.created

{
  "event_id": "evt_item_created_001",
  "type": "conversation.item.created",
  "previous_item_id": null,
  "item": {
    "id": "item_001",
    "type": "message",
    "status": "completed",
    "role": "user",
    "content": [
      {
        "type": "input_text",
        "text": "Show me the second product."
      }
    ]
  }
}

`conversation.item.retrieved`

Returns the item requested with conversation.item.retrieve. event_id: string Unique id for this server event. type: “conversation.item.retrieved” Event type. Must be conversation.item.retrieved. item: object The requested Conversation Item. Uses the same item shape documented in conversation.item.create. id: string Server-assigned or client-provided conversation item identifier. type: string enum Item type, such as message, function_call, or function_call_output. role, content, call_id, name, arguments, output: conditional fields Fields depend on the item type. See the item schema in conversation.item.create. status: optional string enum Optional item status, when returned by the service. conversation.item.retrieved

{
  "event_id": "evt_item_retrieved_001",
  "type": "conversation.item.retrieved",
  "item": {
    "id": "item_001",
    "type": "message",
    "status": "completed",
    "role": "user",
    "content": [
      {
        "type": "input_text",
        "text": "Show me the second product."
      }
    ]
  }
}

`conversation.item.deleted`

Confirms that a conversation item was deleted. event_id: string Unique id for this server event. type: “conversation.item.deleted” Event type. Must be conversation.item.deleted. item_id: string Identifier of the deleted conversation item. conversation.item.deleted

{
  "event_id": "evt_item_deleted_001",
  "type": "conversation.item.deleted",
  "item_id": "item_001"
}

6. Audio Events

`input_audio.speech_started`

Emitted when the service detects user speech on the input audio stream. This can arrive before any transcription text is available. event_id: string Unique id for this server event. type: “input_audio.speech_started” Event type. Must be input_audio.speech_started. item_id: string Provisional user audio item id used to correlate later transcription events and the created conversation item. audio_start_ms: integer Speech start time in milliseconds on the session input-audio timeline. input_audio.speech_started

{
  "event_id": "evt_input_audio_speech_started_001",
  "type": "input_audio.speech_started",
  "item_id": "item_audio_001",
  "audio_start_ms": 1200
}

`input_audio.speech_stopped`

Emitted when the service detects that user speech has stopped. This can arrive before transcription completes or fails. event_id: string Unique id for this server event. type: “input_audio.speech_stopped” Event type. Must be input_audio.speech_stopped. item_id: string Provisional user audio item id used to correlate later transcription events and the created conversation item. audio_end_ms: integer Speech stop time in milliseconds on the session input-audio timeline. input_audio.speech_stopped

{
  "event_id": "evt_input_audio_speech_stopped_001",
  "type": "input_audio.speech_stopped",
  "item_id": "item_audio_001",
  "audio_end_ms": 2600
}

7. Transcription Events

`conversation.item.input_audio_transcription.delta`

Streams partial transcription text for a user audio item when transcription is enabled. event_id: string Unique id for this server event. type: “conversation.item.input_audio_transcription.delta” Event type. Must be conversation.item.input_audio_transcription.delta. item_id: string User audio conversation item being transcribed. content_index: optional integer Index of the audio content part. delta: optional string Incremental transcript text. conversation.item.input_audio_transcription.delta

{
  "event_id": "evt_transcript_delta_001",
  "type": "conversation.item.input_audio_transcription.delta",
  "item_id": "item_audio_001",
  "content_index": 0,
  "delta": "Show me"
}

`conversation.item.input_audio_transcription.completed`

Returns the final transcription for a user audio item. event_id: string Unique id for this server event. type: “conversation.item.input_audio_transcription.completed” Event type. Must be conversation.item.input_audio_transcription.completed. item_id: string User audio conversation item that was transcribed. content_index: integer Index of the audio content part. transcript: string Final transcript text. conversation.item.input_audio_transcription.completed

{
  "event_id": "evt_transcript_completed_001",
  "type": "conversation.item.input_audio_transcription.completed",
  "item_id": "item_audio_001",
  "content_index": 0,
  "transcript": "Show me the second product."
}

`conversation.item.input_audio_transcription.failed`

Reports that transcription failed for a specific user audio item. event_id: string Unique id for this server event. type: “conversation.item.input_audio_transcription.failed” Event type. Must be conversation.item.input_audio_transcription.failed. item_id: string User audio conversation item that failed transcription. content_index: integer Index of the audio content part that failed transcription. error: object Transcription error details. error.type: string Type of transcription error. error.code: optional string Machine-readable transcription error code, when available. error.message: string Human-readable transcription error message. error.param: optional string Request parameter related to the error, when applicable. conversation.item.input_audio_transcription.failed

{
  "event_id": "evt_transcript_failed_001",
  "type": "conversation.item.input_audio_transcription.failed",
  "item_id": "item_audio_001",
  "content_index": 0,
  "error": {
    "type": "transcription_error",
    "code": "audio_unintelligible",
    "message": "The audio could not be transcribed."
  }
}

8. Response Events

Lifecycle distinction. response.created and response.done describe the response resource and logical output stream. response.render.started and response.render.stopped describe visible or audible avatar rendering over RTC. response.render.stopped is emitted before response.done for responses that enter avatar rendering. Responses that do not render, such as tool-call-only responses, proceed directly to response.done.

`response.created`

request_event_id · optional string. Correlates the client’s response.create when it supplied a nonempty event_id. Returned when a new Response is created. This is the first event of response creation, where the response is in an initial state of in_progress. event_id: string The unique ID of the server event. type: “response.created” Event type. Must be response.created. response: realtime_response The response resource. response.id: optional string The unique ID of the response. response.status: optional string The response status. One of in_progress, completed, cancelled, failed, or incomplete. response.status_details: optional object or null Additional details about the response status. response.output: optional array<conversation_item> The output conversation_items generated by the response. response.max_output_tokens: optional number or “inf” The maximum number of output tokens for the response. response.created

{
  "type": "response.created",
  "event_id": "event_response_created_001",
  "response": {
    "id": "resp_created_001",
    "status": "in_progress",
    "status_details": null,
    "output": [],
    "max_output_tokens": "inf"
  },
  "request_event_id": "evt_response_1"
}

`response.done`

Returned when a Response reaches its final state. Always emitted regardless of the final state and, when avatar rendering occurred, only after response.render.stopped. The response includes all generated output items and omits raw audio data. event_id: string The unique ID of the server event. type: “response.done” Event type. Must be response.done. response: realtime_response The response resource. response.id: optional string The unique ID of the response. response.status: optional string The final response status. Clients should check this field for completed, cancelled, failed, or incomplete. response.status_details: optional object or null Additional details about the response status. response.output: optional array<conversation_item> All output conversation_items generated during the response. Raw audio/video is delivered over RTC media tracks and is not included here. response.max_output_tokens: optional number or “inf” The maximum number of output tokens for the response. response.done

{
  "type": "response.done",
  "event_id": "event_response_done_001",
  "response": {
    "id": "resp_done_001",
    "status": "completed",
    "status_details": null,
    "output": [
      {
        "id": "item_message_001",
        "type": "message",
        "status": "completed",
        "role": "assistant",
        "content": [
          {
            "type": "text",
            "text": "Loud and clear! I can hear you perfectly. How can I help you today?"
          }
        ]
      }
    ],
    "max_output_tokens": "inf"
  }
}

`response.render.started`

Returned when avatar output attributable to a response starts rendering over RTC. This can be model-generated avatar output, scripted speech or audio, or a visual-only scripted action. event_id: string The unique ID of the server event. type: “response.render.started” Event type. Must be response.render.started. response_id: string The response whose avatar output started rendering. response.render.started

{
  "event_id": "evt_response_render_started_001",
  "type": "response.render.started",
  "response_id": "resp_001"
}

`response.render.stopped`

Returned when avatar output attributable to a response is no longer rendering over RTC. The final response.done event follows with the completed response resource and final status. event_id: string The unique ID of the server event. type: “response.render.stopped” Event type. Must be response.render.stopped. response_id: string The response whose avatar output stopped rendering. status: string enum Render stop reason. One of completed, cancelled, interrupted, or failed. response.render.stopped

{
  "event_id": "evt_response_render_stopped_001",
  "type": "response.render.stopped",
  "response_id": "resp_001",
  "status": "completed"
}

`response.output_item.done`

Returned when an item is done streaming. Also emitted when a response is interrupted, incomplete, or cancelled. event_id: string The unique ID of the server event. type: “response.output_item.done” Event type. Must be response.output_item.done. response_id: string The ID of the response to which the item belongs. output_index: number The index of the output item in the response. item: conversation_item The item that is done. id: string The unique ID of the item. type: string The item type. status: optional “completed” or “incomplete” or “in_progress” The status of the item. role: optional string Message role, when the output item is a message. content: optional array Final message content, when the output item is a message. call_id: optional string The ID of the function call, when the output item is a function call. name: optional string The function name, when the output item is a function call. arguments: optional string The final function arguments as a JSON string, when the output item is a function call. response.output_item.done

{
  "event_id": "evt_output_item_done_001",
  "type": "response.output_item.done",
  "response_id": "resp_001",
  "output_index": 0,
  "item": {
    "id": "msg_007",
    "type": "message",
    "status": "completed",
    "role": "assistant",
    "content": [
      {
        "type": "text",
        "text": "This model is best for live demos."
      }
    ]
  }
}

response.output_item.done function call

{
  "event_id": "evt_output_item_done_002",
  "type": "response.output_item.done",
  "response_id": "resp_001",
  "output_index": 1,
  "item": {
    "id": "fc_001",
    "type": "function_call",
    "status": "completed",
    "call_id": "call_001",
    "name": "get_product",
    "arguments": "{\"product_id\":\"sku_123\"}"
  }
}

`response.output_text.delta`

Streams incremental assistant text output, including text associated with spoken RTC media. The first delta for an item_id and output_index implicitly introduces that streamed text item. event_id: string Unique id for this server event. type: “response.output_text.delta” Event type. Must be response.output_text.delta. response_id: string The ID of the response. item_id: string The ID of the message item receiving text. If the client has not seen this item before, create it from this event and finalize it with response.output_item.done. output_index: number The index of the output item in the response. delta: string Incremental text. response.output_text.delta

{
  "event_id": "evt_text_delta_001",
  "type": "response.output_text.delta",
  "response_id": "resp_001",
  "item_id": "item_003",
  "output_index": 0,
  "delta": "This model"
}

`response.output_text.done`

Returns final assistant text output, including text associated with spoken RTC media. event_id: string Unique id for this server event. type: “response.output_text.done” Event type. Must be response.output_text.done. response_id: string The ID of the response. item_id: string The ID of the message item. output_index: number The index of the output item in the response. text: string Final text. response.output_text.done

{
  "event_id": "evt_text_done_001",
  "type": "response.output_text.done",
  "response_id": "resp_001",
  "item_id": "item_003",
  "output_index": 0,
  "text": "This model is best for live demos."
}

Continue reading

Character instructions · Speech recognition · Text model · Text to speech · Opening acknowledgements

Overview

A1: Streaming Avatar

W1: Streaming World

Blazing Fast Video Generation

1. Which event should I listen to?

2. Error Events

error

3. Session Events

`session.closed`

`session.updated`

`session.update.done`

`session.bootstrap.opening.done`

4. Registry Events

`asset.added`

`avatar.added`

5. Conversation Events

`conversation.item.created`

`conversation.item.retrieved`

`conversation.item.deleted`

6. Audio Events

`input_audio.speech_started`

`input_audio.speech_stopped`

7. Transcription Events

`conversation.item.input_audio_transcription.delta`

`conversation.item.input_audio_transcription.completed`

`conversation.item.input_audio_transcription.failed`

8. Response Events

`response.created`

`response.done`

`response.render.started`

`response.render.stopped`

`response.output_item.done`

`response.output_text.delta`

`response.output_text.done`

Continue reading

​1. Which event should I listen to?

​2. Error Events

​error

​3. Session Events

​session.closed

​session.updated

​session.update.done

​session.bootstrap.opening.done

​4. Registry Events

​asset.added

​avatar.added

​5. Conversation Events

​conversation.item.created

​conversation.item.retrieved

​conversation.item.deleted

​6. Audio Events

​input_audio.speech_started

​input_audio.speech_stopped

​7. Transcription Events

​conversation.item.input_audio_transcription.delta

​conversation.item.input_audio_transcription.completed

​conversation.item.input_audio_transcription.failed

​8. Response Events

​response.created

​response.done

​response.render.started

​response.render.stopped

​response.output_item.done

​response.output_text.delta

​response.output_text.done

​Continue reading

1. Which event should I listen to?

2. Error Events

error

3. Session Events

`session.closed`

`session.updated`

`session.update.done`

`session.bootstrap.opening.done`

4. Registry Events

`asset.added`

`avatar.added`

5. Conversation Events

`conversation.item.created`

`conversation.item.retrieved`

`conversation.item.deleted`

6. Audio Events

`input_audio.speech_started`

`input_audio.speech_stopped`

7. Transcription Events

`conversation.item.input_audio_transcription.delta`

`conversation.item.input_audio_transcription.completed`

`conversation.item.input_audio_transcription.failed`

8. Response Events

`response.created`

`response.done`

`response.render.started`

`response.render.stopped`

`response.output_item.done`

`response.output_text.delta`

`response.output_text.done`

Continue reading