Conversation and interruptions

Let the character answer using the user’s message, spoken instructions, and conversation history. First connect the player. Control events use window.player.ws from the player examples. Choose the microphone code for your TRTC or Agora player.

1. Send text and request an answer

{
  "type": "conversation.item.create",
  "event_id": "message_1",
  "item": {
    "id": "user_1",
    "type": "message",
    "role": "user",
    "content": [
      {
        "type": "input_text",
        "text": "Can you help me choose a gift?"
      }
    ]
  }
}

Use a new item.id for each message. Wait for conversation.item.created with item.id equal to user_1, then send:

{
  "type": "response.create"
}

Send these events with window.player.ws.send(JSON.stringify(event)) after the WebSocket connects. Waiting for the item acknowledgement ensures the reply includes the new message.

2. Display the reply

window.player.ws.addEventListener("message", ({ data }) => {
  const event = JSON.parse(data);
  if (event.type === "response.output_text.delta") {
    console.log(event.response_id, event.delta);
  }
});

Accumulate delta by response_id and reconcile it with the full text in response.output_text.done. Complete the UI using the actual response.done status, including failures and cancellations. Finished text does not mean finished audio.

3. Enable the microphone

This is ordinary voice conversation: the character recognizes speech and generates an answer. Do not enable A2A passthrough for this. Choose the code for your player rather than adding both versions. Add these buttons to index.html:

<button id="mic-on">Microphone on</button>
<button id="mic-off">Microphone off</button>

TRTC

Add this to the TRTC player’s main.js:

const micOn = document.querySelector("#mic-on");
const micOff = document.querySelector("#mic-off");
micOn.onclick = async () => {
  try {
    if (!window.player?.trtc) throw new Error("Start the TRTC player first");
    await window.player.trtc.startLocalAudio();
  } catch (e) { document.querySelector("#status").textContent = e.message; }
};
micOff.onclick = async () => {
  try { await window.player?.trtc?.stopLocalAudio(); }
  catch (e) { document.querySelector("#status").textContent = e.message; }
};

Agora

Add this to the Agora player’s main.js, where AgoraRTC is already imported:

// In the Agora player's main.js, which already imports AgoraRTC.
let microphone, microphoneClient;
let micBusy = false;
let desiredMic = false;
let micGeneration = 0;
const micOn = document.querySelector("#mic-on");
const micOff = document.querySelector("#mic-off");
micOn.onclick = async () => {
  if (microphone || micBusy) return;
  const client = window.player?.agora;
  if (!client) return;
  desiredMic = true;
  const generation = ++micGeneration;
  const stillWanted = () => desiredMic && generation === micGeneration
    && window.player?.agora === client;
  micBusy = true;
  let track;
  let published = false;
  try {
    track = await AgoraRTC.createMicrophoneAudioTrack();
    if (!stillWanted()) return;
    await client.publish([track]);
    published = true;
    if (!stillWanted()) return;
    microphone = track;
    microphoneClient = client;
    track = undefined; // Ownership passes to stopMicrophone().
  } catch (e) {
    document.querySelector("#status").textContent = e.message;
  } finally {
    try {
      if (track && published) await client.unpublish([track]);
    } catch (e) {
      document.querySelector("#status").textContent = e.message;
    } finally {
      track?.close();
      micBusy = false;
    }
  }
};
async function stopMicrophone() {
  desiredMic = false;
  ++micGeneration; // Also cancels capture awaiting permission or publication.
  const track = microphone;
  const client = microphoneClient;
  microphone = microphoneClient = undefined;
  if (!track) return;
  try { await client.unpublish([track]); }
  finally { track.close(); }
}
micOff.onclick = () => stopMicrophone().catch(e => {
  document.querySelector("#status").textContent = e.message;
});

Turning the microphone off also cancels pending permission or publication work and closes its track when it resolves. In the Agora player’s existing Stop handler, call await stopMicrophone() before leaving the channel. Rebuild main.js and refresh the page for either version. Use HTTPS or localhost and request microphone permission from a button click. Default turn detection starts replies automatically; do not send another response.create for the same turn. Keep text input available if permission is denied.

4. Interrupt or play next

{
  "type": "response.cancel"
}

This cancels the active response and clears pending work. It returns an error when no response is active. Terminal events can still arrive after cancellation; use their actual status to update the UI.

{
  "type": "response.create",
  "response": {
    "scheduling_policy": "after_current_response",
    "script": {
      "vocal": {
        "type": "speech",
        "text": "Next, let us look at the second option."
      }
    }
  }
}

New requests default to interrupt, which interrupts current output and clears pending work. after_current_response waits for the current response; at most one request may be pending. reject_if_busy returns an error while busy and can help prevent repeated button actions. A pending request keeps the input and configuration captured when submitted; later image or configuration changes do not rewrite it.

Overview

A1: Streaming Avatar

W1: Streaming World

Blazing Fast Video Generation

1. Send text and request an answer

2. Display the reply

3. Enable the microphone

TRTC

Agora

4. Interrupt or play next

​1. Send text and request an answer

​2. Display the reply

​3. Enable the microphone

​TRTC

​Agora

​4. Interrupt or play next

1. Send text and request an answer

2. Display the reply

3. Enable the microphone

TRTC

Agora

4. Interrupt or play next