Realtime Text to Speech v2

Realtime TTS v2 is retained for existing clients. New integrations should use the parent v3 page. Its default simple mode needs only one input event; reliable mode adds explicit sequencing, idempotency, and recovery when an application needs them.

Endpoints and protocol

Every connection must request the WebSocket subprotocol realtime.tts.msgpack.v2. Client and server application messages are MessagePack binary frames, not JSON text frames.

Existing /v1/tts/live integrations remain available for compatibility, but v1 is frozen. New integrations should use the parent v3 contract; use this page only when maintaining a v2 client.

Site capabilities

The public voice_id must support the selected model. Read GET /v2/tts/capabilities before connecting for the current site's provider, model, format, parameter, and PCM metadata support. The server explicitly rejects combinations outside that site capability matrix.

Authentication

Backend clients should authenticate during the WebSocket handshake:

Authorization: Bearer API_KEY
Sec-WebSocket-Protocol: realtime.tts.msgpack.v2

The browser WebSocket API cannot set an Authorization header. A trusted backend can exchange its API key for an origin-bound, one-time ticket:

POST /v2/tts/browser-tickets
Authorization: Bearer API_KEY
Content-Type: application/json

{"origin":"https://your-app.example"}

The browser then connects with the v2 subprotocol and sends the returned ticket as its first MessagePack frame:

{ event: 'auth', token: 'rtv2_ticket_...' }

Send the frame before the authentication timeout. Tickets are single-use, expire quickly, and are bound to the site, host, and origin. Never place a credential in the URL: token and api_key query parameters are rejected. Production browser applications must not receive long-lived API keys.

Quick start: simple mode

connect → authenticated → start → ready → input → audio (one or more) → finish

Omit start.mode to use simple mode. The server generates the request, event, sequence, and segment IDs. After ready, send one { event: 'input', text } event. The default commit: true commits that text and automatically finishes the request. Clients only need to handle ready, audio, error, and finish; acknowledgement and usage events may be ignored.

{ event: 'start', request: { voice_id: 'PUBLIC_VOICE_UUID', model: 'fishaudio-s21pro-flash', format: 'mp3' } }
{ event: 'input', text: 'Hello from Realtime TTS v2.' }

One WebSocket connection carries one generation request. Send multiple input events with commit: false to build a larger input, then send the final event with commit: true.

The client event names are auth, start, input, text, flush, stop, and ping. Simple mode normally uses only start and input; text, flush, and stop belong to reliable mode.

Client event parameters

All client events are MessagePack maps. Unknown fields are rejected by the default legacy profile. A product_v1 request tolerantly ignores unknown request controls. Field names use snake_case, and every text value must be a valid UTF-8 string.

event must equal the event name shown in the first column. In reliable mode, event_id must be unique on start and every subsequent client event. IDs generated by the server in simple mode are returned in server events; the client does not need to echo them.

Reliable mode

Use { event: 'start', mode: 'reliable', event_id, request_id, request } for explicit multi-segment control and recovery. Starting with start, every reliable-mode client event needs a unique event_id; every text needs a strictly increasing sequence, and every flush needs a unique segment_id.

start → ready → text → input_ack → flush → segment_accepted
      → audio → usage → segment_completed → stop → finish

request_id must contain 1–128 characters and is an account-, site-, and host-scoped idempotency key. Replaying the same request returns request_status without calling the provider or charging again. Reusing the ID with different generation parameters returns idempotency_conflict.

start.request parameters

The default legacy profile keeps the existing strict behavior. With parameter_profile: "product_v1", supported controls are clamped to the model range, unsupported controls are ignored, and ready.effective_request reports the values actually used.

Example:

{
  event: 'start',
  request: {
    voice_id: '00000000-0000-0000-0000-000000000000',
    model: 'fishaudio-s21pro-flash',
    parameter_profile: 'product_v1',
    format: 'mp3',
    speed: 1,
    volume: 0,
    stability: 1,
    similarity: 1,
    text_normalization: true,
    language: 'en',
    chunk_length: 200,
    latency: 'normal'
  }
}

Server events

Every server event contains a unique event_id, a request-chain trace_id, and a Unix-millisecond timestamp. Direct responses also contain in_reply_to.

audio.audio is a byte array inside the MessagePack frame. Do not decode it as Base64. Concatenate audio frames in sequence order for the selected output format. When format is pcm, both ready and audio include the required sample_rate, channels, and bit_depth fields so the client can construct a WAV header or initialize a PCM player.

Error codes

Every error event has a stable code, a safe message, retryable, and terminal. A non-terminal error can be corrected on the current connection. A terminal error ends the session. stage and path identify the failing phase and field when available. The current codes are:

Capabilities and request status

Before opening a WebSocket, clients can read GET /v2/tts/capabilities. It returns the site-specific providers, models, formats, parameters, limits, and the simple and reliable modes so SDKs do not need to hard-code the capability matrix.

Reliable-mode clients can query GET /v2/tts/requests/{request_id} after a network interruption. The state is running, completed, failed_retryable, or failed_terminal. Both status lookup and WebSocket replay are scoped to the account, site, and host. When result_available is true, the response includes a segments manifest.

To rerun a failed_retryable request, reconnect with the same request_id and set retry_failed: true on start. Every retried segment must reuse the same segment_id, sequence, and exact text. Any mismatch returns idempotency_conflict.

Minimal Node.js client

Install ws and @msgpack/msgpack:

import { writeFileSync } from 'node:fs';
import { decode, encode } from '@msgpack/msgpack';
import WebSocket from 'ws';

const audio = [];
const ws = new WebSocket('wss://realtime.fishaudio.org/v2/tts/live', 'realtime.tts.msgpack.v2', {
  headers: { Authorization: `Bearer ${process.env.API_KEY}` },
});

ws.on('message', (frame) => {
  const event = decode(frame);
  if (event.event === 'authenticated') {
    ws.send(
      encode({
        event: 'start',
        request: {
          voice_id: process.env.VOICE_ID,
          model: 'fishaudio-s21pro-flash',
          format: 'mp3',
        },
      }),
    );
  } else if (event.event === 'ready') {
    ws.send(encode({ event: 'input', text: 'Hello from Realtime TTS v2.' }));
  } else if (event.event === 'audio') {
    audio.push(event.audio);
  } else if (event.event === 'error') {
    console.error(event.code, event.message, event.retryable, event.terminal, event.trace_id);
  } else if (event.event === 'finish') {
    writeFileSync('realtime-output.mp3', Buffer.concat(audio.map((chunk) => Buffer.from(chunk))));
    console.log({ requestId: event.request_id, units: event.units });
    ws.close(1000);
  }
});

Limits, billing, and retries

In simple mode, input with commit: true creates one billed segment and automatically finishes. In reliable mode, each non-empty flush creates an independently billed segment; wait for input_ack and segment_completed before advancing. Quota is reserved before provider generation and committed when the first valid audio frame is delivered.

The protocol defaults are 10,000 characters per simple input, 2,000 characters per reliable text event, 10,000 characters per segment, and 100,000 billable units per session. Deployments can enforce lower limits.

Retry only when retryable is true or the connection closes with 1012 or 1013. Use exponential backoff with jitter. Generate a new request_id only for a new user-intended generation; do not bypass duplicate protection to replay an uncertain request.

Handshake failures are HTTP responses before 101 Switching Protocols. Session failures are MessagePack error events followed by a WebSocket close.