Realtime TTS

Realtime TTS is available to overseas API users as the first public version of the protocol. It is not a migration from an earlier public WebSocket API.

Endpoints and protocol

Every connection must request the realtime.tts.msgpack.v1 WebSocket subprotocol. Client and server application frames are MessagePack binary frames, not JSON text frames.

Authentication

Backend clients authenticate during the WebSocket handshake:

Authorization: Bearer API_KEY
Sec-WebSocket-Protocol: realtime.tts.msgpack.v1

The browser WebSocket API cannot set an Authorization header. Trusted browser tools may connect with the same subprotocol and send { event: "auth", token: "API_KEY" } as the first MessagePack frame within five seconds. Never put an API key in the URL. Production browser applications should keep long-lived API keys on a backend and issue short-lived access tickets instead.

Session flow

connect → authenticated → start → ready → text → flush
        → audio (one or more) → usage → stop → finish

stop automatically flushes any remaining buffered text. A reconnect creates a new session and does not resume the previous provider stream.

Client events

All fields are case-sensitive. request_id is optional on client events.

model_id is a voice model UUID returned by the voice API. Supported format values are mp3 (default), wav, pcm, and opus. speed is 0.5–2, volume is 0–10, chunk_length is 100–300, and latency is normal or balanced.

The current rollout accepts Fish Audio voices and the configured fishaudio-s21pro-flash engine. Omit engine_model_id unless support has instructed you to set it.

Server events

Every server event includes a monotonically increasing sequence and a request_id.

Do not decode audio.data as Base64: it is a byte array inside the MessagePack frame.

Minimal Node.js client

import { decode, encode } from '@msgpack/msgpack';
import WebSocket from 'ws';

const ws = new WebSocket('wss://realtime.fishaudio.org/v1/tts/live', 'realtime.tts.msgpack.v1', {
  headers: { Authorization: `Bearer ${process.env.API_KEY}` },
});

ws.on('open', () => {
  ws.send(
    encode({
      event: 'start',
      request: { model_id: process.env.VOICE_MODEL_ID, format: 'mp3' },
    }),
  );
  ws.send(encode({ event: 'text', text: 'Hello from realtime TTS.' }));
  ws.send(encode({ event: 'flush' }));
  ws.send(encode({ event: 'stop' }));
});

ws.on('message', (frame) => {
  const event = decode(frame);
  if (event.event === 'audio') {
    // Append event.data bytes to the output stream or audio decoder.
  }
});

Wait for authenticated and ready in latency-sensitive production clients before sending the next phase. The shortened example relies on ordered WebSocket delivery.

Limits, billing, and retries

Each flush creates one independently billed segment and reserves quota for its buffered text. A segment that fails before its first audio delivery is refunded. After audio delivery begins, that segment is charged and appears in usage history.

The default limits are 2,000 characters per text event, 5,000 characters per flushed segment, 50,000 billable units per session, and three concurrent connections per API key. Limits may change as the service evolves; handle explicit limit errors rather than hard-coding them into product logic.

Retry only errors with retryable: true or WebSocket close codes 1012 and 1013. Use exponential backoff with jitter and do not automatically replay a segment after any audio was received, because doing so can duplicate audio and usage.

Handshake failures such as invalid authorization are HTTP errors returned before status 101 Switching Protocols. Session errors are MessagePack error events followed by a WebSocket close.