​

Quickstart - Runnable Demo

If you want to get started quickly, you can clone the play-showcase repository and run the tts-websocket app locally.

# Clone this repository
git clone https://github.com/playht/playai-showcase.git
# Navigate to the tts-websocket demo app
cd tts-websocket
# NPM install
npm install
# Run the server and follow the instructions
npm start

​

Establishing a WebSocket Connection

To establish a WebSocket connection, you will need to send a POST request to the https://api.play.ai/api/v1/tts/auth endpoint with the following headers:

Authorization: Bearer <your_api_key>
X-User-Id: <your_user_id>
Content-Type: application/json

You can obtain your api_key and user_id from your PlayAI account.

The response will contain a JSON object with a webSocketUrl field that you can use to connect to the WebSocket server.

{
  "Play3.0-mini": {
    "httpStreamingUrl": "https://fal.run/playht-fal/playht-tts/stream?fal_jwt_token=<your_session_token>",
    "webSocketUrl": "wss://fal.run/playht-fal/playht-tts/ws?fal_jwt_token=<your_session_token>"
  },
  "PlayDialog": {
    "httpStreamingUrl": "https://fal.run/playht-fal/playht-tts-ldm/stream?fal_jwt_token=<your_session_token>",
    "webSocketUrl": "wss://fal.run/playht-fal/playht-tts-ldm/ws?fal_jwt_token=<your_session_token>"
  },
  "expiresAt": "2024-10-28T05:13:04.650Z"
}

After this point, you can forward the webSocketUrl to your WebSocket client to establish a connection, such as in the following example:

const ws = new WebSocket('wss://fal.run/playht-fal/playht-tts/ws?fal_jwt_token=<your_session_token>');

​

Sending TTS Commands

Once connected to the WebSocket, you can send TTS commands as JSON messages. The structure of these commands is similar to our TTS streaming API, but in snake_case.

Here’s an example:

const ttsCommand = {
  text: 'Hello, world! This is a test of the PlayAI TTS WebSocket API.',
  voice: 's3://voice-cloning-zero-shot/775ae416-49bb-4fb6-bd45-740f205d20a1/jennifersaad/manifest.json',
  output_format: 'mp3',
  temperature: 0.7,
};

ws.send(JSON.stringify(ttsCommand));

Examples of the available options for the TTS command are:

• request_id (optional): A unique identifier for the request, useful for correlating responses (see more details below).
• text (required): The text to be converted to speech.
• voice (required): The voice ID or URL to use for synthesis.
• output_format (optional): The desired audio format (default is “mp3”).
• temperature (optional): Controls the randomness of the generated speech (0.0 to 1.0).
• speed (optional): The speed of the generated speech (0.5 to 2.0).

For the complete list of parameters, refer to the TTS API documentation.

​

Receiving Audio Output

After sending a TTS command, you’ll receive two kinds of messages:

• The audio output as a series of binary messages.
• One final text message with the format {"request_id":<request_id>} to indicate the end of the audio stream.
• In this response message, request_id is the unique identifier you provided in the TTS command, or null if you didn’t provide one.

To handle these messages and play the audio, you can use the following approach:

let audioChunks = [];

ws.onmessage = (event) => {
  if (event.data instanceof Blob) {
    // Received binary audio data
    audioChunks.push(event.data);
  } else {
    // Received a text message (e.g., request_id )
    const message = JSON.parse(event.data);
    if ('request_id' in message) {
      // If you provided a request_id, you can use it to correlate responses
      // End of audio stream, play the audio
      // If you specified a different output_format, you may need to adjust the audio player logic accordingly
      const audioBlob = new Blob(audioChunks, { type: 'audio/mpeg' });
      const audioUrl = URL.createObjectURL(audioBlob);
      const audio = new Audio(audioUrl);
      audio.play();

      // Clear the audio chunks for the next request
      audioChunks = [];
    }
  }
};

This code collects the binary audio chunks as they arrive and combines them into a single audio blob when the End or Request message ({"request_id":<request id>}) is received. It then creates an audio URL and plays the audio using the Web Audio API.

​

Error Handling

It’s important to implement error handling in your WebSocket client. Here’s an example of how to handle errors and connection closures:

ws.onerror = (error) => {
  console.error('WebSocket Error:', error);
};

ws.onclose = (event) => {
  console.log('WebSocket connection closed:', event.code, event.reason);
  // Implement reconnection logic if needed
};

​

Connection Timeout

To ensure optimal usage, WebSocket connections may be closed by intermediary proxies if they remain idle for longer than 10 seconds. To keep the connection alive, you can send new TTS commands, which will generate audio in a similar way to the first request.

​

Best Practices

1. Authentication: Always keep your API key secure. While the WebSocket URL can be shared with client-side code, the API Key and User ID should be kept private.

2. Error Handling: Implement robust error handling and reconnection logic in your WebSocket client.

3. Resource Management: Close the WebSocket connection when it’s no longer needed to free up server resources.

4. Rate Limiting: Be aware of rate limits on the API and implement appropriate throttling in your application.

5. Testing: Thoroughly test your implementation with various inputs and network conditions to ensure reliability.

By following these guidelines and using the provided examples, you can effectively integrate the PlayAI TTS WebSocket API into your application, enabling real-time text-to-speech functionality with low latency and high performance.