Skip to main content

​
Overview

The Agent Web SDK is a TypeScript SDK that facilitates real-time, bi-directional audio conversations with your PlayAI Agent via WebSocket API. It takes care of the following:
  • WebSocket connection management
  • Microphone capture and voice activity detection (VAD)
  • Sending user audio to the Agent
  • Receiving Agent audio and playing it back in the browser
  • Managing event listeners such as user or agent transcripts
  • Muting/unmuting the user’s microphone
  • Hanging up (ending) the agent conversation
  • Error handling
This SDK is designed for modern web browsers that support the Web Audio API and WebSockets. If you want to integrate PlayAI Agent into a Flutter app, check out our Flutter SDK. We plan to support other platforms in the future.

​
Installation

npm install @play-ai/agent-web-sdk

​
Create agent

To start a conversation with your agent, first create an agent in PlayAI app. Once you have an agent, you can find the agent ID in the agent “Deploy · Web” section, which is required to connect to the agent.

​
Basic usage

Below is a simple example illustrating how to initiate a conversation with your agent using the connectAgent function: 
import { connectAgent } from '@play-ai/agent-web-sdk';

async function startConversation() {
  try {
    const agentController = await connectAgent('YOUR_AGENT_ID');
    console.log('Connected to agent. Conversation ID:', agentController.conversationId);
    // Use agentController to control the conversation...
  } catch (error) {
    console.error('Failed to start conversation:', error);
  }
}

startConversation();
The function connectAgent returns a Promise
  • If any error occurs during the connection process, the Promise is rejected.
  • When the conversation is successfully established, the Promise resolves to AgentConnectionController object.

​
Config

You can customize the agent’s configuration by passing an optional ConnectAgentConfig object as the second parameter to connectAgent. 
const agentController = await connectAgent('YOUR_AGENT_ID', {
  debug: true, // Enable debug logging in the console
  customGreeting: 'Hello, and welcome to my custom agent!', // Override the default greeting
  prompt: 'You are an AI that helps with scheduling tasks.', // Append additional instructions to the agent's prompt
  continueConversation: 'PREVIOUS_CONVERSATION_ID', // Continue a previous conversation
});
Config Options:
  • debug: Enables debug logging for troubleshooting.
  • customGreeting: Overrides the default greeting used by the agent.
  • prompt: Appends additional instructions to the agent’s core prompt.
  • continueConversation: An optional conversation ID to continue a previous conversation.
  • listeners: Attach various listener callbacks (see Event listeners section).

​
Event listeners

Event listeners enable you to handle specific moments during the conversation: 
const agentController = await connectAgent('YOUR_AGENT_ID', {
  listeners: {
    onUserTranscript: (transcript) => console.log(`USER said: "${transcript}".`),
    onAgentTranscript: (transcript) => console.log(`AGENT will say: "${transcript}".`),
    onUserStartedSpeaking: () => console.log(`USER started speaking...`),
    onUserStoppedSpeaking: () => console.log(`USER stopped speaking.`),
    onAgentDecidedToSpeak: () => console.log(`AGENT decided to speak... (not speaking yet, just thinking)`),
    onAgentStartedSpeaking: () => console.log(`AGENT started speaking...`),
    onAgentStoppedSpeaking: () => console.log(`AGENT stopped speaking.`),
    onHangup: (endedBy) => console.log(`Conversation has ended by ${endedBy}`),
    onError: (err) => console.error(err),
  },
});

​
Mute/unmute

Once you have an active AgentConnectionController from connectAgent, you can mute or unmute the user’s microphone: 
const agentController = await connectAgent('YOUR_AGENT_ID');
agentController.mute(); // The agent won't hear any mic data
agentController.unmute(); // The agent hears the mic data again

​
Hangup

Use agentController.hangup() to end the conversation from the user side. 
const agentController = await connectAgent('YOUR_AGENT_ID');
setTimeout(() => {
  // End the conversation after 60 seconds
  agentController.hangup();
}, 60000);
When the conversation ends (either by user or agent), the onHangup callback (if provided) is triggered.

​
Error handling

Errors can occur at different stages of the conversation:
  • Starting the conversation. For example:
    • Microphone permissions denied
    • WebSocket fails to connect or closes unexpectedly
    • Invalid agent ID
  • During the conversation. For example:
    • Agent fails to generate a response
    • Internal Agent errors
    • Network issues
Errors that occur before the conversation starts are caught by the connectAgent Promise. You can handle these errors in the catch block. Errors that occur during the conversation are caught by the onError listener. 
import { connectAgent } from '@play-ai/agent-web-sdk';

async function startConversation() {
  try {
    const agentController = await connectAgent('YOUR_AGENT_ID', {
      listeners: {
        onError: (error) => {
          console.error('Error occurred:', error.description);
          if (error.isFatal) {
            // Possibly reconnection logic or UI error message
          }
        },
      },
    });
  } catch (err) {
    console.error('Failed to start the conversation:', err);
  }
}
Error object: 
interface ErrorDuringConversation {
  description: string; // Human-readable message
  isFatal: boolean; // Whether the error ended the conversation
  serverCode?: number; // If the server gave a specific error code
  wsEvent?: Event; // Low-level WebSocket event
  cause?: Error; // JS error cause
}