> ## Documentation Index
> Fetch the complete documentation index at: https://docs.play.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Agent Web SDK

> Integrate AI agents into your web applications

## Overview

The **Agent Web SDK** is a TypeScript SDK that facilitates real-time, bi-directional audio conversations with your PlayAI Agent via [WebSocket API](/api-reference/agents/websocket). It takes care of the following:

* WebSocket connection management
* Microphone capture and voice activity detection (VAD)
* Sending user audio to the Agent
* Receiving Agent audio and playing it back in the browser
* Managing event listeners such as user or agent transcripts
* Muting/unmuting the user's microphone
* Hanging up (ending) the agent conversation
* Error handling

<Tip>
  This SDK is designed for modern web browsers that support the [Web Audio
  API](https://developer.mozilla.org/en-US/docs/Web/API/Web_Audio_API) and
  [WebSockets](https://developer.mozilla.org/en-US/docs/Web/API/WebSocket). If you want to integrate PlayAI Agent into a
  Flutter app, check out our [Flutter SDK](/documentation/agent-sdks/flutter-sdk). We plan to support other platforms in
  the future.
</Tip>

## Installation

<CodeGroup>
  ```bash npm theme={null}
  npm install @play-ai/agent-web-sdk
  ```

  ```bash yarn theme={null}
  yarn add @play-ai/agent-web-sdk
  ```

  ```bash pnpm theme={null}
  pnpm add @play-ai/agent-web-sdk
  ```
</CodeGroup>

## Create agent

To start a conversation with your agent, first create an agent in [PlayAI app](https://play.ai/my-agents).
Once you have an agent, you can find the agent ID in the agent "Deploy · Web" section, which is required to connect to the agent.

## Basic usage

Below is a simple example illustrating how to initiate a conversation with your agent using the `connectAgent` function:

```ts theme={null}
import { connectAgent } from '@play-ai/agent-web-sdk';

async function startConversation() {
  try {
    const agentController = await connectAgent('YOUR_AGENT_ID');
    console.log('Connected to agent. Conversation ID:', agentController.conversationId);
    // Use agentController to control the conversation...
  } catch (error) {
    console.error('Failed to start conversation:', error);
  }
}

startConversation();
```

<Note>
  The function `connectAgent` returns a Promise

  * If any error occurs during the connection process, the Promise is rejected.
  * When the conversation is successfully established, the Promise resolves to `AgentConnectionController` object.
</Note>

## Config

You can customize the agent's configuration by passing an optional `ConnectAgentConfig` object as the second parameter to `connectAgent`.

```ts theme={null}
const agentController = await connectAgent('YOUR_AGENT_ID', {
  debug: true, // Enable debug logging in the console
  customGreeting: 'Hello, and welcome to my custom agent!', // Override the default greeting
  prompt: 'You are an AI that helps with scheduling tasks.', // Append additional instructions to the agent's prompt
  continueConversation: 'PREVIOUS_CONVERSATION_ID', // Continue a previous conversation
});
```

**Config Options**:

* **`debug`:** Enables debug logging for troubleshooting.
* **`customGreeting`:** Overrides the default greeting used by the agent.
* **`prompt`:** Appends additional instructions to the agent's core prompt.
* **`continueConversation`:** An optional conversation ID to continue a previous conversation.
* **`listeners`:** Attach various listener callbacks (see [Event listeners](#event-listeners) section).

## Event listeners

Event listeners enable you to handle specific moments during the conversation:

```ts theme={null}
const agentController = await connectAgent('YOUR_AGENT_ID', {
  listeners: {
    onUserTranscript: (transcript) => console.log(`USER said: "${transcript}".`),
    onAgentTranscript: (transcript) => console.log(`AGENT will say: "${transcript}".`),
    onUserStartedSpeaking: () => console.log(`USER started speaking...`),
    onUserStoppedSpeaking: () => console.log(`USER stopped speaking.`),
    onAgentDecidedToSpeak: () => console.log(`AGENT decided to speak... (not speaking yet, just thinking)`),
    onAgentStartedSpeaking: () => console.log(`AGENT started speaking...`),
    onAgentStoppedSpeaking: () => console.log(`AGENT stopped speaking.`),
    onHangup: (endedBy) => console.log(`Conversation has ended by ${endedBy}`),
    onError: (err) => console.error(err),
  },
});
```

## Mute/unmute

Once you have an active `AgentConnectionController` from `connectAgent`, you can mute or unmute the user's microphone:

```ts theme={null}
const agentController = await connectAgent('YOUR_AGENT_ID');
agentController.mute(); // The agent won't hear any mic data
agentController.unmute(); // The agent hears the mic data again
```

## Hangup

Use `agentController.hangup()` to end the conversation from the user side.

```ts theme={null}
const agentController = await connectAgent('YOUR_AGENT_ID');
setTimeout(() => {
  // End the conversation after 60 seconds
  agentController.hangup();
}, 60000);
```

When the conversation ends (either by user or agent), the `onHangup` callback (if provided) is triggered.

## Error handling

Errors can occur at different stages of the conversation:

* Starting the conversation. For example:
  * Microphone permissions denied
  * WebSocket fails to connect or closes unexpectedly
  * Invalid agent ID
* During the conversation. For example:
  * Agent fails to generate a response
  * Internal Agent errors
  * Network issues

Errors that occur before the conversation starts are caught by the `connectAgent` Promise. You can handle these errors in the `catch` block.
Errors that occur during the conversation are caught by the `onError` listener.

```ts theme={null}
import { connectAgent } from '@play-ai/agent-web-sdk';

async function startConversation() {
  try {
    const agentController = await connectAgent('YOUR_AGENT_ID', {
      listeners: {
        onError: (error) => {
          console.error('Error occurred:', error.description);
          if (error.isFatal) {
            // Possibly reconnection logic or UI error message
          }
        },
      },
    });
  } catch (err) {
    console.error('Failed to start the conversation:', err);
  }
}
```

**Error object**:

```ts theme={null}
interface ErrorDuringConversation {
  description: string; // Human-readable message
  isFatal: boolean; // Whether the error ended the conversation
  serverCode?: number; // If the server gave a specific error code
  wsEvent?: Event; // Low-level WebSocket event
  cause?: Error; // JS error cause
}
```