# Get Agent Stats GET /api/v1/agent-stats/{agentId} Returns all available statistics for a specific agent. # Get Agent Conversations GET /api/v1/agents/{agentId}/conversations Get all conversations for an agent. Retrieve all information about an agent's conversations. ### Response Headers for Pagination | Header Name | Type | Description | | -------------------- | ------- | --------------------------------------------- | | `X-Page-Size` | integer | The number of items per page. | | `X-Start-After` | string | The ID of the last item on the previous page. | | `X-Next-Start-After` | string | The ID of the last item on the current page. | | `X-Total-Count` | integer | The total number of items. | These headers are included in the response to help manage pagination when retrieving conversations for a specific agent. # Get Agent Conversation GET /api/v1/agents/{agentId}/conversations/{conversationId} Get a conversation for an agent. Retrieve all information about an agent conversation. # Get Agent Conversation Transcript GET /api/v1/agents/{agentId}/conversations/{conversationId}/transcript Get a conversation transcript for an agent. Retrieve the transcript of a specific agent conversation. ### Response Headers for Pagination | Header Name | Type | Description | | -------------------- | ------- | --------------------------------------------- | | `X-Page-Size` | integer | The number of items per page. | | `X-Start-After` | string | The ID of the last item on the previous page. | | `X-Next-Start-After` | string | The ID of the last item on the current page. | | `X-Total-Count` | integer | The total number of items. | These headers are included in the response to help manage pagination when retrieving conversation transcript for a specific agent conversation. # Get Agent GET /api/v1/agents/{agentId} Retrieve all information about an agent. Retrieve all information about an agent. # Update Agent PATCH /api/v1/agents/{agentId} Updates a Play.ai Agent. Updates the properties of the agent with the specified ID. # Create Agent POST /api/v1/agents Creates a new Play.ai Agent. Use this endpoint to create new agents. Required parameters include the agent's name and the agent's prompt. After you create your agent, you can proceed to start a conversation using our [Websocket API](/api-reference/websocket), or you can try it out through our web interface at `https://play.ai/agent/`. To update the agents see the [Update Agent](/api-reference/endpoints/v1/agents/patch) endpoint. # Delete External Function Delete /api/v1/external-functions/{functionId} Delete an external function. Deletes the external function with the specified ID. # Get All External Functions GET /api/v1/external-functions Get all external functions. Retrieve all information about all external functions that you have created. # Get External Function GET /api/v1/external-functions/{functionId} Get an external function. Retrieve all information about the external function with the specified ID. # Update External Function PATCH /api/v1/external-functions/{functionId} Update an external function. Updates the properties of the external function with the specified ID. # Create External Function POST /api/v1/external-functions Create a new external function. Use this endpoint to create new external functions. Required parameters include the external function's name and the external function's description. After you create your agent, you can attach the external function to an agent. To update the external functions see the [Update External Function](/api-reference/endpoints/v1/external-functions/patch) endpoint. # Introduction HTTP API endpoints Play.ai provides a simple and easy to use HTTP API to create and manage AI Agents. After you create your agent, you can proceed to start a conversation using our [Websocket API](/api-reference/websocket), or you can try it out through our web interface at `https://play.ai/agent/`. ## Authentication All API endpoints are authenticated using a User ID and API Key. After you have created an account and logged in, you can get your API Key from the [For Developers](https://play.ai/developers) page. # Web Embed API API for embedding a play.ai agent on your website This document provides detailed information about the key components of the play.ai web embed API: the events array, the onEvent handler, and the openEmbed function. ## Installation ```bash npm install @play-ai/web-embed ``` ## `openEmbed` Function The `openEmbed` function initializes and opens the play.ai web embed on the webpage. It is imported from the `@play-ai/web-embed` package as follows ```typescript import { open as openEmbed } from '@play-ai/web-embed'; ``` It has the following signature: ```typescript function openEmbed( webEmbedId: string, options: { events?: ReadonlyArray; onEvent?: OnEventHandler; prompt?: string; customGreeting?: string; }, ): { setMinimized: (minimize?: boolean) => void }; ``` ### Parameters * `webEmbedId`: A string representing your unique web embed ID provided by play.ai. * `options`: An object containing: * `events?`: The array of custom events your application can handle. * `onEvent?`: The event handler function. * `customGreeting?`: A custom greeting that replaces the default greeting. * `prompt?`: A prompt to give the agent in addition to the default prompt. Use this to give context that is page-specific or user-specific, e.g. "The form fields on the current page are X, Y, and Z". ### Return type * `setMinimized(minimize?: boolean)`: A function that allows you to toggle the minimized state of the web embed. Pass in `true` to minimize and `false` to maximize the web embed. Toggle the minimize state by passing in `undefined`. ### Example ```javascript import { open as openEmbed } from '@play-ai/web-embed'; // ... (events and onEvent definitions) ... useEffect(() => { const { setMinimized } = openEmbed(webEmbedId, { events, onEvent, customGreeting: "Let's fill out this form together!", prompt: 'The form fields on the current page are name, email, and shipping address', }); }, []); ``` In this example, the openEmbed function is called inside a useEffect hook to initialize the web embed when the component mounts. ## Events Array The events array defines the custom events that your application can handle. Each event in the array is an object with the following structure: ```typescript type Event = { name: string; when: string; data: { [key: string]: { type: string; description?: string; values?: string[]; }; }; }; ``` ### Properties * `name`: A string that uniquely identifies the event. * `when`: A string describing the condition that triggers this event. * `data`: An object describing the data that should be passed to the event handler. Each key in this object represents the name of the data field, and its value is an object with: * `type`: The data type of the field (currently supports `string`, `number`, `boolean`, and `enum`). * `description?`: A brief description of what this data field represents. * `values?`: An array of strings representing the possible values for the field if the type is `enum`. ### Example ```javascript const events = [ { name: "change-text", when: "The user says what they want to change the text on the screen to", data: { text: { type: "string", description: "The text to change to" }, }, }, ] as const; ``` ## onEvent Handler The onEvent handler is a function that processes events triggered by the play.ai web embed. It has the following signature: ```typescript type OnEventHandler = (event: { name: string; data: Record }) => void; ``` ### Parameters * `event`: An object containing: * `name`: The name of the triggered event (matching an event name from the events array). * `data`: An object containing the data associated with the event (matching the data structure from the events array). ### Example ```javascript const onEvent = (event) => { console.log('onEvent: ', event); if (event.name === 'change-text') { setText(event.data.text); } }; ``` In this example, the handler logs all events and updates the text state when a "change-text" event is received. ## Putting It All Together Here's how these components work together: 1. You define your custom events in the events array. 2. You implement the onEvent handler to process these events. 3. You call the openEmbed function, passing your web embed ID, the events array, the onEvent handler, and optionally customGreeting and prompt. 4. When a user interacts with the AI agent, it may trigger one of your defined events. 5. The onEvent handler receives the triggered event and processes it according to your implementation. This structure allows for flexible, event-driven interactions between your web application and the play.ai web embed. [Learn more about how to use the web embed API in this guide](/documentation/web-embed/embedding-an-agent-on-your-website). # Websocket API Enhance your app with our audio-in, audio-out API, enabling seamless, natural conversations with your PlayAI agent. Transform your user experience with the power of voice. To use our WebSocket, you will need beforehand: * A [Play.ai account](https://play.ai/pricing) * An [API key to authenticate](https://play.ai/developers) with the Play.ai API * An agent ID of a Play.ai Agent (created via our [Web UI](https://play.ai/my-agents) or our [Create Agent endpoint](/api-reference/endpoints/v1/agents/post)) To fully leverage our WebSocket API, the steps are: * Connect to our `wss://api.play.ai/v1/talk/` URL * Send a `{"type":"setup","apiKey":"yourKey"}` message as first message * Send audio input as base64 encoded string in `{"type":"audioIn","data":"base64Data"}` messages * Receive audio output in `{"type":"audioStream","data":"base64Data"}` messages # Establishing a Connection To initiate a conversation, establish a websocket connection to our `talk` URL, including the `agentId` as a path parameter: ```text wss://api.play.ai/v1/talk/ ``` For example, assuming `Agent-XP5tVPa8GDWym6j` is the ID of an agent you have created via our [Web UI](https://play.ai/my-agents) or through our [Create Agent endpoint](/api-reference/endpoints/v1/agents/post), the WebSocket URL should look like: ```js const myWs = new WebSocket('wss://api.play.ai/v1/talk/Agent-XP5tVPa8GDWym6j'); ``` # Initial Setup Message Before you can start sending and receiving audio data, you must first send a `setup` message to authenticate and configure your session. ```mermaid graph TB subgraph "conversation" C --> D[Send 'audioIn' messages containing your user's audio data] D --> C end B --> C[Receive 'audioStream' messages containing Agent's audio data] subgraph setup A[Establish WebSocket Connection] --> B[Send 'setup' message] end ``` The only required field in the setup message is the `apiKey`. This assumes you are comfortable with the default values for audio input and audio output formats. In this scenario, your first setup message could be as simple as: ```json { "type": "setup", "apiKey": "yourKey" } ``` Get your API Key at our [Developers](https://play.ai/developers) page Code example: ```js const myWs = new WebSocket('wss://api.play.ai/v1/talk/Agent-XP5tVPa8GDWym6j'); myWs.onopen = () => { console.log('connected!'); myWs.send(JSON.stringify({ type: 'setup', apiKey: 'yourApiKey' })); }; ``` ## Setup Options The setup message configures important details of the session, including the format/encoding of the audio that you intend to send us and the format that you expect to receive. ```json Example setup messages with various options: // mulaw 16KHz as input { "type": "setup", "apiKey": "...", "inputEncoding": "mulaw", "inputSampleRate": 16000 } // 24Khz mp3 output { "type": "setup", "apiKey": "...", "outputFormat": "mp3", "outputSampleRate": 24000 } // mulaw 8KHz in and out { "type": "setup", "apiKey": "...", "inputEncoding": "mulaw", "inputSampleRate": 8000, "outputFormat": "mulaw", "outputSampleRate": 8000 } ``` The following fields are available for configuration:
Property Accepted values Description Default value
`type`
(required)		 `"setup"` Specifies that the message is a setup command. -
`apiKey`
(required)		 `string` [Your API Key](https://play.ai/developers). -
`outputFormat`
(optional)		 * `"mp3"` * `"raw"` * `"wav"` * `"ogg"` * `"flac"` * `"mulaw"` The format of audio you want our agent to output in the `audioStream` messages. * `mp3` = 128kbps MP3 * `raw` = PCM\_FP32 * `wav` = 16-bit (uint16) PCM * `ogg` = 80kbps OGG Vorbis * `flac` = 16-bit (int16) FLAC * `mulaw` = 8-bit (uint8) PCM headerless `"mp3"`
`outputSampleRate`
(optional)		 `number` The sample rate of the audio you want our agent to output in the `audioStream` messages `44100`
`inputEncoding`
(optional)		 For non-headerless formats: `"media-container"` For headerless formats: * `"mulaw"` * `"linear16"` * `"flac"` * `"amr-nb"` * `"amr-wb"` * `"opus"` * `"speex"` * `"g729"` The encoding of the audio you intend to send in the `audioIn` messages. If your are sending audio formats that use media containers (that is, audio that contain headers, such as `mp4`, `m4a`, `mp3`, `ogg`, `flac`, `wav`, `mkv`, `webm`, `aiff`), just use `"media-container"` as value for `inputEncoding` (or don't pass any value at all since `"media-container"` is the default). This will instruct our servers to process the audio based on the data headers. If, on the other hand, you will send us audio in headerless formats, you have to specify the format you will be sending. In this case, specify it by, e.g., setting `inputEncoding` to `"mulaw"`, `"flac"`, etc. `"media-container"`
`inputSampleRate`
(optional)		 `number` The sample rate of the audio you intend to send. Required if you are specifying an `inputEncoding` different than `"media-container"`. Optional, otherwise -
`customGreeting`
(optional)		 `string` Your agent will say this message to start every conversation. This overrides the agent's greeting. -
`prompt`
(optional)		 `string` Give instructions to your AI about how it should behave and interact with others in conversation. This is appended to the agent's prompt. `""`
`continueConversation`
(optional)		 `string` If you want to continue a conversation from a previous session, pass the `conversationId` here. The agent will continue the conversation from where it left off. -


# `audioIn`: Sending Audio Input After the setup, you can send audio input in the form of an `audioIn` message. The audio must be sent as a base64 encoded string in the `data` field. The message format is: ```json { "type": "audioIn", "data": "" } ``` The audio you send must match the `inputEncoding` and `inputSampleRate` you configured in the setup options. ## Sample Code for Sending Audio Assuming `myWs` is a WebSocket connected to our `/v1/talk` endpoint, the sample code below would send audio directly from the browser: ```javascript const stream = await navigator.mediaDevices.getUserMedia({ audio: { channelCount: 1, echoCancellation: true, autoGainControl: true, noiseSuppression: true, }, }); const mediaRecorder = new MediaRecorder(stream); mediaRecorder.ondataavailable = async (event) => { const base64Data = await blobToBase64(event.data); // Relevant: myWs.send(JSON.stringify({ type: 'audioIn', data: base64Data })); }; async function blobToBase64(blob) { const reader = new FileReader(); reader.readAsDataURL(blob); return new Promise((resolve) => { reader.onloadend = () => resolve(reader.result.split(',')[1]); }); } ``` # `audioStream`: Receiving Audio Output Audio output from the server will be received in an `audioStream` message. The message format is: ```json { "type": "audioStream", "data": "" } ``` The audio you receive will match the `outputFormat` and `outputSampleRate` you configured in the setup options. ## Sample Code for Receiving Audio ```javascript myWs.on('message', (message) => { const event = JSON.parse(message); if (event.type === 'audioStream') { // deserialize event.data from a base64 string to binary // enqueue/play the binary data at your player return; } }); ``` # Voice Activity Detection: `voiceActivityStart` and `voiceActivityEnd` During the conversation, you will receive `voiceActivityStart` and `voiceActivityEnd` messages indicating the detection of speech activity in the audio input. These messages help in understanding when the user starts and stops speaking. When our service detects that the user started to speak, it will emit a `voiceActivityStart` event. Such a message will have the format: ```json { "type": "voiceActivityStart" } ``` It is up to you to decide how to react to this event. We highly recommend you stop playing whatever audio is being played, since the `voiceActivityStart` generally indicates the user wanted to interrupt the agent. Similarly, when our service detects that the user stopped speaking, it emits a `voiceActivityEnd` event: ```json { "type": "voiceActivityEnd" } ``` # `newAudioStream`: Handling New Audio Streams A `newAudioStream` message indicates the start the audio of a new response. It is recommended to clear your player buffer and start playing the new stream content upon receiving this message. This message contains no additional fields. # Error Handling Errors from the server are sent as `error` message type, a numeric code and a message in the following format: ```json { "type": "error", "code": , "message": "" } ``` The table below provides a quick reference to the various error codes and their corresponding messages for the Agent Websocket API. | Error Code | Error Message | | ---------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | 1001 | Invalid authorization token. | | 1002 | Invalid agent id. | | 1003 | Invalid authorization credentials. | | 1005 | Not enough credits. | | 4400 | Invalid parameters. Indicates the message sent to the server failed to match the expected format. Double check the logic and try again. | | 4401 | Unauthorized. Invalid authorization token or invalid authorization credentials for specified agent. | | 4429 | You have reached the maximum number of concurrent connections allowed by your current plan. Please consider upgrading your plan or reducing the number of active connections to continue. | | 4500 | Generic error code for internal errors, such as failures to generate responses.
Generally, the user is not at fault when these happen. An appropriate reaction is to wait a few moments and try again. If the problem persists, contacting support is advised. | *** This documentation covers the essential aspects of interacting with the PlayAI Websocket API for agent conversations. Ensure that your implementation handles the specified message types and follows the outlined protocols for a seamless integration. # Bring Your Own LLM ```bash Updating an agent to use your own LLM curl --request PATCH \ -- url 'https://api.play.ai/api/v1/agents/{{agentId}}' \ --header 'Authorization : Bearer {{yourApiKey}}' \ --header 'X-User-Id: {{yourUserId}}' \ --header 'Content-Type: application/json' \ --data-raw '{ "llm": { "baseURL": "https://my.own.llm.api.example.com", "apiKey": "sk-the-api-key-for-the-llm-api", } }' ``` Use Play.ai agents with responses generated by your own LLM. At [Play.ai](https://play.ai), we provide a built-in LLM+RAG system that you can use to power your agents. However, we understand that you may want to use your own LLM for various reasons, such as: * You have a custom LLM that you have trained on your own data. * You have a license to use a specific LLM that your company has purchased. * You want to use a specific LLM that we do not provide. In these cases, you can have your own LLM generate responses for your agents at Play.ai. We provide an API that allows you to configure your agent with your own LLM, as long as it is [OpenAI API-Compatible](#what-are-the-compatibility-requirements-for-my-llm-api), and a simple API call to update your agent is all you need to start using it. ## How to bring your own LLM To bring your own LLM, you need: * A [Play.ai account](https://play.ai/pricing) * An [API key to authenticate](https://play.ai/developers) with the Play.ai API * An LLM that provides an [OpenAI-Compatible API](#what-are-the-compatibility-requirements-for-my-llm-api) * Calling the API with the `llm` field set in the [Update Agent endpoint](/api-reference/endpoints/v1/agents/patch) To bring your own [OpenAI API-Compatible](#what-are-the-compatibility-requirements-for-my-llm-api) LLM, you will need to provide us with the base URL of your LLM API and an API key to authenticate with it. We will use this information to communicate with your LLM when your agent needs to generate responses. ### Updating an agent to use your own LLM Once you have created an agent, you can update it with your LLM. To do so, call our [Update Agent endpoint](/api-reference/endpoints/v1/agents/patch) with the `llm` field set. For a full step-by-step guide on how to bring your own LLM, see the [tutorial below](#bring-your-own-llm-tutorial). For example, assuming `agentId` is the ID of an agent you have created via our [Web UI](https://play.ai/my-agents) or through our [Create Agent endpoint](/api-reference/endpoints/v1/agents/post), the request below would update the agent: ```http request PATCH https://api.play.ai/api/v1/agents/{{agentId}} Authorization: Bearer {{yourApiKey}} X-User-Id: {{yourUserId}} Content-Type: application/json { "llm": { "baseURL": "https://my.own.llm.api.example.com", "apiKey": "sk-the-api-key-for-the-llm-api" } } ``` ```bash curl curl --request PATCH \ -- url 'https://api.play.ai/api/v1/agents/{{agentId}}' \ --header 'Authorization : Bearer {{yourApiKey}}' \ --header 'X-User-Id: {{yourUserId}}' \ --header 'Content-Type: application/json' \ --data-raw '{ "llm": { "baseURL": "https://my.own.llm.api.example.com", "apiKey": "sk-the-api-key-for-the-llm-api", } }' ``` For a full step-by-step guide on how to bring your own LLM, make sure to check the [tutorial below](#bring-your-own-llm-tutorial).
# Frequently Asked Questions ## Can I set custom headers or other configurations? Yes, you can set custom headers or other configurations through the `baseParams` field. Current configuration options include: custom headers (`defaultHeaders`) , `model` , `temperature` , and more. Check the details on the `llm` field in the [Update Agent](/api-reference/endpoints/v1/agents/patch) or [Create Agent](/api-reference/endpoints/v1/agents/post) API pages. ## What are the compatibility requirements for my LLM API? We expect the LLM URL provided at `llm.baseURL` to be OpenAI API-Compatible. In other words, it must be an API that serves an endpoint at `/chat/completions` that returns an SSE response when prompted. Typically, these APIs can be used directly with OpenAI's or LangChain's SDKs. Below, you can find an example of a request and response to an OpenAI-Compatible API. ```http Example Request POST https://my.own.llm.api.example.com/chat/completions Authorization: Bearer sk-the-api-key-for-the-llm-api Content-Type: application/json { "model": "gpt-3.5-turbo", "messages": [ { "role": "user", "content": "Repeat with me: the smoke test has passed." } ], "stream": true } ``` ```http Example Response HTTP 200 OK content-type: text/event-stream data: {"id":"chatcmpl-fewiojLJ6ZYcGhMs1FCp","object":"chat.completion.chunk","created":1714599207,"model":"gpt-3.5-turbo-0125","system_fingerprint":"fp_123456","choices":[{"index":0,"delta":{"role":"assistant","content":""},"logprobs":null,"finish_reason":null}]} data: {"id":"chatcmpl-fewiojLJ6ZYcGhMs1FCp","object":"chat.completion.chunk","created":1714599207,"model":"gpt-3.5-turbo-0125","system_fingerprint":"fp_123456","choices":[{"index":0,"delta":{"content":"The"},"logprobs":null,"finish_reason":null}]} data: {"id":"chatcmpl-fewiojLJ6ZYcGhMs1FCp","object":"chat.completion.chunk","created":1714599207,"model":"gpt-3.5-turbo-0125","system_fingerprint":"fp_123456","choices":[{"index":0,"delta":{"content":" smoke"},"logprobs":null,"finish_reason":null}]} data: {"id":"chatcmpl-fewiojLJ6ZYcGhMs1FCp","object":"chat.completion.chunk","created":1714599207,"model":"gpt-3.5-turbo-0125","system_fingerprint":"fp_123456","choices":[{"index":0,"delta":{"content":" test"},"logprobs":null,"finish_reason":null}]} data: {"id":"chatcmpl-fewiojLJ6ZYcGhMs1FCp","object":"chat.completion.chunk","created":1714599207,"model":"gpt-3.5-turbo-0125","system_fingerprint":"fp_123456","choices":[{"index":0,"delta":{"content":" has"},"logprobs":null,"finish_reason":null}]} data: {"id":"chatcmpl-fewiojLJ6ZYcGhMs1FCp","object":"chat.completion.chunk","created":1714599207,"model":"gpt-3.5-turbo-0125","system_fingerprint":"fp_123456","choices":[{"index":0,"delta":{"content":" passed"},"logprobs":null,"finish_reason":null}]} data: {"id":"chatcmpl-fewiojLJ6ZYcGhMs1FCp","object":"chat.completion.chunk","created":1714599207,"model":"gpt-3.5-turbo-0125","system_fingerprint":"fp_123456","choices":[{"index":0,"delta":{"content":"."},"logprobs":null,"finish_reason":null}]} data: {"id":"chatcmpl-fewiojLJ6ZYcGhMs1FCp","object":"chat.completion.chunk","created":1714599207,"model":"gpt-3.5-turbo-0125","system_fingerprint":"fp_123456","choices":[{"index":0,"delta":{},"logprobs":null,"finish_reason":"stop"}]} data: [DONE] ``` This is, therefore, the contract your LLM's API must fulfill to be compatible with Play.ai. ## How can I test if my LLM API is compatible? Besides manually verifying if your API returns a response similar to the one mentioned above, when you try to update an agent with your LLM, our backend will test your API for compatibility before returning a successful response. If the provided LLM API fails the compatibility check, our servers will return a `400` error with a message describing the issues found. ## How to use my own RAG and integrations? Currently, you can use the built-in RAG and integrations provided by Play.ai, having your LLM work only as a response generator. Alternatively, it is entirely possible to use your own RAG and integrations under the hood of your LLM API, having your LLM interact with them before generating responses. We are working on supporting custom RAGs and integrations, in a similar way to how we support custom LLMs. Stay tuned for updates on this page.

# Bring Your Own LLM Tutorial To bring your own LLM, you need: * A [Play.ai account](https://play.ai/pricing) * An [API key to authenticate](https://play.ai/developers) with the Play.ai API * An LLM that provides an [OpenAI-Compatible API](#what-are-the-compatibility-requirements-for-my-llm-api) * Calling the API with the `llm` field set in the [Update Agent endpoint](/api-reference/endpoints/v1/agents/patch) To bring your own LLM to Play.ai, follow the steps below: First, you need to have a Play.ai account. If you don't have one, you can sign up [here](https://play.ai/pricing). Next, you need to have an API key to authenticate with the Play.ai API. You can get your User ID and create an API key in your [developers page](https://play.ai/developers). Example user ID: `WNLMXuPV1y123b7Tusx7ZftkpJE3` Example API Key: `ak-4912345671ea4db7943be123f5212953` To set the LLM, you will need the ID of an existing agent, e.g. `myagent-123tVPa7XMBX5Dmym613j`. Your existing agents (created via the API or the Web UI) can be found in your [my agents page](https://play.ai/my-agents). To create a new agent, you can use our [Web UI](https://play.ai/create-agent) or our [Create Agent (POST) endpoint](/api-reference/endpoints/v1/agents/post). Agent creation via the [Web UI](https://play.ai/my-agents) or the [API](/api-reference/endpoints/v1/agents/post) has the same effect. You can use either method to create an agent that will use your LLM. You can also edit an agent created via the Web UI using the API and vice versa. Creating an agent through the [API](/api-reference/endpoints/v1/agents/post) with the `llm` field set would have the same effect described in this step. Finally, you need to have an LLM that provides an [OpenAI-Compatible API](#what-are-the-compatibility-requirements-for-my-llm-api) and call our endpoint to update your agent. If you don't have a compatible LLM right now -- or just want to try our API out --, you can use a Fake LLM we have built for this tutorial. Visit and fork [https://replit.com/@antonio-play/fake-llm](https://replit.com/@antonio-play/fake-llm) With your URL and API key in hand, you can update your agent with your LLM. To do this, call our [Update Agent endpoint](/api-reference/endpoints/v1/agents/patch) with the `llm` field set, as described next. Assuming `agentId` is the ID of an agent -- created either via our [Web UI](https://play.ai/my-agents) or through our [Create Agent endpoint](/api-reference/endpoints/v1/agents/post) --, the request below would update the agent: ```http request PATCH https://api.play.ai/api/v1/agents/{{agentId}} Authorization: Bearer {{yourApiKey}} X-User-Id: {{yourUserId}} Content-Type: application/json { "llm": { "baseURL": "https://my.own.llm.api.example.com", "apiKey": "sk-the-api-key-for-the-llm-api" } } ``` ```bash curl curl --request PATCH \ -- url 'https://api.play.ai/api/v1/agents/{{agentId}}' \ --header 'Authorization : Bearer {{yourApiKey}}' \ --header 'X-User-Id: {{yourUserId}}' \ --header 'Content-Type: application/json' \ --data-raw '{ "llm": { "baseURL": "https://my.own.llm.api.example.com", "apiKey": "sk-the-api-key-for-the-llm-api", } }' ``` If you get a `200` response, your agent is now configured to use your LLM. The new settings will affect all conversations *started after* the update. There are checks in place: When updating the `llm` property of an agent (or creating a new one with `llm` set), our backend will test your API for compatibility before returning a successful response. If the provided LLM API fails the compatibility check, our servers will return a `400` error with a message describing the issues found. To test your agent, you can talk to it using our Web UI at the direct URL `https://play.ai/agent/{{yourAgentId}}` or you can leverage our [Websocket API](/api-reference/websocket). Agents created via the API also appear in the [Agents list page at the Web UI](https://play.ai/my-agents). You can click on the agent there to access the direct URL and have conversations. # Built-in LLMs When creating an agent with Play.ai, you have the flexibility to choose from a variety of language models to power your agent's intelligence. This customization allows you to tailor your agent's capabilities to your specific needs, and have fun experimenting! Let's explore the options and their pros and cons. ## Available Models Play.ai offers a range of models, including: * GPT models (e.g., gpt-4o, gpt-4o-mini, gpt-3.5-turbo, gpt-4, gpt-4-turbo) * Llama models (e.g., llama3-70b-8192, llama3-8b-8192) * Hermes models (e.g., hermes-3-llama-3.1-405b-fp8, hermes-3-llama-3.1-405b-fp8-128k) * Gemma models (e.g., gemma-7b-it, gemma2-9b-it) * Mixtral model (mixtral-8x7b-32768) Each model has its own strengths and trade-offs, which we'll discuss below. ## Pros and Cons of Different Model Types ### GPT Models Pros: * High-quality outputs and strong general knowledge * Excellent at understanding context and nuance * Constantly updated with new information Cons: * May have longer response times for larger models (e.g., GPT-4) * Can be overly formal and assistant like. ### Llama Models Pros: * Various sizes available to balance performance and speed * More conversational, funny, and creative * Some versions optimized for specific tasks (e.g., tool use) Cons: * May not have as broad knowledge as GPT models * Performance can vary depending on the specific version and task * Generally less accurate at action execution than gpt models. And ### Hermes Models Pros: * Optimized for conversational use cases * Based on Llama architecture with additional training on massive conversational datasets Cons: * May be less versatile than more general-purpose models * Limited availability of different sizes and variants * Larger model -> higher latency * No support for actions ## Choosing the Right Model We're still very early in the world of AI and LLMS. There is no "one size fits all" model, and we're still learning what works best for different use cases. That said, it is clear to us that gpt-4o is the best all-around model for agents that use actions, and the llama models seem to be the funniest on average. Happy agent building! # Actions and Integrations Supercharge your Play.ai agents with the capacity to interact with external services. Prerequisites: Basic understanding of REST APIs. If you’re new to APIs, check out [this video](https://www.youtube.com/watch?v=Yzx7ihtCGBs) first, then come back. ## What is an Action? An action is a task that your agent can execute that involves interfacing with the outside world. For example: * Calling the Google Calendar API to book an appointment for a user. * Fetching a funny quote from a public API and reciting it to a user. Fundamentally, actions are flexible and discrete pieces of functionality that you can build into any agent — and unlock limitless utility! ## How do I create an Action? ### Step 1 - Get to the right place Navigate to our [Actions page](https://play.ai/integrations) Where you will find a form that looks like this: ### Step 2 - Describe your action Agents determine when to invoke actions using the description that you assign when creating an action. If your action description is “Get a funny quote”, when you ask your agent during a conversation “hi, can you get me a funny quote”, your agent will happily do so. However, if your action description is “fry me a hot dog”, the agent will have no idea what to do. ### Step 3 - Enter your endpoint and choose your request method This is where the (slightly technical work) comes in. At this stage, you’ll enter the URL for the API you want your agent to retrieve data from or send data to. It can be a public API, an API you built yourself, or anything in between. ### Step 4 - Configure your parameters (optional) This is where the fun begins. Actions can be configured with headers, body parameters, query, parameters, and URL parameters using any combination of static and dynamic parameters. #### What are static parameters? Static parameters are parameters that you want to remain the same every time the action is invoked. This is often useful for parameters such as API keys, which generally remain the same for each call: For static values, be sure to uncheck the “Conversation time parameter” checkbox. #### What are dynamic parameters? Dynamic parameters are parameters that you’d like the agent to determine based off of the conversation. For example, the user’s name, or email address. ### Step 5 - Enable your action To enable your action, either [edit an existing agent](https://play.ai/my-agents), or [create your own](https://play.ai/create-agent). At the bottom of the composition form, you should see your new action! Simply check the box corresponding to the action you’d like to enable - and you’re all set. #### How do I test my action? Once your action is enabled, your agents will intelligently invoke it during conversations. To test your first action, navigate to [your agents](https://play.ai/my-agents), start a conversation, and ask the agent to do whatever it is that your action description entails. If the action succeeds, you’re done! However, if for some reason the action initially fails, you debug by navigating to the [conversations page](https://play.ai/conversations) and viewing the transcript of the relevant conversation. Note that a conversation ID will be included in the body of the request to your endpoint. This allows you to identify requests coming from the same conversation. That’s all for now. Feel free to shoot any questions or provide any feedback you have to [support@play.ht](mailto:support@play.ht) Thanks for reading! # Introduction Welcome to Play.ai documentation Hero Light Hero Dark ### Coding with LLMs? You can access the entire docs as one plain text file here: [https://docs.play.ai/llms-full.txt ](https://docs.play.ai/llms-full.txt). Save the file and add it as context to any LLM for easier development. ### Ready-to-use code snippets You can easily get ready-to-use code snippets from our playground here: [https://play.ai/playground](https://play.ai/playground) by switching from "form" to "code" ![](https://mintlify.s3.us-west-1.amazonaws.com/playhtinc/documentation/get-started/playground-code-snippet) ## Setting up The first step to world-class agents and conversational AI. Our API enables every business, every developer, every tinkerer to easily build capable and useful conversational AI voice Solutions. Get your User ID and API Key Create your first Play.ai agent Start a conversation with your agent Create single or multi speaker speech from text with our latest PlayDialog model. Turn any PDF, document, video, or image file into a multi-speaker podcast. # Build your own AI Voice Agents Learn how to build your own AI-powered Voice Agents using Play.ai # Creating Your First AI Voice Agent This guide will walk you through creating a custom AI voice agent using the Play.AI platform. These agents can be deployed to your website or a phone number, opening up a wide range of possibilities for interactive experiences. ## Prerequisites * A Play.AI account. If you don't have one, you can create one at [play.ai](play.ai). ## Step-by-Step Guide 1. **Agent Creation:** * Navigate to [play.ai](https://play.ai/) and log in. * Click "Create an Agent." This will open the agent creation portal. 2. **Agent Identity:** * **Name:** Give your agent a unique and descriptive name. This name will be visible to users. * **Voice:** Choose a pre-built voice from the available library. Voices are categorized by gender, accent, and style (conversational, storytelling, etc.). You can audition each voice before selection. Alternatively, create a clone of your own voice for a personalized touch (this feature may have specific requirements). * **Speed:** Adjust the speaking speed of the chosen voice. The default is 1.0x, with options to increase or decrease the speed. * **Avatar:** Select a visual representation for your agent. Choose from a library of pre-designed avatars or upload a custom image. * **Privacy:** Configure the agent's accessibility. Options include: * **Private:** Only you can access and interact with the agent. * **Unlisted:** Accessible to anyone with the unique link to the agent. Ideal for sharing with a select group. * **Public:** Anyone can interact with and clone the agent. 3. **Agent Behavior:** * **Agent Greeting:** Craft the initial message your agent will use to greet users. This sets the tone for the conversation. Example: "Hello! I'm Jarvis. How can I help you today?" * **Agent Prompt:** This is the core instruction set that defines your agent's purpose and behavior. Be clear and specific about the agent's role and the types of interactions it should handle. This acts similarly to a system prompt in large language models. Example: "You are a helpful assistant. You will provide information and answer questions about the services our company offers." 4. **Agent Knowledge (Optional):** * **LLM Selection:** Choose the underlying language model that powers your agent's intelligence. Several options may be available, each with varying capabilities and costs. * **Custom Knowledge:** Enhance your agent's knowledge with specific information relevant to its purpose. Upload files (PDFs, FAQs, Epub, .txt) containing information like product details, company policies, or specialized knowledge. * **Guardrails (Optional):** Restrict the agent's responses to only the uploaded custom knowledge base, preventing it from relying on its general knowledge training. This is important for ensuring accuracy and control in specific domains. * **Dynamic Context (Optional):** Provide the agent with contextually relevant information. This might include: * Current Date & Time * Caller Information (phone number, email) - if applicable to the deployment method. 5. **Agent Actions (Optional):** * Select specific actions that your agent can perform. These actions might integrate with external services like Zapier and Make or tools. Examples: * Get a funny quote * Get Weather information * Schedule a meeting (integration with Google Calendar) * Send an email/SMS message * Upload an order (integration with backend systems) * Get Financial Data 6. **Deployment:** * **Phone:** Deploy the agent to a phone number. This allows users to call and interact with the agent through voice calls. Pricing details for call minutes may apply. * **Web:** Embed the agent on your website by copying and pasting provided code into the `` section of your HTML. This enables text-based or voice interaction directly on your site. Customize styling and content as needed. 7. **Testing & Iteration:** Once deployed, thoroughly test your agent to refine its behavior, prompts, and knowledge base for optimal performance. ## Additional Notes * The "Actions" functionality is a powerful way to extend your agent's capabilities and integrate it with your existing workflows. * Carefully consider the privacy setting for your agent to control its accessibility. * The Agent Prompt is critical for shaping your agent's responses. Experiment with different prompts to achieve the desired behavior. This comprehensive guide should enable you to create and deploy your first AI voice agent. For advanced functionalities and integrations, refer to the [Play.AI documentation](https://docs.play.ai/) for further details. ## Video Guide to build a Production-grade Voice Agent