Agent SDKs
Agent Flutter SDK
Integrate AI agents into your Flutter applications
🚀 Features
- 🎙️ Two-way voice conversations with AI agents
- 🔊 Voice Activity Detection (VAD) for natural conversations
- 🧠 Custom actions that allow agents to trigger code in your app
- 📱 Cross-platform - works on iOS, Android, and Web
- 🔌 Audio session management for handling interruptions and device changes
- 📝 Real-time transcripts of both user and agent speech
- 🚦 Rich state management with ValueNotifiers for UI integration
Installation
Add the package to your pubspec.yaml
:
Then save, or run:
Platform Configuration
iOS
- Add the following to your
Info.plist
:
- Add the following to your
Podfile
, since we depend onpermission_handler
to manage permissions andaudio_session
to manage audio sessions.
- Due to an issue of the Onnx Runtime getting stripped by XCode when archived, you need to follow these steps in XCode for the voice activity detector (VAD) to work on iOS builds:
- Under “Targets”, choose “Runner” (or your project’s name)
- Go to “Build Settings” tab
- Filter for “Deployment”
- Set “Stripped Linked Product” to “No”
- Set “Strip Style” to “Non-Global-Symbols”
Android
- Add the following permissions to your
AndroidManifest.xml
:
- Add the following to
android/gradle.properties
(unless they’re already there):
- Add the following settings to
android/app/build.gradle
:
Web
For VAD to work on web platforms, please following the instructions here.
Getting Started
1. Create an Agent on PlayAI
Follow the instructions here to create an agent on PlayAI.
2. Implement the Agent in Your Flutter App
3. Connect the Agent to Start a Conversation
4. Mute and Unmute the User during a Conversation
5. Disconnect the Agent
Key Features
Monitor the Agent’s State
AgentState
: The agent can be in one of four states:
idle
: Not connected to a conversationconnecting
: In the process of establishing a connectionconnected
: Connected and ready to conversedisconnecting
: In the process of ending a conversation
Agent
also exposesValueListenable
s which you can listen to for changes in the agent’s state.
- Pass callbacks as
AgentCallbackConfig
to theAgent
constructor to handle events from the agent.
Agent Actions
One of the most exciting features of the PlayAI Agents SDK is the ability to define custom actions that allow the agent to interact with your app.
Developer Messages
Send contextual information to the agent during a conversation to inform it of changes in your app.
Error Handling
The package uses a robust error handling system with specific exception types:
Lifecycle Management
Don’t forget to dispose of the agent when it’s no longer needed to free up resources.
UI Integration Examples
Mute Button
Speaking Indicator
Tips for Effective Usage
- Prompt Engineering: Craft clear, specific prompts to guide agent behavior
- Action Design: Design actions with clear trigger instructions and parameter descriptions
- Context Management: Use
sendDeveloperMessage
to keep the agent updated on app state - Error Handling: Implement comprehensive error handling for a smooth user experience
- UI Feedback: Use the provided
ValueListenable
s to give clear feedback on conversation state
Acknowledgments
- Voice Activity Detection powered by vad
- Audio session management by audio_session
Was this page helpful?