-# Voice AI Assistant with LiveKit Agents
+# LiveKit Agents Starter - Python
-- Deploy a sandbox app - • - LiveKit Agents Docs - • - LiveKit Cloud - • - Blog -
+A complete starter project for building voice AI apps with [LiveKit Agents for Python](https://github.com/livekit/agents). -A simple voice AI assistant built with [LiveKit Agents for Python](https://github.com/livekit/agents). +The starter project includes: + +- A simple voice AI assistant based on the [Voice AI quickstart](https://docs.livekit.io/agents/start/voice-ai/) +- Voice AI pipeline based on [OpenAI](https://docs.livekit.io/agents/integrations/llm/openai/), [Cartesia](https://docs.livekit.io/agents/integrations/tts/cartesia/), and [Deepgram](https://docs.livekit.io/agents/integrations/llm/deepgram/) + - Easily integrate your preferred [LLM](https://docs.livekit.io/agents/integrations/llm/), [STT](https://docs.livekit.io/agents/integrations/stt/), and [TTS](https://docs.livekit.io/agents/integrations/tts/) instead, or swap to a realtime model like the [OpenAI Realtime API](https://docs.livekit.io/agents/integrations/realtime/openai) +- Eval suite based on the LiveKit Agents [testing & evaluation framework](https://docs.livekit.io/agents/testing/) +- [LiveKit Turn Detector](https://docs.livekit.io/agents/build/turns/turn-detector/) for contextually-aware speaker detection, with multilingual support +- [LiveKit Cloud enhanced noise cancellation](https://docs.livekit.io/home/cloud/noise-cancellation/) +- Integrated [metrics and logging](https://docs.livekit.io/agents/build/metrics/) + +This starter app is compatible with [SIP-based telephony](https://docs.livekit.io/agents/start/telephony/) or any [custom web/mobile frontend](https://docs.livekit.io/agents/start/frontend/). ## Dev Setup @@ -27,30 +29,55 @@ uv sync Set up the environment by copying `.env.example` to `.env` and filling in the required values: -- `LIVEKIT_URL` +- `LIVEKIT_URL`: Use [LiveKit Cloud](https://cloud.livekit.io/) or [run your own](https://docs.livekit.io/home/self-hosting/) - `LIVEKIT_API_KEY` - `LIVEKIT_API_SECRET` -- `OPENAI_API_KEY` -- `DEEPGRAM_API_KEY` +- `OPENAI_API_KEY`: [Get a key](https://platform.openai.com/api-keys) or use your [preferred LLM provider](https://docs.livekit.io/agents/integrations/llm/) +- `DEEPGRAM_API_KEY`: [Get a key](https://console.deepgram.com/) or use your [preferred STT provider](https://docs.livekit.io/agents/integrations/stt/) +- `CARTESIA_API_KEY`: [Get a key](https://play.cartesia.ai/keys) or use your [preferred TTS provider](https://docs.livekit.io/agents/integrations/tts/) -You can also do this automatically using the LiveKit CLI: +You can load the LiveKit environment automatically using the [LiveKit CLI](https://docs.livekit.io/home/cli/cli-setup): ```bash lk app env -w .env ``` -Run the agent in console mode: +## Run the agent + +Run this command to speak to your agent directly in your terminal: ```console uv run python src/agent.py console ``` +To run the agent for use with a frontend or telephony, use the `dev` command: -This agent requires a frontend application to communicate with. Use a [starter app](https://docs.livekit.io/agents/start/frontend/#starter-apps), our hosted [Sandbox](https://cloud.livekit.io/projects/p_/sandbox) frontends, or the [LiveKit Agents Playground](https://agents-playground.livekit.io/). +```console +uv run python src/agent.py dev +``` +In production, use the `start` command: -Run evals +```console +uv run python src/agent.py start +``` + +## Web and mobile frontends + +To use a prebuilt frontend or build your own, see the [agents frontend guide](https://docs.livekit.io/agents/start/frontend/). + +## Telephony + +To add a phone number, see the [agents telephony guide](https://docs.livekit.io/agents/start/telephony/). + +## Tests and evals + +This project includes a complete suite of evals, based on the LiveKit Agents [testing & evaluation framework](https://docs.livekit.io/agents/testing/). To run them, use `pytest`. ```console uv run pytest evals -``` \ No newline at end of file +``` + +## License + +This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details. \ No newline at end of file diff --git a/pyproject.toml b/pyproject.toml index db58cc7..dfd9cf6 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta" [project] name = "agent-starter-python" -version = "0.1.0" +version = "1.0.0" description = "Simple voice AI assistant built with LiveKit Agents for Python" requires-python = ">=3.9" diff --git a/src/agent.py b/src/agent.py index 5f80ce9..cf42bcc 100644 --- a/src/agent.py +++ b/src/agent.py @@ -66,15 +66,21 @@ async def entrypoint(ctx: JobContext): "room": ctx.room.name, } + # Set up a voice AI pipeline using OpenAI, Cartesia, Deepgram, and the LiveKit turn detector session = AgentSession( - vad=ctx.proc.userdata["vad"], # any combination of STT, LLM, TTS, or realtime API can be used llm=openai.LLM(model="gpt-4o-mini"), stt=deepgram.STT(model="nova-3", language="multi"), tts=cartesia.TTS(), # use LiveKit's turn detection model turn_detection=MultilingualModel(), + vad=ctx.proc.userdata["vad"], ) + + # To use the OpenAI Realtime API, use the following session setup instead: + # session = AgentSession( + # llm=openai.realtime.RealtimeModel() + # ) # log metrics as they are emitted, and total usage after session is over usage_collector = metrics.UsageCollector()