bigsk1/voice-chat-ai

Voice Chat AI 🎙️

Voice Chat AI allows you to interact with AI characters using speech. Choose between various characters, each with unique personalities and voices. Run locally with Ollama, or use cloud providers like OpenAI, Anthropic, or xAI.

Mix and match: Use ElevenLabs voices with Ollama models, OpenAI TTS with Anthropic chat, or run completely locally with Spark-TTS voice cloning.

WebRTC Real-Time: Experience instant conversations with OpenAI's Realtime API - interrupt the AI and get immediate responses.

Full Documentation: https://github.com/bigsk1/voice-chat-ai

---

✨ Features

• Multiple LLM Providers: OpenAI, xAI, Anthropic, or Ollama (local)
• Flexible TTS Options:
  • Spark-TTS (local zero-shot voice cloning)
  • OpenAI TTS (cloud, emotional voices)
  • ElevenLabs (premium cloud voices)
  • Kokoro TTS (fast, local)
• OpenAI Realtime API: Real-time speech-to-speech conversations
• Enhanced Mode: Emotional, expressive AI responses
• Local or Cloud: Run entirely locally or mix local/cloud services
• 50+ Characters: Pre-built personalities, games, and story modes
• Interactive Experiences: 15+ game types, escape rooms, trivia, word puzzles
• WebUI + CLI: Full web interface or terminal mode
• Mood Analysis: Sentiment-aware AI responses

---

🐳 Docker Quick Start

CPU Version (Recommended)

Windows with WSL2:

bash
docker run -d \
  -e "PULSE_SERVER=/mnt/wslg/PulseServer" \
  -v \\wsl$\Ubuntu\mnt\wslg:/mnt/wslg/ \
  -v ./elevenlabs_voices.json:/app/elevenlabs_voices.json \
  --env-file .env \
  --name voice-chat-ai \
  -p 8000:8000 \
  bigsk1/voice-chat-ai:latest

Linux/WSL2 Ubuntu:

bash
docker run -d \
  -e "PULSE_SERVER=/mnt/wslg/PulseServer" \
  -v /mnt/wslg/:/mnt/wslg/ \
  -v ./elevenlabs_voices.json:/app/elevenlabs_voices.json \
  --env-file .env \
  --name voice-chat-ai \
  -p 8000:8000 \
  bigsk1/voice-chat-ai:latest

Native Linux (PulseAudio):

bash
docker run -d \
  -e PULSE_SERVER=unix:/tmp/pulse/native \
  -v ~/.config/pulse/cookie:/root/.config/pulse/cookie:ro \
  -v /run/user/$(id -u)/pulse:/tmp/pulse:ro \
  -v ./elevenlabs_voices.json:/app/elevenlabs_voices.json \
  --env-file .env \
  --name voice-chat-ai \
  -p 8000:8000 \
  bigsk1/voice-chat-ai:latest

CUDA Version (GPU)

Requirements: NVIDIA GPU, nvidia-container-toolkit, CUDA drivers

Windows with WSL2:

bash
docker run -d --gpus all \
  -e "PULSE_SERVER=/mnt/wslg/PulseServer" \
  -v \\wsl$\Ubuntu\mnt\wslg:/mnt/wslg/ \
  -v ./elevenlabs_voices.json:/app/elevenlabs_voices.json \
  --env-file .env \
  --name voice-chat-ai-cuda \
  -p 8000:8000 \
  bigsk1/voice-chat-ai:cuda

Linux/WSL2 Ubuntu:

bash
docker run -d --gpus all \
  -e "PULSE_SERVER=/mnt/wslg/PulseServer" \
  -v /mnt/wslg/:/mnt/wslg/ \
  -v ./elevenlabs_voices.json:/app/elevenlabs_voices.json \
  --env-file .env \
  --name voice-chat-ai-cuda \
  -p 8000:8000 \
  bigsk1/voice-chat-ai:cuda

Docker Compose

CPU:

yaml
services:
  voice-chat-ai:
    image: bigsk1/voice-chat-ai:latest
    container_name: voice-chat-ai
    ports:
      - "8000:8000"
    env_file:
      - .env
    environment:
      - PULSE_SERVER=/mnt/wslg/PulseServer
    volumes:
      - /mnt/wslg/:/mnt/wslg/
      - ./elevenlabs_voices.json:/app/elevenlabs_voices.json
    restart: unless-stopped

CUDA:

yaml
services:
  voice-chat-ai-cuda:
    image: bigsk1/voice-chat-ai:cuda
    container_name: voice-chat-ai-cuda
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: all
              capabilities: [gpu]
    ports:
      - "8000:8000"
    env_file:
      - .env
    environment:
      - PULSE_SERVER=/mnt/wslg/PulseServer
    volumes:
      - /mnt/wslg/:/mnt/wslg/
      - ./elevenlabs_voices.json:/app/elevenlabs_voices.json
    restart: unless-stopped

---

⚙️ Configuration

Create a .env file with your settings:

env
# LLM Provider (ollama, openai, anthropic, xai)
MODEL_PROVIDER=ollama
OLLAMA_MODEL=llama3:latest
OLLAMA_BASE_URL=http://localhost:11434

# OpenAI (optional)
OPENAI_API_KEY=sk-...
OPENAI_MODEL=gpt-4o-mini

# Anthropic (optional)
ANTHROPIC_API_KEY=sk-ant-...
ANTHROPIC_MODEL=claude-3-5-sonnet-20241022

# xAI (optional)
XAI_API_KEY=xai-...
XAI_MODEL=grok-2-latest

# TTS Provider (sparktts, openai, elevenlabs, kokoro)
TTS_PROVIDER=openai

# ElevenLabs (optional)
ELEVENLABS_API_KEY=...

# Transcription (openai or local)
TRANSCRIPTION_PROVIDER=openai
FASTER_WHISPER_LOCAL=false

# Default character
CHARACTER_NAME=samantha

Access: http://localhost:8000

---

🎮 Interactive Modes

Story Mode

Experience immersive narrative adventures with AI characters. Each story has unique plot lines, choices, and outcomes.

Game Mode

Play interactive games with specialized game master characters:

• Word Puzzles: Anagrams, rhymes, word chains
• Trivia: General knowledge, science, history
• Escape Rooms: Solve puzzles to escape
• Detective Mysteries: Gather clues and solve cases
• And more: 15+ game types

---

🚀 Installing Spark-TTS (Optional)

For local zero-shot voice cloning inside the container:

bash
# Enter container
docker exec -it voice-chat-ai bash

# Run setup script (GPU recommended)
python setup_sparktts.py

# Restart container
exit
docker restart voice-chat-ai

Set TTS_PROVIDER=sparktts in your .env file.

Model size: ~2GB download (Hugging Face)

---

📋 Available Tags

• latest - Latest stable release (CPU, Python 3.11-slim)
• cuda - CUDA 12.4 support for GPU acceleration
• v1.0.0 - Specific version tags
• main - Development branch (may be unstable)

---

🔧 Environment Variables

Full configuration guide: https://github.com/bigsk1/voice-chat-ai#configuration

---

📚 Documentation

• Installation Guide: https://github.com/bigsk1/voice-chat-ai/blob/main/INSTALL.md
• Requirements Files: https://github.com/bigsk1/voice-chat-ai/blob/main/README_REQUIREMENTS.md
• Spark-TTS Setup: https://github.com/bigsk1/voice-chat-ai/blob/main/docs/SPARKTTS.md
• Games Documentation: https://github.com/bigsk1/voice-chat-ai/blob/main/docs/games.md
• Stories Documentation: https://github.com/bigsk1/voice-chat-ai/blob/main/docs/stories.md

---

🐧 Native Installation (Recommended)

For better performance and audio support, we recommend native installation over Docker:

With pip:

bash
git clone https://github.com/bigsk1/voice-chat-ai.git
cd voice-chat-ai
python -m venv venv
source venv/bin/activate  # Windows: venv\Scripts\activate

# Install PyTorch (choose CPU or CUDA)
pip install torch torchaudio torchvision --index-url https://download.pytorch.org/whl/cpu

# Install dependencies
pip install -r requirements.txt

With uv (faster):

bash
git clone https://github.com/bigsk1/voice-chat-ai.git
cd voice-chat-ai
uv sync  # Creates venv and installs everything

Start: uvicorn app.main:app --host 0.0.0.0 --port 8000

---

🎯 Use Cases

• Personal AI Assistant: Talk naturally with AI for tasks, questions, or brainstorming
• Language Practice: Converse with AI characters in different languages
• Entertainment: Role-play scenarios, play games, or experience stories
• Education: Interactive learning with AI tutors and quiz masters
• Voice Cloning: Clone any voice with a short audio sample (Spark-TTS)
• Development: Build voice-enabled AI applications on this framework

---

🛠️ Troubleshooting

Audio Issues (WSL2)

If microphone doesn't work in WSL2:

bash
# In WSL2
sudo apt-get update
sudo apt-get install pulseaudio
pulseaudio --start

CUDA Not Detected

Verify GPU access in container:

bash
docker exec -it voice-chat-ai-cuda nvidia-smi

Model Download Errors

If Spark-TTS or Whisper models fail to download, check:

• Internet connection
• Hugging Face availability
• Disk space (models can be 1-3GB each)

More troubleshooting: https://github.com/bigsk1/voice-chat-ai#troubleshooting

---

📦 What's New in v1.0.0

🚨 Breaking Changes:

• Removed XTTS (replaced with Spark-TTS)
• Requires Python 3.11+
• New installation methods (pip, uv, conda)

✨ New Features:

• Spark-TTS for faster, better local voice cloning
• pyproject.toml for modern dependency management
• Leaner Docker images (Python 3.11-slim base)
• Improved documentation with detailed install guide

Upgrade Guide: https://github.com/bigsk1/voice-chat-ai/releases/tag/v1.0.0

---

🤝 Contributing

Contributions welcome! Please read our contributing guidelines and submit pull requests.

Areas for contribution:

• New character personalities
• Additional game types
• TTS provider integrations
• UI/UX improvements
• Documentation updates

---

📄 License

MIT License - See https://github.com/bigsk1/voice-chat-ai/blob/main/LICENSE

---

🔗 Links

• GitHub: https://github.com/bigsk1/voice-chat-ai
• Issues: https://github.com/bigsk1/voice-chat-ai/issues
• Docker Hub: https://hub.docker.com/r/bigsk1/voice-chat-ai
• Author: https://github.com/bigsk1 | bigsk1.com

---

⭐ Show Your Support

If you find this project useful, please ***:

• ⭐ Starring the GitHub repository
• 🐛 Reporting bugs or requesting features
• 🤝 Contributing code or documentation
• 📢 Sharing with others who might enjoy it

Made with ❤️ by bigsk1