Self-Host an OpenAI-Compatible TTS API: Kokoro-FastAPI + Speaches

Short answer: run docker run -p 8880:8880 ghcr.io/remsky/kokoro-fastapi-cpu:latest and you have a self-hosted, OpenAI-compatible TTS API — point any OpenAI SDK at http://localhost:8880/v1 with api_key="not-needed" and model="kokoro", and it just works. On an NVIDIA GPU (swap in the -gpu image) it streams first audio in ~300ms and generates 35-100x faster than realtime in about 3.1GB of VRAM, per the project's published benchmarks. Everything is Apache 2.0 — the server and the Kokoro-82M model weights.

That one paragraph is the whole pitch. The rest of this page is the working detail: the exact Docker commands for CPU, NVIDIA, ROCm, and Apple Silicon; the verified latency and VRAM numbers; the Open WebUI settings that make your local chat UI talk out loud; the OpenAI SDK wiring for agents; and Speaches, the sibling project you should pick instead when you want speech-to-text and text-to-speech out of one container. Commands verified against the official repos and docs in August 2026.

---

What You Are Deploying {#what-it-is}

Kokoro-FastAPI is a Dockerized FastAPI wrapper around the Kokoro-82M model that implements OpenAI's speech endpoint — ~5,300 GitHub stars, latest release v0.7.1 (August 2, 2026), Apache 2.0. It exists to solve one problem: every app, agent framework, and chat UI already speaks OpenAI's /v1/audio/speech API, so the fastest way to give them a free local voice is a server that speaks it too.

The model underneath is Kokoro-82M — an 82-million-parameter StyleTTS2-derived model whose model card lists 54 voices across 8 languages in v1.0 (English US/GB, Spanish, French, Hindi, Italian, Japanese, Brazilian Portuguese, Mandarin), all Apache 2.0. Tiny by TTS standards, which is exactly why it can stream in ~300ms on a mid-range GPU while bigger cloning models are still warming up. If you want the model story — quality comparisons, how it was trained, running it raw in Python — that's our separate Kokoro TTS local setup guide. This page is about the serving layer.

On top of the model, the server adds the things you'd otherwise build yourself:

• The OpenAI surface: /v1/audio/speech plus a /v1/audio/voices listing, with mp3, wav, opus, flac, m4a, and pcm output
• Streaming with configurable chunking, so playback starts before generation finishes
• Voice mixing: af_bella(2)+af_sky(1) gives you a 67/33 blend of two voicepacks, normalized automatically
• Long-form handling: the base model is only configured for roughly 30-second output, so the server splits text at sentence boundaries (default cap 450 phoneme tokens per chunk) and stitches the results — which is what makes audiobook-length input practical
• Word-level timestamps (/dev/captioned_speech) and inline control tokens — [pause:1.5s] for silence, [Worcester](/wˈʊstər/) for pronunciation overrides
• A web UI at :8880/web and interactive API docs at :8880/docs

---

Deploy with Docker {#docker-deploy}

The prebuilt images have the model baked in — there is no separate download step. CPU works on any machine including Apple Silicon; the GPU image needs the NVIDIA Container Toolkit. All commands below are from the repo README as of August 2026; the maintainer recommends pinning a release tag (e.g. v0.7.1) rather than :latest for stable use.

CPU only — laptops, mini-PCs, any server:

docker run -p 8880:8880 ghcr.io/remsky/kokoro-fastapi-cpu:latest

NVIDIA GPU (GTX 900-series through RTX 40, CUDA 12.6):

docker run --gpus all -p 8880:8880 ghcr.io/remsky/kokoro-fastapi-gpu:latest

RTX 50-series / Blackwell gets its own CUDA 12.8 tag:

docker run --gpus all -p 8880:8880 ghcr.io/remsky/kokoro-fastapi-gpu:latest-cu128

AMD GPU (ROCm — experimental, x86_64, native Linux host only; passthrough does not work through Docker Desktop or WSL2):

docker run --device=/dev/kfd --device=/dev/dri -p 8880:8880 ghcr.io/remsky/kokoro-fastapi-rocm:latest

Apple Silicon: the CPU image works fine in Docker (and an M3 Pro streams first audio in under a second — see the numbers below). For actual Metal/MPS acceleration you run from a clone via ./start-gpu_mac.sh instead of Docker; the GPU image is CUDA-only and will not run on a Mac.

Sanity-check any of them with:

curl http://localhost:8880/health

Then open http://localhost:8880/web and type a sentence — hearing it speak is the fastest way to confirm the whole path works before you wire anything else up.

---

First Request: SDK + curl {#first-request}

Any OpenAI SDK works unmodified — set base_url to your server, pass the literal string not-needed as the API key, and model="kokoro". This snippet is straight from the project README and is the same code you'd write against OpenAI's hosted TTS, minus the bill:

from openai import OpenAI

client = OpenAI(base_url="http://localhost:8880/v1", api_key="not-needed")

with client.audio.speech.with_streaming_response.create(
    model="kokoro",
    voice="af_sky+af_bella",   # single voice, or a mixed voicepack
    input="Hello world!"
) as response:
    response.stream_to_file("output.mp3")

No SDK? Plain HTTP does the same job:

curl -X POST http://localhost:8880/v1/audio/speech \
  -H "Content-Type: application/json" \
  -o output.mp3 \
  -d '{"model": "kokoro", "input": "Hello world!", "voice": "af_bella",
       "response_format": "mp3", "speed": 1.0}'

GET /v1/audio/voices lists every installed voice ID. For a real-time assistant, request "response_format": "pcm" and feed the chunks straight to your audio device as they arrive — the README ships a PyAudio example (24kHz, 16-bit mono). That streaming path is what separates a server like this from calling the model in a script: your agent starts talking while the rest of the sentence is still generating. If you're building that kind of pipeline end to end, our Whisper + Ollama + Piper voice assistant guide covers the listen-and-think half of the loop.

One practical note from the repo's known-issues list: the server normalizes input text by default (expanding numbers, URLs, and so on), which occasionally rewrites a phrase you wanted verbatim. Pass "normalization_options": {"normalize": false} to switch it off.

---

Performance Numbers {#performance}

On GPU, first audio arrives in ~300ms and full generation runs 35-100x faster than realtime in ~3.1GB of VRAM. On CPU, hardware matters enormously: under 1 second to first audio on an M3 Pro, ~3.5 seconds on an older desktop i7. These are the project's own published benchmarks (RTX 4060 Ti 16GB + i7-11700, Windows 11/WSL2, measured on full-book-length text), not our measurements — treat them as what the maintainer reproduces, and expect your numbers to vary with hardware and chunk settings:

Source: remsky/Kokoro-FastAPI README benchmarks, retrieved August 2026.

Two takeaways worth acting on. First, ~3GB of VRAM means the TTS server coexists with a small LLM on one 8GB card — a 3-4GB quantized model from our 8GB VRAM picks plus Kokoro is a complete talking assistant on a single mid-range GPU. If even that is too tight, POST /dev/unload frees the model between requests and reloads it lazily at a ~5-second cost. Second, the same benchmark suite round-trips generated audio back through faster-whisper and measures a word error rate of roughly 0.033-0.047 across an entire synthesized English audiobook — a useful independent signal that long-form output stays intelligible, and exactly the property you want if the plan is a local audiobook pipeline.

---

Wire It Into Open WebUI {#open-webui}

Open WebUI treats Kokoro-FastAPI as if it were OpenAI: five fields in the audio settings and every chat reply gets a speaker button. The values below are from the Kokoro-FastAPI integration page in Open WebUI's official docs (a community-contributed tutorial hosted there — which tells you how common this exact pairing is):

1. Open Admin Panel → Settings → Audio
2. Set the TTS fields:

The one place this breaks is Docker networking, and the fix depends on your layout. If Open WebUI runs in its own container, localhost points inside that container — use http://host.docker.internal:8880/v1 on Docker Desktop (Windows/Mac). On Linux, the docs recommend a shared network instead:

docker network create local-llm
docker network connect local-llm open-webui
docker network connect local-llm kokoro-fastapi

…then set the base URL to http://kokoro-fastapi:8880/v1, using the container name as the hostname. In a single compose file, the service name works the same way. If you don't have Open WebUI running yet, our Open WebUI setup guide gets you there; combined with an Ollama backend you end up with a fully local ChatGPT-with-voice, no API key in sight.

---

Speaches: TTS + STT in One Server {#speaches}

Speaches (~3,600 stars, MIT) is the pick when you want both speech directions behind one OpenAI-compatible API: faster-whisper transcription plus Kokoro and Piper TTS, with models loaded on demand and unloaded when idle — the project's stated aim is to be "Ollama, but for TTS/STT models." It's the second server this page recommends, and the right one for a different job.

Deploy it (commands from the official install docs, August 2026 — note it listens on 8000, not 8880):

# GPU (CUDA)
docker run --rm --detach --publish 8000:8000 --name speaches \
  --volume hf-hub-cache:/home/ubuntu/.cache/huggingface/hub \
  --gpus=all ghcr.io/speaches-ai/speaches:latest-cuda

# CPU
docker run --rm --detach --publish 8000:8000 --name speaches \
  --volume hf-hub-cache:/home/ubuntu/.cache/huggingface/hub \
  ghcr.io/speaches-ai/speaches:latest-cpu

Unlike Kokoro-FastAPI, models are not baked into the image — you pull them through the project's CLI (the volume mount above is what makes them survive container restarts):

uvx speaches-cli model download speaches-ai/Kokoro-82M-v1.0-ONNX
uvx speaches-cli model ls --task text-to-speech   # verify

Then speech generation is the same OpenAI shape, with the model ID swapped in:

curl -s http://localhost:8000/v1/audio/speech \
  -H "Content-Type: application/json" \
  --output audio.mp3 \
  --data '{"input": "Hello World!",
           "model": "speaches-ai/Kokoro-82M-v1.0-ONNX",
           "voice": "af_heart"}'

The payoff is the other endpoint: the same container serves /v1/audio/transcriptions from faster-whisper, with streaming SSE results — so one base_url covers your agent's ears and voice. It runs Kokoro via ONNX (which is also its CPU story) and additionally serves the very lightweight Piper voices when you want minimum-footprint TTS. The documented caveats, so you're not surprised: the TTS endpoint does not support opus or aac output, and voice listing is still marked TODO in the docs — the well-known voice IDs like af_heart work. For the transcription side in depth, see our faster-whisper guide.

---

Which Server, When {#which-server}

Default to Kokoro-FastAPI for TTS-only; default to Speaches when one container should handle both transcription and speech. Side by side, verified from both projects' repos and docs (August 2026):

If you're still deciding whether Kokoro is even the right voice — versus a cloning model or something more expressive — that's a model question, not a serving question: our best local TTS models roundup covers it properly, including how Kokoro stacks up against the cloning-capable XTTS and Chatterbox.

---

Honest Limitations {#limitations}

This stack is excellent at one thing — fast, cheap, OpenAI-shaped speech — and you should know what it is not.

• No voice cloning. Kokoro ships fixed voicepacks. Mixing gets you new blends of existing voices, never your voice. Cloning needs a different model class entirely.
• No authentication. The API key is literally the string not-needed — the server trusts whoever can reach the port. Fine on localhost or a LAN; if you must expose it, put a reverse proxy with real auth in front. Do not forward port 8880 to the internet as-is.
• CPU latency is hardware-lottery. Sub-second first audio on Apple Silicon, ~3.5s on an older i7 — per the repo's own numbers. Batch narration is fine anywhere; a real-time assistant on an aging CPU will feel laggy.
• English is the proven path. The repo's per-language checks are single-sentence transcription tests — they confirm each voice speaks its language, and the maintainer explicitly flags deeper per-language quality evaluation as open work. Long-form quality is benchmarked in English.
• Long-form has seams. The model generates ~30-second chunks that the server stitches at sentence boundaries. It's good, and the README is upfront that small chunk sizes can introduce intonation artifacts. Expect occasional prosody resets on hour-long output.
• ROCm is experimental — x86_64 native-Linux only, with a first-request kernel-search delay on RDNA 3 that the repo documents workarounds for. It works; it is not turnkey the way CUDA is.
• Only /v1/* is a stable API. The handy extras — captioned speech, unload, debug — live under /dev/* and /debug/*, which the project explicitly reserves the right to change between minor releases (several are off by default behind env flags).

None of these are dealbreakers for the intended job. They are the difference between "deploy it this afternoon" and "discover it in production."

---

Verdict {#verdict}

If an app you run can talk to OpenAI TTS, it can talk to your own hardware by tonight — and for self-hosted speech in 2026, this is the highest-leverage single container you can run.

1. TTS only? Kokoro-FastAPI, GPU image if you have ~3GB of VRAM to spare, CPU image otherwise. Pin a release tag.
2. Ears and voice in one? Speaches, plus one speaches-cli download per model you need.
3. Wire-up is five fields in Open WebUI and one base_url in any OpenAI SDK — that compatibility layer is the entire reason to run a server instead of importing the model in a script.
4. Know the ceiling: fixed voices, English-first quality, no auth. For cloning or maximum expressiveness, pick the model first, then worry about serving it.

The bigger point stands on its own: the OpenAI audio API became a de-facto standard, and Apache-2.0 software now implements it well enough that "hosted TTS" is a convenience, not a requirement. Eighty-two million parameters, three gigabytes of VRAM, one docker run.

---

Sources {#sources}

• remsky/Kokoro-FastAPI — README, Docker commands, benchmark figures, known issues (v0.7.1, August 2026)
• Open WebUI documentation — Kokoro-FastAPI integration tutorial (community-contributed): settings values and Docker networking fixes
• speaches-ai/speaches + speaches.ai — install commands, model management CLI, TTS usage and caveats
• hexgrad/Kokoro-82M — model card: parameters, voices, languages, Apache 2.0 license

---

FAQ {#faq}