Voxtral Local Guide: Mistral's Audio Models for Transcription That Understands

Run Voxtral Mini 3B if you want local transcription you can ask questions about. It is Apache 2.0, posts a 7.05% average WER on the Open ASR Leaderboard's English set — more accurate than Whisper large-v3's 7.44% — and needs ~9.5GB of GPU RAM in bf16 via vLLM, or just 2.47GB as the official Q4_K_M GGUF in llama.cpp. It is not in Ollama's library as of August 2026. And if all you need is fast bulk transcription of English, keep using faster-whisper or Parakeet — Voxtral's case is the understanding, not the speed.

That last sentence is the whole guide in miniature. Voxtral is the first genuinely good open-weight family where transcription and comprehension live in one model: feed it a 30-minute meeting and get the transcript, the summary, and answers to "what did we decide about the launch date?" from a single 3B model — no second LLM pass. Below: what each of the four Voxtral models does, the verified accuracy numbers against Whisper, and working setups for both vLLM and llama.cpp.

---

The Voxtral Family: Four Models, Two Licenses

Four Voxtral models exist as of August 2026. The two you can build a business on are Apache 2.0; the TTS model is not. Here is the map, with specs from each Hugging Face model card:

The two 2507 models share the core spec set from Mistral's announcement and the model cards: a 32k-token context window, audio up to 30 minutes for transcription or 40 minutes for understanding in a single pass, 8 languages (English, Spanish, French, Portuguese, Hindi, German, Dutch, Italian) with automatic language detection, and function calling triggered directly from voice. Mistral pitches the 3B explicitly at local and edge deployment; the 24B is the server-class option.

Mistral's naming convention, for the confused: the four-digit suffix is year-month. 2507 = July 2025, 2602 = February 2026, 2603 = March 2026.

---

Voxtral vs Whisper: The Numbers

Voxtral Mini 3B is more accurate than Whisper large-v3 on the Open ASR Leaderboard's English short-form suite — 7.05% vs 7.44% average WER — at a broadly similar speed class. Voxtral Small 24B pushes accuracy to 6.62% but runs half as fast. Parakeet still embarrasses everyone on throughput.

All numbers below come from the Hugging Face Open ASR Leaderboard's English short-form results (the leaderboard repo's published CSV, data as of March 26, 2026). WER is lower-is-better; RTFx is higher-is-better (an RTFx of 100 = the model transcribes 100 seconds of audio per second of compute on the leaderboard's reference hardware):

Three honest readings of that table:

1. On accuracy, Voxtral beat the model everyone defaults to. A 3B model outscoring Whisper large-v3 (1.55B parameters, but trained purely for ASR) while also being able to reason about the audio is the headline. Mistral's launch post claimed Voxtral "comprehensively outperforms Whisper large-v3" across Common Voice and FLEURS; the leaderboard's independent English suite agrees on the direction.
2. On speed, it did not. Voxtral Mini's RTFx of ~110 sits below Whisper large-v3's ~146, and Parakeet TDT v3 is ~30x faster than either while being the most accurate of all. If your job is "transcribe 500 English podcast episodes," Voxtral is the wrong tool and no amount of understanding features changes that.
3. On breadth, Whisper is untouched. Whisper large-v3 covers 99 languages; Voxtral's cards list 8. Mandarin, Japanese, Arabic, Korean audio? This comparison is over before it starts — use Whisper.

So why does Voxtral matter? Because WER tables measure the wrong thing for half of real workloads. Which brings us to the actual pitch.

---

What "Transcription That Understands" Buys You

The Whisper workflow is a pipeline: transcribe, then feed the text to an LLM for the summary or the answer. Voxtral collapses that into one model and one pass — up to 40 minutes of audio in a 32k context, with Q&A and summarization built in.

Concretely, the same locally-served Voxtral Mini handles all of these without a second model in the loop:

• "Transcribe this." Classic ASR, up to 30 minutes per pass.
• "What did the customer object to, and what did we promise them?" Direct Q&A over a 40-minute call recording — no transcript-then-prompt shuffle, no context lost to transcription formatting.
• "Summarize this lecture into revision notes." One request against the audio file.
• Voice-triggered function calling (per Mistral's announcement): spoken intent mapped straight to API calls — the building block for local voice agents that act instead of just typing.

The pipeline alternative isn't wrong — Whisper piped into a local LLM works, and for long archives it is still how you'd do it. But every pipeline step costs VRAM, latency, and glue code, and a purpose-trained audio-understanding model can use cues a text transcript flattens. We have covered audio-understanding models before — Qwen2-Audio 7B pioneered the category — but Voxtral is the first with Apache licensing, leaderboard-verified ASR accuracy, and first-class serving support all at once.

One spec worth restating because it defines what the model can and cannot do: 32k tokens of context, 30/40-minute audio caps. A 2-hour board meeting still needs chunking. Within those caps, though, you get something Whisper structurally cannot do at any speed.

---

Hardware Requirements

A 12GB GPU runs Voxtral Mini comfortably in bf16 (~9.5GB per the model card). An 8GB card runs the 2.47GB Q4_K_M GGUF through llama.cpp. Voxtral Small 24B wants ~55GB — server or multi-GPU territory.

Practical translations. The bf16 Mini fits an RTX 3060 12GB — the same card our 12GB VRAM model picks are built around — with room left for KV cache on long files. The GGUF path fits basically anything, including laptops. And for calibration on the incumbent: faster-whisper runs large-v3 in about 3GB of VRAM at int8 (our own measurement from the Parakeet comparison), so Voxtral's bf16 footprint is roughly 3x the cost of the model it out-scores — the premium you pay for the language-model half of the architecture.

Voxtral Small at ~55GB is honest server hardware: two 32GB-class GPUs (Mistral's own serve command assumes tensor parallelism across 2) or a big unified-memory box. For most readers the Mini is the right call; the leaderboard gap between them is 0.43 WER points.

---

Setup Path 1: vLLM (Recommended)

Mistral's recommended stack is vLLM ≥ 0.10.0 with the audio extra — two commands to a running OpenAI-compatible server. Straight from the model card:

uv pip install -U "vllm[audio]" --system

vllm serve mistralai/Voxtral-Mini-3B-2507 \
  --tokenizer_mode mistral --config_format mistral --load_format mistral

The three mistral-format flags are required, not decoration — Voxtral ships in Mistral's own weight format rather than vanilla HF layout. For the 24B, the card's command adds tensor parallelism and tool calling:

vllm serve mistralai/Voxtral-Small-24B-2507 \
  --tokenizer_mode mistral --config_format mistral --load_format mistral \
  --tensor-parallel-size 2 --tool-call-parser mistral --enable-auto-tool-choice

Once serving, plain transcription goes through the standard OpenAI-style audio endpoint — the card's own example uses the OpenAI Python client with client.audio.transcriptions.create(...) and a language="en" parameter, with mistral_common[audio] installed client-side to build the request. In outline:

from openai import OpenAI

client = OpenAI(base_url="http://localhost:8000/v1", api_key="EMPTY")
resp = client.audio.transcriptions.create(
    model="mistralai/Voxtral-Mini-3B-2507",
    file=open("meeting.mp3", "rb"),
    language="en",
)
print(resp.text)

Copy the full request-building code from the model card verbatim if this shape errors on your vLLM version — the card is the canonical reference, and it shows both the transcription and the chat/understanding examples (understanding runs through normal chat completions with the audio attached, which is where the Q&A and summarization live).

Yes, Mistral also sells this as an API from $0.001 per minute. That is the price of shipping your meetings, calls, and voice notes to a third party — the entire reason this site exists is that you don't have to.

---

Setup Path 2: llama.cpp on 8GB or Less

llama.cpp officially supports Voxtral Mini as an audio-input model, and ggml-org publishes the official GGUF — one command, 2.47GB:

# OpenAI-compatible server with audio input
llama-server -hf ggml-org/Voxtral-Mini-3B-2507-GGUF:Q4_K_M

# or one-shot in the terminal
llama-mtmd-cli -hf ggml-org/Voxtral-Mini-3B-2507-GGUF:Q4_K_M

This runs through llama.cpp's multimodal (mtmd) stack — Voxtral sits in the audio-models section of the official multimodal docs alongside Ultravox and Qwen's ASR models. The published quant is Q4_K_M at 2.47GB, which fits 8GB cards, older 6GB cards, and CPU-only boxes at a speed penalty. Quantized WER will drift from the bf16 leaderboard number — we found no published measurement of how much, so treat the 7.05% as the bf16 ceiling, not the Q4 promise.

The Ollama answer, because everyone asks: there is no Voxtral in the official Ollama library as of August 2026. We checked the library search directly. llama-server gives you nearly the same developer experience — an OpenAI-compatible endpoint on localhost — so the practical gap is small, but if your whole stack assumes ollama pull, Voxtral is not there yet.

---

The Other Two Voxtrals: Realtime STT and TTS

The February 2026 Realtime model is the sleeper: Apache 2.0 streaming transcription with configurable 80ms–2.4s latency on a single 16GB GPU. The March 2026 TTS model is technically interesting but CC-BY-NC — you cannot ship it commercially.

Voxtral Mini 4B Realtime (2602). A streaming speech-to-text model — roughly a 3.4B language model fused to a ~970M causal audio encoder — that transcribes as audio arrives instead of after the file ends. The card claims sub-500ms delay while matching offline transcription quality, covers 13 languages (adding Chinese, Japanese, Korean, Russian, and Arabic to the 2507 set), and serves via vLLM or Transformers v5.2+, with community C, Rust, and MLX implementations already listed. On the leaderboard it scores 7.68% WER — worse than its offline sibling, which is the expected price of streaming causality, and still in Whisper large-v3's neighborhood. This is the model for live captions or the ears of a voice agent; pair it with our Moshi realtime speech guide if you're mapping the full local-latency landscape.

Voxtral 4B TTS (2603). Streaming text-to-speech: 20 preset voices with adaptation to new ones, 9 languages, 24kHz output across WAV/FLAC/MP3/Opus and more, batch and streaming inference, served via vLLM-Omni ≥ 0.18.0, minimum 16GB GPU per the card. Two honest flags. First, the license: CC-BY-NC 4.0, non-commercial — a hard stop for anything you sell, and a sharp contrast with the Apache transcription models. Second, adoption: roughly 2.3K downloads in the past month on Hugging Face when we checked — early days, tiny next to the established local TTS ecosystem. For commercial-safe voices with real communities behind them, start from our best local TTS models roundup instead.

---

Honest Limitations

Voxtral loses to the specialists at their specialties. Know these before you commit:

• 8 languages. Whisper's 99-language coverage remains the reason it is the default. Audio outside Voxtral's list isn't a benchmark loss, it's a can't.
• 3x the VRAM of the incumbent. ~9.5GB bf16 versus ~3GB for faster-whisper large-v3 at int8. On small GPUs the GGUF closes the gap, with unmeasured quantization cost.
• Not built for bulk. RTFx ~110 versus Parakeet's ~3,333. Overnight batch jobs over big English archives belong to the transducer models.
• 30/40-minute caps. Long recordings need chunking and stitching, same as always — the caps are just generous enough that most meetings fit in one piece.
• The subtitle/diarization ecosystem lives elsewhere. Word-level timestamps, SRT pipelines, speaker labeling — the mature open tooling for all of that grew up around Whisper, and our local speech-to-text guide covers it. Voxtral's serving story is excellent (vLLM, llama.cpp); its downstream-tooling story is young.
• No Ollama, and the TTS is non-commercial. Covered above, repeated here so nobody skims past either.

None of these are disqualifying. They just define the slot: Voxtral is not a Whisper replacement, it's a pipeline replacement.

---

Verdict

Voxtral Mini 3B earns a permanent slot if you ever ask questions about your audio — and changes nothing if you don't.

• Run Voxtral Mini (vLLM, 12GB) for meeting summaries, voice notes you interrogate, call review, and local voice agents that need comprehension. One Apache-2.0 model, transcript + answers, leaderboard-verified accuracy above Whisper large-v3.
• Run the GGUF (llama.cpp, 8GB or less) for the same on modest hardware, accepting unmeasured quant drift.
• Stay with faster-whisper or Parakeet for bulk English transcription, subtitles, and anything needing the mature timestamp/diarization tooling — or any of the 91 languages Voxtral doesn't speak.
• Watch the Realtime model if you're building live voice; it is the most interesting Apache-licensed streaming STT release of 2026 so far.
• Skip the TTS for commercial work — CC-BY-NC settles that regardless of quality.

The quiet story is bigger than one model: audio understanding went Apache 2.0 and leaderboard-competitive in the same family, and it runs on the GPU you already own.

---

Sources

• mistralai/Voxtral-Mini-3B-2507 — license, ~9.5GB bf16 requirement, 32k context, 30/40-min caps, languages, vLLM install/serve commands, transcription client example
• mistralai/Voxtral-Small-24B-2507 — ~55GB bf16 requirement, tensor-parallel serve command
• mistralai/Voxtral-Mini-4B-Realtime-2602 — streaming architecture, latency range, 13 languages, 16GB requirement
• mistralai/Voxtral-4B-TTS-2603 — CC-BY-NC 4.0 license, voices, languages, vLLM-Omni serving, download count (retrieved August 2026)
• Mistral AI — Voxtral announcement — July 15, 2025 release, capability claims, Whisper comparison claims, API pricing
• Hugging Face Open ASR Leaderboard — all WER/RTFx figures (en_shortform results CSV, March 26, 2026)
• llama.cpp multimodal docs and ggml-org/Voxtral-Mini-3B-2507-GGUF — audio support, run commands, Q4_K_M size
• Ollama model library search — absence of Voxtral, checked August 2026

---

FAQ