Voice AI VRAM Requirements by GPU: TTS and Whisper Tiers

Whisper large-v2 at fp16 with beam size 5 uses 4,525MB of VRAM, and at int8 it uses 2,926MB — both measured by the faster-whisper maintainers on an 8GB RTX 3070 Ti, which settles the most-asked question on this page: yes, Whisper large runs on an 8GB card, with room left over. A 6GB card wants int8 or the turbo model. Add a voice clone and the picture changes by which one: Kokoro's weights are a 327MB file, F5-TTS and XTTS-v2 are ~1.35GB and ~1.87GB, and IndexTTS-2 ships roughly 5.9GB of weights across three components before you load anything else. The realistic floor for a full local voice assistant — speech-to-text, text-to-speech and a 7B LLM resident at the same time — is about 9GB, so 12GB is the tier where that stops being an exercise in unloading models.

Every voice project's README says "GPU recommended" and almost none of them say how many gigabytes. This page fills that gap with the numbers that are actually published, clearly separated into two kinds: measured peak VRAM where a maintainer benchmarked it, and derived floors where nobody did and we are showing our arithmetic instead of inventing a measurement. Where a number does not exist, we say so rather than making one up.

The Short Version, by GPU Tier

Two things to internalise before the detail. First, speech models are small. The largest thing on this page is a 1.55-billion-parameter Whisper — an order of magnitude below the LLMs people size their cards around. Second, your VRAM number is dominated by settings, not by the model: the same Whisper large-v2 moves from 2,926MB to 6,090MB purely on precision and batch size, a 2.1× swing on identical weights. That is the part the READMEs never tell you, and it is the part this page is really about.

Where These Numbers Come From

There is exactly one voice project that publishes a proper peak-VRAM benchmark table, and it is the anchor for everything below.

The faster-whisper README benchmarks transcription of 13 minutes of audio with CUDA 12.4 on an NVIDIA RTX 3070 Ti 8GB. Quoted verbatim from that table:

Notice what the benchmark hardware itself proves. Every row in that table was produced on an 8GB card. If you searched "can I run Whisper large on 8GB", the maintainers' own test rig is your answer, including the batched fp16 run at 6,090MB.

That table also gives us a usable constant. Whisper large-v2 is 1,550M parameters, so its fp16 weights are about 3.1GB and its int8 weights about 1.55GB. Subtract those from the measured figures:

• fp16: 4,525MB measured − ~3,100MB weights = ~1.4GB of runtime overhead
• int8: 2,926MB measured − ~1,550MB weights = ~1.4GB of runtime overhead

The overhead is essentially constant across precision, which makes sense — it is the CUDA context, the encoder activations and the beam-search state, none of which care what dtype the weights are stored in. So the working formula for the rest of this page is:

Peak VRAM ≈ (parameters × bytes-per-parameter) + ~1.4GB runtime + ~1.5GB more if you batch.

That +1.5GB batching figure comes from the same table: 6,090 − 4,525 = 1,565MB at fp16, and 4,500 − 2,926 = 1,574MB at int8. Remarkably consistent, and it is per batch-of-8, not per item.

Everything labelled "measured" below is from that benchmark. Everything labelled "derived" is that formula applied to a verified parameter count or on-disk file size. We did not run these on our own hardware for this page, and we are not going to present arithmetic as if it were a measurement.

Whisper: Every Size, Both Precisions

The official model card table gives approximate requirements; faster-whisper's benchmark gives the real ones — and the real ones are lower. Parameter counts and the "Required VRAM" column below are from the OpenAI Whisper README:

The gap between OpenAI's "~10 GB" for large and faster-whisper's measured 4,525MB is not a contradiction — it is the difference between the reference PyTorch implementation's headroom guidance and an optimised CTranslate2 runtime. Use the measured column when you are deciding whether a card is enough.

large-v3-turbo is the single best value on this table for most people. OpenAI describes it as "an optimized version of large-v3 that offers faster transcription speed with a minimal degradation in accuracy" at ~8× relative speed and roughly half the parameters. The one caveat, straight from the README: "the turbo model is not trained for translation tasks." If you need speech-to-English-translation rather than transcription, stay on large-v3. Our faster-whisper setup guide covers the install and the compute-type flags in detail.

What Beam Size and Batching Actually Cost

Three settings move your number, in this order of impact:

1. Precision. fp16 → int8 saved 1,599MB on large-v2 (4,525 → 2,926). This is the biggest single lever and it is one argument: compute_type="int8" (or int8_float16 on GPU). It is also the lever that gets you from "does not fit in 6GB" to "fits in 6GB".
2. Batch size. batch_size=8 cost ~1,565MB and cut wall-clock time from 1m03s to 17s. That is a genuinely good trade if you have the headroom — a 3.7× speedup for 1.5GB. If you do not, WhisperX's docs suggest --batch_size 4 as the first thing to reduce.
3. Beam size. Every row in the benchmark uses beam 5, so the table does not isolate its cost. Beam search keeps N candidate sequences alive in the decoder, so memory scales roughly linearly in beam width — but the decoder is the small half of Whisper, so dropping to beam 1 frees far less than dropping to int8. Reach for it third, not first.

Does int8 cost accuracy? The faster-whisper README's benchmark table reports time and memory, not a word-error-rate comparison between its own fp16 and int8 runs, so we are not going to quote a number for that trade. What we can say honestly: int8 quantisation of a 1.5B encoder-decoder is a well-trodden path, the option exists precisely because the degradation is small enough to be worth it, and the way to find out for your audio is to transcribe five minutes both ways and diff the transcripts. That takes ten minutes and beats any general claim, including ours.

WhisperX and Diarization

WhisperX's README states it "requires <8GB gpu memory for large-v2 with beam_size=5" using the faster-whisper backend, and reports 70× realtime transcription with large-v2 through batched inference.

Diarization — the "who said what" layer — loads additional models on top of that, and the README's own memory advice if you are tight is to reduce batch size (--batch_size 4), drop to a smaller ASR model, or use --compute_type int8. It also requires a Hugging Face access token and accepting the licence agreement for the speaker-diarization model before anything will download.

We are not going to publish a diarization peak-VRAM figure, because no one publishes one and we have not measured it. What we can tell you concretely is what to check: run nvidia-smi --query-gpu=memory.used --format=csv -l 1 in a second terminal while a diarized job runs, and watch the peak. On a 12GB card there is enough headroom that the answer rarely matters; on 8GB, run the transcription and the diarization as two passes rather than one resident pipeline. Our WhisperX guide covers the alignment and diarization pipeline end to end.

Parakeet

NVIDIA's model card for parakeet-tdt-0.6b-v3 states 0.6 billion parameters, stored as F32, under a CC-BY-4.0 licence, covering 25 European languages. That is roughly 2.4GB on disk at fp32 and ~1.2GB of weights at fp16 — derived, not measured, because NeMo does not publish a peak-VRAM benchmark.

Practically it sits between Whisper turbo and Whisper large in size, and it is the speed option rather than the memory option. Parakeet vs Whisper covers the accuracy and language-coverage trade in full.

TTS and Voice Cloning: The Weights Nobody Lists

Not one of the TTS model cards below publishes a VRAM requirement. We checked. So this table reports the thing that is verifiable — the actual on-disk size of the weight files in each official repository, read from the Hugging Face file listings — and derives a floor from it. On-disk size is a good proxy: the weights have to be somewhere, and for these models activations are small relative to an image or video diffusion pipeline.

Read that table for the shape, not for three-decimal precision. The shape is what matters:

Kokoro is in a class of its own for footprint. 82 million parameters, a single 327MB file, Apache-2.0. It is roughly 4× smaller than the next thing on the list and it is the only entry here that is genuinely trivial to keep resident next to everything else. It does not clone voices — it ships fixed voices (47 voice files in the repo, 523KB each) — which is exactly why it is small. Setup in our Kokoro TTS guide.

IndexTTS-2 is the heavyweight, and it is heavy for a structural reason. It is not one model: it is a 3.48GB GPT-style backbone, a 1.20GB semantic-to-mel converter, and a separate ~1.19GB Qwen-based emotion model, plus a wav2vec2-BERT feature extractor. Roughly 5.9GB of weights before activations. That is a 12GB-card model in practice, and the emotion control is what you are paying the VRAM for.

Chatterbox's file layout is worth understanding before you size it. Resemble AI describe a "0.5B Llama backbone" trained on 0.5M hours of cleaned data, MIT-licensed, 23+ languages. The repo ships the T3 backbone (~2.13GB, i.e. fp32 for 0.5B parameters), an S3Gen vocoder (~1.06GB) and a tiny voice encoder (5.7MB). Load it in fp16 and the weights roughly halve. Our Chatterbox setup guide walks through the install.

XTTS-v2's constraint is legal, not technical. At 1.87GB it is mid-pack on memory, but its licence is the Coqui Public Model License, not an open one — which matters a great deal if the voice is going into anything commercial. We wrote that up separately in the XTTS commercial licence question. F5-TTS has the same shape of problem: MIT code, CC-BY-NC weights.

For a straight quality-and-usability comparison of the three most-used cloning models rather than a memory one, Kokoro vs XTTS vs Chatterbox is the page you want.

Running Them All at Once: The Voice-Assistant Budget

This is the case most people are really sizing for, and the answer is about 9GB — meaning a 12GB card. Here is the arithmetic, laid out so you can substitute your own models:

On a 12GB card that leaves roughly 3GB of slack for display output, the OS, and the fact that peak allocation is spikier than steady state. On an 8GB card it does not fit, and you have two honest options: swap Whisper for the small model at int8 (~244MB of weights), which brings the total near 6.5GB, or load and unload on demand — accept a second or two of latency at each turn in exchange for never holding two models at once.

Substitute a cloning TTS for Kokoro and the sum moves accordingly: XTTS-v2 pushes it to roughly 11.5GB (12GB card, tight), and IndexTTS-2 pushes it past 15GB (24GB card). That single substitution is the biggest decision in the whole build, which is why it is worth knowing the file sizes before you pick.

If you want to size the LLM half of that budget properly rather than using our 7B placeholder, the Ollama model RAM and VRAM table has the per-model figures and our VRAM calculator does the quantisation maths for you. For the assembled pipeline rather than the arithmetic, building a local voice assistant with Whisper, Ollama and Piper is the walkthrough.

Peak vs Steady State: Why Your Number Is Higher Than Ours

Every figure on this page is a peak-allocation figure, and peak is what determines whether you get an out-of-memory error. Steady state — what nvidia-smi shows you if you glance at it mid-run — is meaningfully lower, and that gap is where most "but it said it fits" complaints come from.

Three things make peak exceed steady state in voice pipelines specifically:

• Model loading transiently holds two copies. Weights are read into a buffer and then moved to the target dtype and device. For a moment you hold both.
• The Whisper encoder processes a fixed 30-second window regardless of your clip length, so encoder activations spike at a size that does not shrink for short audio.
• PyTorch's caching allocator does not return memory to the driver. Once a run has peaked, that reservation stays claimed for the process, which is why the second model you load can OOM even though the first one has "finished".

The consequence is practical: leave 15-20% headroom over any number here, and if you want your own real figure rather than ours, run this in a second terminal during a job:

nvidia-smi --query-gpu=memory.used,memory.total --format=csv -l 1

That one-second poll catches peaks that a single glance misses. It is also the only way to get a trustworthy diarization or IndexTTS-2 number, because nobody has published one.

Honest Limitations

• We did not measure the TTS models on our own hardware for this page. The Whisper figures are the faster-whisper maintainers' published benchmark; the TTS figures are derived from verified on-disk weight sizes in the official repositories. Both are labelled as such throughout. We would rather give you a stated method you can check than a fabricated measurement you cannot.
• On-disk size is a floor, not a peak. It ignores activations, the CUDA context and the allocator's caching behaviour. For speech models the gap is smaller than for image or video diffusion, but it is not zero — that is why the derived column gives a range rather than a figure.
• The faster-whisper benchmark is one GPU, one audio file, one CUDA version. RTX 3070 Ti 8GB, CUDA 12.4, 13 minutes of audio. Different architectures, different cuDNN versions and different audio lengths will move the numbers.
• We have no verified parameter count for XTTS-v2, F5-TTS or IndexTTS-2, because their model repositories do not expose one in the Hugging Face model index. We used file sizes instead and said so, rather than back-calculating a parameter count from an assumed dtype.
• Licences are not VRAM but they will stop your project anyway. XTTS-v2 is under the Coqui Public Model License and F5-TTS's weights are CC-BY-NC. Kokoro (Apache-2.0), Chatterbox (MIT) and VoxCPM (Apache-2.0) are the permissive options. Check before you build, not after.
• This is a moving target. Model repositories get repacked, quantised builds appear, and runtimes get more efficient. Every figure was read from the linked source in September 2026.

FAQ

Can I run Whisper large-v3 on an 8GB GPU?

Yes, comfortably. The faster-whisper maintainers benchmarked large-v2 at 4,525MB with fp16 and beam size 5 — on an RTX 3070 Ti, which is an 8GB card. Even the batched run (batch_size=8) came in at 6,090MB and still fit. At int8 it drops to 2,926MB. The "~10 GB" figure in OpenAI's own README is guidance for the reference PyTorch implementation, not a hard requirement for an optimised runtime like CTranslate2.

What are XTTS-v2's VRAM requirements?

Coqui never published one. What is verifiable is the size of the weights in the official repository: model.pth is 1.87GB and dvae.pth is 211MB. Applying the runtime overhead derived from the faster-whisper benchmark puts a realistic floor around 2.5-3.5GB, which makes XTTS-v2 an 8GB-card model on its own and a 12GB-card model if you want an LLM resident alongside it. Note the licence too — XTTS-v2 ships under the Coqui Public Model License, not an open one.

Which voice model fits on a 6GB card?

Speech-to-text: Whisper large-v3 at int8 (2,926MB measured) or large-v3-turbo at fp16 (~1.6GB of weights plus overhead). Text-to-speech: Kokoro-82M, whose weights are a single 327MB file. That combination leaves headroom on 6GB. What does not fit is fp16 Whisper large with batching (6,090MB), or any cloning model resident next to an LLM.

Does int8 quantisation hurt Whisper's accuracy?

The published benchmark measures time and memory, not a like-for-like word-error-rate comparison between fp16 and int8, so we will not quote a number. The saving is large and certain — 1,599MB on large-v2 — and the accuracy cost is small enough that int8 is the standard recommendation for tight cards. Test it on your own audio: transcribe the same five minutes both ways and diff the output. That answers the question for your accent, domain and noise floor better than any published average.

How much VRAM do I need for a full local voice assistant?

About 9GB for a sensible build: Whisper turbo at int8 (~2.2GB), Kokoro (~0.5GB) and a 7B LLM at Q4 with context (~6.4GB), all resident. That means a 12GB card is the first comfortable tier. On 8GB, either drop Whisper to small at int8 or load models on demand instead of keeping them resident. Swap Kokoro for a cloning model and the requirement rises sharply — XTTS-v2 takes it to roughly 11.5GB and IndexTTS-2 past 15GB.

Why does my GPU show more usage than the numbers here?

Because these are model figures and your GPU is doing other things. Your desktop compositor, browser and any other CUDA process hold VRAM too, PyTorch's caching allocator keeps memory reserved after a peak rather than returning it, and Whisper's encoder always processes a fixed 30-second window so its activation spike does not shrink for short clips. Budget 15-20% headroom over any figure on this page, and measure your own peak with nvidia-smi --query-gpu=memory.used --format=csv -l 1 if the margin is tight.

Sources

• SYSTRAN/faster-whisper — the GPU benchmark table (RTX 3070 Ti 8GB, CUDA 12.4, 13 minutes of audio); every "measured" figure on this page
• openai/whisper — model sizes, parameter counts, stated VRAM and the turbo notes
• m-bain/whisperX — the "<8GB for large-v2 with beam_size=5" figure and the low-memory guidance
• Hugging Face model repositories and file listings: nvidia/parakeet-tdt-0.6b-v3, hexgrad/Kokoro-82M, SWivid/F5-TTS, coqui/XTTS-v2, ResembleAI/chatterbox, IndexTeam/IndexTTS-2, openbmb/VoxCPM-0.5B