Fine-Tuning VRAM Calculator
On 8GB you can QLoRA anything up to about 8B. On 16GB, up to about 14B. On 24GB, up to about 32B — or an 8B in full 16-bit LoRA. Full fine-tuning is off the table on all three: an 8B full fine-tune needs roughly 120GB because Adam stores about 16 bytes per parameter. The calculator below works from each model's real config.json geometry, shows every term in the sum, and tells you which knob to turn when it does not fit.
Will This Fine-Tune Fit?
8.03B parameters (derived from config.json geometry)
Fits — 5.86 GB of 12 GB
QLoRA 4-bit, rank 16, 0.52% of parameters trainable
| Base model weights | 3.89 GB | NF4 4-bit, frozen |
| Adapter + optimizer | 0.47 GB | 41.9M trainable params |
| Activations | 0.50 GB | gradient checkpointing ON |
| Output logits | 0 MB | fused cross-entropy — negligible |
| CUDA overhead | 1.00 GB | context + fragmentation |
| Peak VRAM | 5.86 GB | 6.14 GB headroom |
49% of RTX 3060 / 5070 (12 GB)
How Long Will It Take?
There is no honest way to guess your throughput, so this asks for it. Start the run, let 20 steps go by, and read s/it off the progress bar — that is the number to type below. (If the bar shows it/s instead, use 1 divided by it.)
Roughly 0.75 tokens per English word.
steps = dataset_tokens x epochs / (batch x sequence_length); time = steps x seconds_per_step. Gradient accumulation does not change the total token count, so it does not change this estimate.
What Fits on 8GB, 16GB and 24GB
The short version: your card decides the model size, and the method decides whether that number is generous or brutal. Everything below is this page's own formula at LoRA rank 16 on all linear layers, 2,048-token sequences, batch size 1, gradient checkpointing on and a fused cross-entropy kernel — the settings a sane first run uses. Change any of them in the calculator and the numbers move.
| Model | Params | QLoRA (4-bit) | LoRA (16-bit) | Smallest card that fits |
|---|---|---|---|---|
| Llama 3.2 3B | 3.21B | ~3.2 GB | ~7.6 GB | 8 GB (either method) |
| Qwen3 4B Instruct | 4.02B | ~3.7 GB | ~9.2 GB | 8 GB QLoRA / 12 GB LoRA |
| Mistral 7B v0.3 | 7.25B | ~5.5 GB | ~15.5 GB | 8 GB QLoRA / 16 GB LoRA |
| Llama 3.1 8B | 8.03B | ~5.9 GB | ~16.9 GB | 8 GB QLoRA / 24 GB LoRA |
| Qwen3 8B | 8.19B | ~6.0 GB | ~17.3 GB | 8 GB QLoRA / 24 GB LoRA |
| Qwen3 14B | 14.8B | ~9.7 GB | ~30 GB | 12 GB QLoRA / 32 GB LoRA |
| Qwen2.5 32B | 32.8B | ~19.6 GB | ~65 GB | 24 GB QLoRA |
| Llama 3.3 70B | 70.6B | ~40 GB | ~137 GB | 48 GB QLoRA |
8 GB (RTX 4060, 3070)
QLoRA on anything up to ~8B, at 2K sequences, comfortably. 14B QLoRA does not fit even at short sequences — the frozen 4-bit weights alone are 7.2 GB. Sixteen-bit LoRA tops out around 3B.
16 GB (4060 Ti, 4080, T4)
QLoRA up to ~14B with room for 4K sequences. Note the trap: 16-bit LoRA on an 8B lands at ~17 GB by our maths and 22 GB by Unsloth's conservative table — an 8B LoRA is a 24 GB job, not a 16 GB one.
24 GB (3090, 4090)
QLoRA on a 32B, or 16-bit LoRA on an 8B, or long-context QLoRA on a 14B. A 70B QLoRA still needs ~40 GB, so 48 GB is the real entry point there.
The Formula, Spelled Out
Peak VRAM = base weights + adapter and optimizer state + activations + output logits + ~1 GB of CUDA overhead. No single term dominates at every size, which is exactly why one-line rules of thumb keep giving people the wrong answer.
1. Base weights
Full fine-tune: 16 bytes per parameter — 2 (bf16 weights) + 2 (bf16 gradients) + 4 (fp32 master copy) + 4 + 4 (Adam's two fp32 moments). LoRA: 2 bytes, frozen, no gradients or optimizer state. QLoRA: ~0.52 bytes — 4-bit NF4 plus the double-quantised quantisation constants.
This is where a full fine-tune dies: 8B x 16 bytes = ~120 GB before a single activation is stored.
2. Adapter + optimizer
Adapter parameters = r x (d_in + d_out) summed over every targeted projection, times the layer count. Each of those parameters costs 12 bytes (2 weights + 2 gradients + 8 fp32 Adam moments), or ~6 with an 8-bit optimizer.
Concrete: rank 16 on all seven linear projections of Llama 3.1 8B is ~42M trainable parameters — 0.5% of the model, about 0.5 GB all-in. Doubling the rank doubles this term and nothing else.
3. Activations
batch x seq_len x hidden_size x n_layers x k. With gradient checkpointing, k is about 2 bytes — one bf16 tensor stored per layer boundary, everything else recomputed. Without it, k is roughly 24, covering the attention and MLP intermediates each layer keeps for its backward pass.
k = 24 is a rule of thumb, not a measurement — treat the checkpointing-off figure as indicative. The formula also assumes SDPA or FlashAttention, so there is no seq² attention-matrix term.
4. Output logits — the term everyone forgets
batch x seq_len x vocab_size x 4 bytes, plus their gradient. For a Qwen3 model with a 151,936-token vocabulary at 2,048 tokens, that is about 2.3 GB for one sequence. Fused cross-entropy kernels (Unsloth, Liger) compute it in chunks and make it nearly free.
This is why big-vocabulary models OOM on cards that comfortably hold their weights, and why turning on a fused CE kernel is often the single cheapest fix.
Checked Against Unsloth's Published Numbers
A calculator you cannot check is a guess with a nicer font. Unsloth publishes its own minimum-VRAM table, and it is the best public reference point for this question — the project is Apache-2.0, has over 73,000 GitHub stars, and was pushed to the day we last checked it. Here is our formula next to theirs, at rank 16 on all linear layers, 2,048 tokens, batch 1, checkpointing on, fused cross-entropy on.
| Model size | Unsloth QLoRA | This calculator | Unsloth LoRA 16-bit | This calculator |
|---|---|---|---|---|
| 3B | 3.5 GB | ~3.2 GB | 8 GB | ~7.6 GB |
| 7B | 5 GB | ~5.5 GB | 19 GB | ~15.5 GB |
| 8B | 6 GB | ~5.9 GB | 22 GB | ~16.9 GB |
| 70B | 41 GB | ~40 GB | 164 GB | ~137 GB |
Unsloth figures from its published requirements documentation. Its own caveat applies: “Keep in mind that sometimes more VRAM is required depending on the model so these numbers are the absolute minimum.”
Where we differ, and why: our QLoRA column tracks Unsloth closely across three orders of magnitude. Our 16-bit LoRA column runs consistently below theirs — most likely because their published minimum does not assume a fused cross-entropy kernel and leaves more slack for allocator fragmentation. If you are provisioning a rented GPU, budget to Unsloth's number, not ours. If you are deciding whether your own card is worth trying, ours is the closer estimate. Both are estimates: peak allocation depends on your trainer, your attention implementation, and how PyTorch's allocator happens to fragment on the day.
Measure Your Own Peak in Two Lines
Every estimate on this page is beaten by one number from your own run. PyTorch tracks peak allocation for you; print it after training and you will know exactly how much headroom you had.
import torch
# reset before training starts
torch.cuda.reset_peak_memory_stats()
trainer.train()
peak = torch.cuda.max_memory_allocated() / 1024**3
reserved = torch.cuda.max_memory_reserved() / 1024**3
print(f"peak allocated: {peak:.2f} GiB | peak reserved: {reserved:.2f} GiB")max_memory_allocated is what the tensors actually used — the number to compare against this calculator. max_memory_reserved is what the caching allocator held from the driver, which is what nvidia-smi shows and is always larger. If reserved is far above allocated, you are losing memory to fragmentation, and PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True is worth a try before you give up on a configuration.
Once you have a number that fits, the next question is how to spend the headroom. Longer sequences usually beat a bigger rank: rank 16 to 32 costs half a gigabyte on an 8B and rarely changes output much, while doubling sequence length doubles the activation term and lets you train on documents you were previously truncating.
What This Calculator Does Not Model
Five things will make your real run differ from this estimate, and it is better to know them up front.
- 1.Mixture-of-Experts models. The formula assumes a dense transformer. An MoE holds every expert in memory even though only a few are active per token, so the base term is much larger than an “active parameters” figure suggests. Use the parameter count of the whole model, not the active slice.
- 2.Vision and audio towers. Multimodal checkpoints carry encoders that are not in the language-model geometry, plus image activations that dwarf text ones.
- 3.Preference training. DPO and its relatives keep a frozen reference model in memory — roughly double the base-weight term. GRPO-style RL also stores generated rollouts.
- 4.Multi-GPU sharding. FSDP and DeepSpeed split weights, gradients and optimizer state across devices, which changes the arithmetic entirely. This tool is single-GPU.
- 5.Allocator fragmentation. The 1 GB overhead constant is a stand-in for CUDA context plus slack. Long runs with variable sequence lengths fragment more; padding to a fixed length fragments less.
For the inference side of the same question — how much VRAM to run a model rather than train one — use the VRAM calculator, and the quantization calculator for GGUF file sizes. When you are ready to run the training itself, the QLoRA guide and the LoRA guide are the walkthroughs; training on your own data covers dataset prep, and fine-tuning for a business use case covers whether you should be fine-tuning at all. Training image LoRAs is a different memory profile entirely — see the image LoRA training guide. If nothing fits, best GPUs for AI covers what to buy.
Frequently Asked Questions
How much VRAM do I need to fine-tune an 8B model?
About 6 GB with QLoRA and around 22 GB with 16-bit LoRA, per Unsloth's own published requirements table. Our calculator lands at ~5.9 GB for QLoRA on Llama 3.1 8B at rank 16, 2048 tokens, batch 1, gradient checkpointing on and a fused cross-entropy kernel — which matches Unsloth almost exactly. A full fine-tune of the same model is a different universe: roughly 16 bytes per parameter once you count bf16 weights, bf16 gradients, an fp32 master copy and two fp32 Adam moments, which is about 120 GB. That is the gap the "24GB minimum" versus "8GB works" arguments are really about — they are describing different methods.
What is the difference between LoRA, QLoRA and a full fine-tune?
A full fine-tune updates every weight, so you pay for weights, gradients, an fp32 master copy and optimizer moments — roughly 16 bytes per parameter. LoRA freezes the base model in bf16 (2 bytes per parameter, no gradients, no optimizer state) and trains small low-rank adapter matrices instead; only the adapters carry gradients and Adam moments. QLoRA keeps the same adapters but stores the frozen base in 4-bit NF4, cutting the base term roughly 4x again. The QLoRA paper reports this being enough to fine-tune a 65B model on a single 48 GB GPU "while preserving full 16-bit finetuning task performance."
Why is the optimizer state so much bigger than the adapter?
Because Adam stores two fp32 moments per trainable parameter, on top of the parameter and its gradient. For a bf16 adapter that is 2 bytes of weights + 2 bytes of gradient + 8 bytes of moments = 12 bytes per trainable parameter, six times the weight itself. Switching to an 8-bit optimizer (adamw_bnb_8bit) drops the moments to about 1 byte each and takes that 12 down to roughly 6. It matters much less than people expect for LoRA, though, because the adapter is often under 1% of the model — at rank 16 on all linear layers of Llama 3.1 8B it is about 42M parameters, or half a gigabyte all-in.
Does gradient checkpointing slow training down?
Yes, by about 20%. The Hugging Face Transformers performance guide puts it plainly: gradient checkpointing "reduces memory usage by only storing some of the intermediate activations during the backward pass and recomputing the remaining activations... However, it comes at the cost of slower training speed (~20%)." In exchange, activation memory drops from tens of bytes per hidden-state element down to roughly one stored tensor per layer boundary. On any consumer card, take the trade — a 20% slower run that finishes beats a fast run that OOMs.
How long does it take to fine-tune an LLM?
Steps = dataset_tokens x epochs / (batch_size x sequence_length), and wall clock = steps x seconds_per_step. The only honest way to get seconds_per_step is to read it off your own progress bar after about 20 steps, which is why the calculator asks for it instead of inventing a throughput number. As a sense of scale: a 1M-token dataset for 3 epochs at batch 1 and 2048 tokens is about 1,465 steps, so at 1.5 s/step you are looking at roughly 37 minutes, and at 6 s/step roughly 2.4 hours. Small instruction-tuning datasets finish in an evening; multi-hundred-million-token runs do not.
Can I fine-tune a 70B model on a 24GB card?
No. Unsloth lists 41 GB as the absolute minimum for 70B QLoRA, and our formula gives ~40 GB at rank 16 with 2048-token sequences — both well past 24 GB, and neither number leaves room for a longer sequence length. A 48 GB card (A6000, L40S) is the realistic single-GPU entry point for 70B QLoRA. On 24 GB, the sensible ceiling is a 32B model with QLoRA, or an 8B model with 16-bit LoRA.
Sources
- Model geometry (hidden size, layers, intermediate size, heads, KV heads, head dim, vocab, tied embeddings) transcribed from each model's public
config.jsonon Hugging Face. Parameter counts on this page are derived from that geometry, not copied from marketing names. - unslothai/unsloth and its published requirements documentation — the QLoRA and 16-bit LoRA minimum-VRAM table, and the batch-size OOM guidance. Apache-2.0.
- QLoRA: Efficient Finetuning of Quantized LLMs — 4-bit NormalFloat, double quantisation, paged optimizers, and the single-48GB-GPU 65B claim.
- Hugging Face Transformers — GPU training performance guide — the ~20% gradient-checkpointing slowdown and 8-bit optimizer notes.
- Liger Kernel — fused linear cross-entropy, the kernel family behind the “fused cross-entropy” toggle.
Free to use — just keep the attribution link. Works on any site.
<iframe src="https://localaimaster.com/embed/fine-tuning-vram-calculator" width="100%" height="560" style="border:1px solid var(--line);border-radius:12px;max-width:680px" title="Fine-Tuning VRAM Calculator — Local AI Master" loading="lazy"></iframe>
<p style="font:13px/1.5 system-ui,sans-serif;max-width:680px;margin:6px 0 0"><a href="https://localaimaster.com/tools/fine-tuning-vram-calculator">Fine-Tuning VRAM Calculator</a> by <a href="https://localaimaster.com">Local AI Master</a></p>Know what to actually run on it
All 561 chapters — running local models, RAG, agents, fine-tuning — plus the Python Lab and every course added later.
Ready to Go Beyond Tutorials?
25 structured courses with hands-on chapters - build RAG chatbots, AI agents, and ML pipelines on your own hardware.
Was this helpful?
Go from reading about AI to building with AI
25 structured courses. Hands-on projects. Runs on your machine. Start free.
Written by the Local AI Master Team
The team behind Local AI Master
We build Local AI Master around practical, testable local AI workflows: model selection, hardware planning, RAG systems, agents, and MLOps. The goal is to turn scattered tutorials into a structured learning path you can follow on your own hardware.
Related Guides
Continue your local AI journey with these comprehensive guides
QLoRA Fine-Tuning Guide
The 4-bit walkthrough, start to finish.
LoRA Fine-Tuning Local Guide
How to actually run the training script.
Train an AI Model on Your Own Data
Getting your dataset into the right shape.
VRAM Calculator (Inference)
The same question, but for running models.
Grab the AI Starter Kit — career roadmap, cheat sheet, setup guide
No spam. Unsubscribe with one click.