Best AI Models for the RTX 3090
The 27–32B class at Q4 and the 14B class at Q8 on a card you can only buy second-hand — footprints from Ollama’s tag sizes, speeds as bandwidth ceilings
The best AI model for the RTX 3090 is Qwen3.6 27B — the q4_K_M build is 17GB as Ollama lists it, which leaves ~7GB of the 24GB for context, and its bandwidth ceiling is ~55 tok/s (936 GB/s ÷ 17GB — a ceiling, not a measurement). The rule this whole family runs on: weights plus KV cache must sit entirely inside VRAM, so a 24GB card’s comfortable class is 27–32B at Q4 (17–20GB) or 14B at Q8 (16GB). That gives you Qwen 2.5 Coder 32B (20GB, ceiling ~47) for code, Gemma 4 31B (20GB, dense, image input) and DeepSeek-R1 32B (20GB) for reasoning. What it does not give you is a 70B: Llama 3.3 70B is 43GB at Q4 and still 26GB at Q2_K. Same model list as an RTX 4090, at a used-card price.
Models that run well on the RTX 3090
| Model | Size | Build | Memory used | Ceiling tok/s* |
|---|---|---|---|---|
| WINNERQwen3.6 27B The headline pick for any 24GB card — Qwen’s strongest dense open model, 256K context, with ~7GB of the card left for the KV cache. Pull the explicit q4_K_M tag: the bare 27b tag now resolves to an 18–19GB build. ollama pull qwen3.6:27b-q4_K_M | 27B (dense) | Q4_K_M | ~17GB | ceiling ~55 tok/s (936 ÷ 17GB) |
| Qwen 2.5 Coder 32B The coding pick and the reason people hunt used 3090s — the strongest dense open coder that fits a single consumer card. ~4GB left for context, so set num_ctx on purpose. ollama pull qwen2.5-coder:32b | 32B (dense) | Q4_K_M | ~20GB | ceiling ~47 tok/s (936 ÷ 20GB) |
| Gemma 4 31B Google’s dense 31B (30.7B params) with text and image input and a 256K window — the multimodal pick for 24GB. The 26b tag is a different MoE model (~4B active); the 31b tag is the dense one. ollama pull gemma4:31b | 31B (dense) | Q4_K_M | ~20GB | ceiling ~47 tok/s (936 ÷ 20GB) |
| DeepSeek-R1 32B Visible chain-of-thought at full 32B quality. Reasoning traces eat context, and ~4GB of KV room is enough for most single problems, not for long agent loops. ollama pull deepseek-r1:32b | 32B (dense) | Q4_K_M | ~20GB | ceiling ~47 tok/s (936 ÷ 20GB) |
| Qwen 2.5 Coder 14B The 14B-at-Q8 lane that only 24GB cards get: near-lossless weights with ~8GB of context budget — the better daily driver than the 32B when you feed an agent whole modules. ollama pull qwen2.5-coder:14b-instruct-q8_0 | 14B (dense) | Q8_0 | ~16GB | ceiling ~58 tok/s (936 ÷ 16GB) |
| Flux.1 dev FP8 Ampere has no FP8 tensor cores (those arrived with Ada), so the FP8 file is a memory saving, not a speed-up — but 24GB holds it with the T5 encoder resident and room for LoRAs. The FP16 build (~24GB) fills the card with nothing else loaded. ComfyUI: flux1-dev-fp8.safetensors | 12B | FP8 | ~11GB (~13GB with LoRAs) | diffusion — compute-bound, not timed |
* Ceiling tok/s = the device’s vendor-listed memory bandwidth ÷ the model’s file size as Ollama lists it — the rate a dense model cannot exceed, because every generated token reads the whole weight set once. We have not measured this hardware. Real-world generation usually lands at 50–80% of the ceiling and falls as the context grows; mixture-of-experts rows read only their active parameters, so no ceiling is quoted for them, and image models are compute-bound, so no render time is.
What won't fit on the RTX 3090
- ✗Llama 3.3 70B (needs 43GB at Q4_K_M) — Even the 2-bit llama3.3:70b-instruct-q2_K tag is 26GB — 2GB past the card before any context. Two 3090s (48GB) hold the 43GB Q4 build; our dual-3090 70B guide covers that build.
- ✗Qwen3.6 27B at Q8_0 (needs 30GB) — The near-lossless build of the winner is a 32GB-card job (RTX 5090). On 24GB you run it at Q4 — the 32GB card buys a quant level, not a new model.
- ✗Qwen 2.5 Coder 32B at Q6_K (needs 27GB) — The q5_K_M tag (23GB) technically loads with ~1GB spare, which is no room for a coding context; Q4_K_M at 20GB is the practical ceiling for a 32B on this card.
- ✗gpt-oss:120b (needs 65GB) — Workstation class — an 80GB card or a 128GB unified-memory box.
RTX 3090-specific notes
- →Where this page’s numbers come from: we do not own a 3090. Footprints are the file sizes Ollama prints on each tag page (read 2 Nov 2026), and every tok/s is a ceiling — 936 GB/s, the figure in NVIDIA’s Ampere whitepaper, divided by the gigabytes a dense model reads per token. Real engines land at roughly 50–80% of that and slow as the context fills. We quote no measured rate for this card.
- →The 3090 is a used-only card now: NVIDIA ended production years ago, and there is no laptop variant to be fooled by — NVIDIA’s laptop comparison page lists no RTX 3090 Laptop GPU; the 30-series mobile line stops at the 3080 Ti. What you are buying is a 2020 card that still runs the exact model list of an RTX 4090, because what fits is decided by the 24GB, not the architecture.
- →Power, from NVIDIA’s RTX 3090 spec page: 350 W graphics card power, a 750 W required system power supply, and 2× PCIe 8-pin on the Founders Edition (a 12-pin adapter ships in the box); partner cards mostly use two or three plain 8-pin plugs, so no 12VHPWR cable is needed. The 3090 is known for millisecond power spikes well above 350 W; a quality 750–850 W unit is the no-drama choice. Token generation is a sustained load, not a gaming burst.
- →Used-market reality on 2 Nov 2026: resaleprices.com’s tracker (updated 1 Nov 2026, 328 listings) puts the used market average at $1,568 with a fair asking range of $1,499–1,606, and videocardprices.com showed an eBay used listing at $1,299 on the same day. That is far above the ~$700–1,000 this card sold for in mid-2026 — the memory shortage pushed every 24GB card up. Compare against a used 4090 and a new RX 7900 XTX before paying it; our GPU-prices guide tracks the moves.
- →Used-card checks that matter for AI specifically: run `nvidia-smi -q -d TEMPERATURE` under a long decode and watch the memory junction temperature — the GDDR6X modules on the back of the 3090 are the part that ran hottest in mining service, and thermal-pad replacements are common on ex-mining cards. Then load a 20GB model and run `ollama ps`: it must read 100% GPU. Any CPU share means the card or the driver is not giving you the full 24GB.
- →The 3090 is the only card on this hub with an NVLink connector, which is why 2×3090 is the classic 48GB build: llama3.3:70b at Q4 (43GB) fits the pair. The bridge is optional for inference — Ollama and llama.cpp split a model across two cards over PCIe without it — so do not pay a premium for one unless you plan to train.
- →What a 4090 buys over this card is speed, not capacity: NVIDIA’s whitepapers put them at 936 vs 1,008 GB/s, so the decode ceilings on this page are only ~8% apart; the 4090’s larger gap is compute and L2 cache, which shows on prompt processing and image generation. We have no measured ratio to quote. If the model list is the point, the 3090 runs all of it; the full comparison is in our 4090 vs 3090 guide.
Quick start
curl -fsSL https://ollama.com/install.sh | shollama run qwen3.6:27b-q4_K_Mollama psGo from "it runs" to actually building
All 561 chapters — running local models, RAG, agents, fine-tuning — plus the Python Lab and every course added later.
Frequently asked questions
What is the best LLM to run on an RTX 3090?
Qwen3.6 27B at Q4_K_M — 17GB as Ollama lists the tag, ~7GB of the 24GB left for context, and a bandwidth ceiling of ~55 tok/s (936 ÷ 17GB). For code, Qwen 2.5 Coder 32B (20GB); for reasoning, DeepSeek-R1 32B (20GB); for image input, Gemma 4 31B (20GB). Every figure here is a tag size or a ceiling, not a rate we measured.
Is the RTX 3090 still good for AI in 2026?
Yes, because the thing that decides what runs is 24GB of VRAM, and the 3090 has as much as a 4090. It holds the entire 27–32B class at Q4 and the 14B class at Q8, which no 16GB card does at any price. What it lacks is Ada and Blackwell compute (slower prompt processing and image generation) and FP8 hardware. At the current used price (~$1,300–1,600 on 2 Nov 2026) the value case is thinner than it was at $700–1,000, so compare it against a used 4090 and a new 7900 XTX before buying.
RTX 3090 vs RTX 4090 for local AI — same models?
Identical model list: both are 24GB cards, so everything on this page runs on either. The 4090 is faster — its memory bandwidth is ~8% higher (1,008 vs 936 GB/s per NVIDIA’s whitepapers) and its compute is much higher, which matters most for prompt processing and diffusion. We have not measured the ratio ourselves. If you only need the models to fit, the 3090 does the job for less; our 4090 vs 3090 comparison goes through the trade-offs in depth.
Can an RTX 3090 run a 70B model?
Not on one card. Llama 3.3 70B is 43GB at Q4_K_M and 26GB even at the 2-bit q2_K tag, so it does not fit 24GB without offloading to system RAM, at which point generation runs at system-RAM pace. Two 3090s give you 48GB and the Q4 build fits — that is the classic cheapest-70B build, and it works over plain PCIe without the NVLink bridge.
How many tokens per second does an RTX 3090 get?
We have not measured one, so we will not print a number as if we had. The ceiling is computable for any dense model: NVIDIA’s whitepaper lists 936 GB/s of memory bandwidth, and every generated token reads the whole weight set once, so a 17GB model cannot exceed 936 ÷ 17 ≈ 55 tok/s and a 20GB model cannot exceed ≈ 47. Real engines land at roughly 50–80% of that, and any page quoting an exact 3090 figure without naming the engine, quant and context length is guessing.
Related guides
Ready to Go Beyond Tutorials?
25 structured courses with hands-on chapters - build RAG chatbots, AI agents, and ML pipelines on your own hardware.
Was this helpful?