Best PC for Local LLMs: Three Builds from $1,970, Parts Priced November 2026
Want to go deeper than this article?
Free account unlocks the first chapter of all 25 courses — RAG, agents, MCP, voice AI, MLOps, real GitHub repos.
Got the hardware sorted? Now build on it. You know what to buy — the courses show you what to actually run, fine-tune, and ship on it. First chapter free, no card.
The best PC for local LLMs on November 2, 2026 is a Ryzen 5 9600X with an RTX 5060 Ti 16GB, 32GB of DDR5-6000 and a 1TB NVMe drive, about $1,970 at that day's prices, because 16GB of VRAM runs the whole 8B-14B class with room for context and the 20B mixture-of-experts models, and nothing cheaper does. If you want the 27B-32B class, where the strongest single-GPU models now live, you need 24GB, and the only route to 24GB below a $3,340 RTX 5090 is a used RTX 3090 in the same tower (about $2,840 with the parts below, plus whatever the card costs you). The 32GB RTX 5090 workstation at about $5,800 is for people who are VRAM-bound or speed-bound today, not a default.
Those three builds are the whole page. Every part has a price read from a retailer or vendor page on November 2, 2026, with the URL kept in this page's source; the RAM and GPU markets moved a long way in 2026 and the numbers are not the ones you remember from last year. Speeds are bandwidth ceilings, not benchmarks; the one table of measured numbers this site has is further down. If you want a headless box rather than a desktop, the best local AI server guide is the companion page, and the hardware hub holds the five-tier overview at MSRP.
What is the best PC for local LLMs?
Pick the card by how much memory it has, then spend what is left on everything else. A model that fits in VRAM generates at a speed set by the card's memory bandwidth; a model that does not fit runs at system-RAM speed, several times slower, however good the rest of the PC is. So the question "best PC for local LLM" is really "how much VRAM, and at what bandwidth", and the three honest answers in November 2026 are:
| If you want to run | You need | The build | Price read Nov 2, 2026 | Ceiling on an 8B (bandwidth / 4.9GB) |
|---|---|---|---|---|
| 8B-14B models with context, 20B MoE (gpt-oss:20b) | 16GB | Budget: RTX 5060 Ti 16GB | ~$1,970 | ~91 tok/s |
| The same models at twice the speed | 16GB, 896GB/s | Mid-range: RTX 5070 Ti | ~$2,840 | ~183 tok/s |
| The 27B-32B class (Qwen3.6 27B, Qwen 2.5 Coder 32B) | 24GB | Mid-range, used RTX 3090 variant | ~$1,570 of platform + the card | ~191 tok/s |
| 32B with long context, the fastest single card | 32GB | High-end: RTX 5090 | ~$5,800 ($4,460 at the $1,999 FE price, if you can get one) | ~366 tok/s |
| 70B dense models | 48GB+ | Not a single-card PC: dual 3090s or a Mac Studio M5 Ultra | See the server guide | ~22 tok/s on a 70B |
The best budget AI PC and the best PC for local LLMs are the same machine for most people: the 16GB build. What changed in 2026 is the price of "budget". The RTX 5060 Ti 16GB launched at $429 and listed at $799.99-$879.99 on NVIDIA's own marketplace on November 2; a 32GB DDR5-6000 kit that was a $100 line is now $480-$510. The parts lists below are priced at what you can actually click "add to cart" on, not at MSRP.
Reading articles is good. Building is better.
Free account = the first chapter of all 25 courses, with a per-chapter AI tutor. No card.
Why Build an AI PC?
Because the hardware is a one-time cost and a subscription is not, and because the model and the data stay on your desk. Whether it saves money depends on how many people use it and which tier you would otherwise pay for; the arithmetic is worked through in full, with every price sourced, on the best local AI server page. The short version, from OpenAI's published prices on November 2, 2026:
| Approach | Published price | 3-year cost, one person | What you get |
|---|---|---|---|
| ChatGPT Plus | $20/month | $720 | Frontier models, usage limits, data leaves the building |
| ChatGPT Pro 100 / 200 / 500 | $100 / $200 / $500 per month | $3,600 / $7,200 / $18,000 | Higher limits and the Pro reasoning tier |
| Budget AI PC (this page) | ~$1,970 once | ~$1,970 + electricity | Open models up to the 14B class, unlimited, offline, private |
| Mid-range AI PC | ~$2,840 once | ~$2,840 + electricity | Same, at twice the ceiling, or the 27B-32B class with a used 3090 |
Electricity is small for a desktop that is not a 24/7 server: an RTX 5060 Ti is rated at 180W, so two loaded hours a day is 0.36 kWh, about 11 kWh a month, and at the US average residential rate of 18.16 cents per kWh (EIA, August 2026) that is about $2 a month. The RTX 5090 at 575W is three times that per hour. A one-person Plus subscriber does not save money by building; a Pro subscriber, a household, or anyone whose data cannot leave the building does.
What a local PC runs: Qwen3.6 27B, Qwen 2.5 Coder 32B, DeepSeek R1, Llama 3.3 70B with enough memory, and image generators such as FLUX, with no rate limits and no per-token bill.
The One Rule: VRAM Above All
Spend most of the budget on the graphics card's memory, and choose the card by the largest model file you want to load. Ollama publishes the file size of every tag, and the file has to fit in VRAM with a few gigabytes to spare for the context window. Here is the ladder, with the Q4_K_M sizes from the Ollama registry on November 2, 2026:
| VRAM | Largest model class that fits with context | Example Ollama files | Cards |
|---|---|---|---|
| 8 GB | 7B-8B, tight | llama3.1:8b 4.9GB, qwen3:8b 5.2GB | RTX 5060 Ti 8GB, RTX 4060 |
| 12 GB | 8B with long context; 14B tight | qwen3:14b 9.3GB, qwen2.5-coder:14b 9.0GB | RTX 5070 12GB, used RTX 3060 12GB |
| 16 GB | 14B with context; 20B MoE | gpt-oss:20b 14GB | RTX 5060 Ti 16GB, RTX 5070 Ti, RTX 5080 |
| 24 GB | 27B-32B | qwen3.6:27b 17-19GB, qwen2.5-coder:32b 20GB, deepseek-r1:32b 20GB | Used RTX 3090, RTX 4090 |
| 32 GB | 32B with long context; 35B MoE | qwen3.6:35b-a3b 23GB | RTX 5090 |
| 48 GB+ | 70B | llama3.3:70b 43GB, deepseek-r1:70b 43GB | 2x RTX 3090, Mac Studio 64GB+ |
Use the VRAM calculator to check any specific model, or the VRAM requirements guide for a model-by-model breakdown. The second half of the rule is bandwidth: two cards with the same VRAM can differ two-to-one in speed, because every token means reading every weight once. NVIDIA's specification comparison lists the RTX 5060 Ti at 448GB/s, the RTX 5070 Ti at 896GB/s, the RTX 3090 at 936GB/s and the RTX 5090 at 1,792GB/s. Divide by the model file and you have the ceiling; real output lands below it.
Why a used RTX 3090 (24GB) still beats a new RTX 5070 Ti (16GB) for LLMs: the 3090 holds the 27B-32B class and the 5070 Ti does not, at almost the same bandwidth. The 5070 Ti wins on warranty, power (300W against 350W) and everything that is not an LLM.
Budget Build: 16GB, about $1,970
Target: the 8B-14B class with a working context window, plus 20B mixture-of-experts models. Runs Llama 3.1 8B, Qwen 2.5 14B, DeepSeek R1 14B, gpt-oss:20b and every coding model up to 14B. This is the best budget AI PC on this page and the best PC for local LLMs for most people.
| Component | Part | Price read Nov 2, 2026 | Where |
|---|---|---|---|
| GPU | PNY GeForce RTX 5060 Ti 16GB Dual-Fan (448GB/s, 180W) | $799.99 | NVIDIA marketplace, Add to Cart |
| CPU | AMD Ryzen 5 9600X (6-core) | $179.99 | Walmart |
| Motherboard | MSI MAG B650 Tomahawk WiFi (ATX, AM5) | $154.83 | Walmart |
| RAM | Patriot Viper Venom 32GB (2x16GB) DDR5-6000 | $479.99 | Walmart |
| Storage | Crucial P310 1TB PCIe 4.0 NVMe | $174.99 | Walmart |
| PSU | MSI MAG A850GL, 850W 80+ Gold, ATX 3.1 | $109.99 | Walmart |
| Case | Montech AIR 903 Base, mesh mid-tower | $69.00 | Walmart |
| Total | $1,968.78 |
Notes on the choices:
- The 16GB card is the whole point of the build. The 8GB RTX 5060 Ti was $609.99 on the same marketplace page and it is not a $190 saving, it is a different machine: 8GB cannot hold a 14B model.
- 850W is more supply than a 180W card needs; it is there so a used 3090 or a 5070 Ti drops in later without a second purchase. The Thermaltake GF1 850W was $84.99 if you want to shave it.
- The memory kit is the second-largest line in the build and the one that moved most this year. Corsair's own store listed its 32GB DDR5-6000 CL36 Vengeance kit at $499.99 the same day; buy whichever 6000 MT/s EXPO kit is cheapest the week you order.
- A B650 board is enough: PCIe 4.0 x16 does not limit any card here for inference, and the Tomahawk has three M.2 slots for the model drive you will add later.
Cheaper, with a ceiling: skip the new card and put a used RTX 3060 12GB in the same platform. You lose the 14B class with context and 20B MoE models, and generation slows to a 360GB/s card, but you are well under $1,500. The $200 budget local AI machine goes further down that ladder, to a used office PC and a GTX 1060 for 7B models.
Run this on your own machine and stop paying every month
Pay once and keep it. No renewal, no per-token bill, and nothing you feed it ever leaves your hardware.
Mid-Range Build: 16GB fast or 24GB, about $2,840
Target: everything the budget build runs at twice the ceiling, or, with the used-3090 variant, the 27B-32B class: Qwen3.6 27B, Qwen 2.5 Coder 32B, DeepSeek R1 32B, and image generation with FLUX.
| Component | Part | Price read Nov 2, 2026 | Where |
|---|---|---|---|
| GPU | PNY GeForce RTX 5070 Ti Triple-Fan OC 16GB (896GB/s, 300W) | $1,269.99 | NVIDIA marketplace, Add to Cart |
| CPU | AMD Ryzen 7 9700X (8-core) | $334.99 | Walmart |
| Motherboard | MSI MAG B650 Tomahawk WiFi (ATX, AM5) | $154.83 | Walmart |
| RAM | Patriot Viper Venom 32GB (2x16GB) DDR5-6000 | $479.99 | Walmart |
| Storage | WD_BLACK SN7100 2TB PCIe 4.0 NVMe | $339.99 | Walmart |
| PSU | Corsair RM850x, 850W 80+ Gold, ATX 3.1 | $146.99 | Walmart |
| Case | NZXT H6 Flow, mid-tower | $109.97 | Walmart |
| Total | $2,836.75 |
The 24GB variant, and why it is the one LLM people should build: replace the RTX 5070 Ti with a used RTX 3090. The platform without a GPU is $1,566.76, and the 3090 brings 24GB at 936GB/s, which holds the whole 27B-32B class the 5070 Ti cannot load at all. This page could not read a used-market price from where it was checked; the only RTX 3090 listings Walmart showed on November 2, 2026 were new old stock at $2,049-$2,899, which is not a price anyone should pay for a 2020 card. Price it from completed listings, not asking prices, the week you buy, and the homelab server guide has the inspection checklist for a used card.
Why the 5070 Ti and not the 5080: NVIDIA lists the RTX 5080 at 960GB/s against the 5070 Ti's 896GB/s, seven percent more bandwidth on the same 16GB. The 5080 Founders Edition was $999 and out of stock on NVIDIA's marketplace; partner cards started at $1,579 at Walmart. Seven percent is not worth $300.
What the 8-core CPU is for: nothing the GPU does. It is there because this tower is also a workstation, a dev environment and a browser with sixty tabs, and because when a 32B model spills a couple of gigabytes past 24GB, the spilled layers run on the CPU and system memory. If this machine only serves models, the 9600X from the budget build is fine and saves $155.
High-End Build: 32GB RTX 5090, about $5,800
Target: the 32B class with long context at the highest single-card bandwidth there is, Qwen3.6 35B-A3B and every 27B-32B model with room to spare, and FLUX at full resolution. What it does not do is run a 70B on the card; see the note below.
| Component | Part | Price read Nov 2, 2026 | Where |
|---|---|---|---|
| GPU | GIGABYTE AORUS GeForce RTX 5090 32GB (1,792GB/s, 575W) | $3,340.00 (was $4,649.95) | Walmart; the $1,999 Founders Edition was out of stock on NVIDIA's marketplace |
| CPU | AMD Ryzen 9 9950X (16-core) | $539.00 (was $599.00) | Walmart |
| Motherboard | MSI MAG X870E Tomahawk WiFi (ATX, AM5) | $279.99 | Walmart |
| RAM | G.Skill Flare X5 64GB (2x32GB) DDR5-6000 | $833.63 | Walmart |
| Storage | Samsung 990 Pro 2TB PCIe 4.0 NVMe | $409.99 | Walmart |
| PSU | Corsair RM1000x, 1000W 80+ Gold, ATX 3.1 | $184.99 | Walmart |
| Case | Corsair FRAME 4000D RS ARGB, mid-tower | $99.99 | Corsair |
| Cooling | Corsair Nautilus 360 RS, 360mm AIO | $109.99 | Corsair |
| Total | $5,797.58 |
The RTX 5090 price is the whole story of this build. NVIDIA's launch price was $1,999 and that is what its own marketplace still shows for the Founders Edition, out of stock. The cheapest partner card that was actually buyable on November 2, 2026 was $3,340 at Walmart, marked down from $4,649.95; the same page had a GIGABYTE AORUS at $6,599 on NVIDIA's marketplace and marketplace sellers at $9,499 and up. At the Founders Edition price this build is $4,456.58. Decide which of those two numbers you are willing to pay before you buy any other part, because nothing else in the list makes sense without the card.
What 32GB buys over 24GB: the 32B class with a 64K-plus context window instead of a cramped one, the 35B-A3B mixture-of-experts file (23GB) with room, and roughly double the 3090's ceiling on everything. What it does not buy is a 70B: llama3.3:70b is 43GB, so about 11GB spills to system RAM and those layers run at DDR5 speed. That is why the 64GB memory kit is in this build and not a 32GB one.
Dual-GPU instead: two used RTX 3090s (48GB pooled) on this platform hold a 70B entirely in VRAM. Needs a second full-length slot with dual-slot spacing (the X870E Tomahawk has it) and a 1200W-class supply. The full recipe, including the 4U rack version, is on the best local AI server page.
Component Deep Dive
GPU: the only choice that changes what you can run
New cards, by what they are for, with the price actually seen on November 2, 2026 against the launch price:
| GPU | VRAM | Bandwidth (NVIDIA) | Power | Launch price | Seen Nov 2, 2026 | Largest model class | Verdict |
|---|---|---|---|---|---|---|---|
| RTX 5090 | 32GB | 1,792GB/s | 575W | $1,999 | $1,999 FE out of stock; $3,340-$6,599 partner | 32B with long context | The fastest single card; buy only at a price you can stomach |
| RTX 5080 | 16GB | 960GB/s | 360W | $999 | $999 FE out of stock; $1,579-$3,295 partner | 14B + 20B MoE | Seven percent faster than the 5070 Ti for a lot more money |
| RTX 5070 Ti | 16GB | 896GB/s | 300W | $749 | $1,269.99-$1,499.99 | 14B + 20B MoE | Best new 16GB card for LLMs |
| RTX 5070 | 12GB | 672GB/s | 250W | $549 | not priced on this check | 8B with context | 12GB is the wrong size for the money |
| RTX 5060 Ti 16GB | 16GB | 448GB/s | 180W | $429 | $799.99-$879.99 | 14B + 20B MoE | Best budget AI PC card; half the 5070 Ti's ceiling, same models |
| RTX 5060 Ti 8GB | 8GB | 448GB/s | 180W | $379 | $609.99-$679.99 | 7B-8B | Not for LLMs; the 16GB is $190 more and a different machine |
Used cards: the RTX 3090 (24GB, 936GB/s, 350W) is the one that matters, because no current GeForce card has 24GB below the 5090's 32GB. The RTX 4090 (24GB, faster) holds its value too well to be a bargain. The Tesla P40 (24GB, 346GB/s) is the cheapest 24GB alive and a project, not a part; the P40 guide explains the shroud, the power adapter and the missing Tensor Cores. Dollars per gigabyte across every card, new and used, is tabulated in the GPU price-per-GB guide.
CPU: less than you think
Once the model is on the card, the CPU waits. A Ryzen 5 9600X ($179.99) and a Ryzen 9 9950X ($539) give the same token rate on a GPU-resident model.
- Budget and mid-range: Ryzen 5 9600X ($179.99 at Walmart on November 2, 2026). Six cores is plenty for Ollama, a browser and an IDE.
- Mid-range as a workstation: Ryzen 7 9700X ($334.99). Worth it if the machine is also your compiler.
- High-end: Ryzen 9 9950X ($539, was $599). Only earns its price for fine-tuning, data preparation, or CPU-offloaded layers when a model spills past VRAM.
RAM: 32GB floor, 64GB on the 24GB and 32GB builds
- 32GB (2x16GB DDR5-6000): $479.99-$509.99 at Walmart on November 2, 2026; Corsair direct $499.99-$559.99. The floor for any GPU build.
- 64GB (2x32GB DDR5-6000): $833.63-$1,029.99 at Walmart; Corsair direct $989.99. Worth it where a model can overflow VRAM, which means the 3090 and 5090 builds.
- Buy an EXPO-rated 6000 MT/s kit for AM5 and enable the profile in the BIOS; the speed only matters for spilled layers and CPU-only runs, but a kit that will not run its rated profile is a kit you will fight with.
Storage: any PCIe 4.0 NVMe
Models are 5-45GB files and load in seconds from NVMe. Read speed above 5,000MB/s makes no difference you will notice.
- 1TB: Crucial P310 $174.99, Kingston NV3 $156.50, SanDisk Optimus 5100 $189.00 (Walmart, November 2, 2026). Holds a dozen 8B-14B models.
- 2TB: WD_BLACK SN7100 $339.99, SanDisk Optimus 5100 $299.00, Samsung 990 Pro $409.99. Holds twenty-plus models or a serious 27B-32B collection.
Power supply: size it for the card you will own next
NVIDIA's rated board power, and the supply each build uses:
| GPU | Total graphics power (NVIDIA) | Supply in this guide | Priced Nov 2, 2026 |
|---|---|---|---|
| RTX 5060 Ti 16GB | 180W | 850W (room for a later 3090 or 5070 Ti) | MSI MAG A850GL $109.99 |
| RTX 5070 Ti | 300W | 850W | Corsair RM850x $146.99 |
| RTX 3090 (used) | 350W | 850W | Corsair RM850x $146.99 |
| RTX 5080 | 360W | 850W | |
| RTX 5090 | 575W | 1000W, ATX 3.1 with a native 12V-2x6 cable | Corsair RM1000x $184.99 |
| 2x RTX 3090 | 700W | 1200W-class | See the server guide |
Buy ATX 3.1. The 5070 Ti, 5080 and 5090 all take the 12V-2x6 connector and a supply with a native cable avoids the adapter.
Case and cooling
Inference is a sustained load, not a burst: a long generation, a batch job or a fine-tune keeps the card busy for minutes or hours at a time. A mesh-front mid-tower with three intake fans is enough for every build here; the Montech AIR 903 Base ($69.00), NZXT H6 Flow ($109.97) and Corsair FRAME 4000D RS ($99.99) were the prices seen on November 2, 2026. Only the 9950X needs an AIO (Corsair Nautilus 360 RS, $109.99); the 9600X and 9700X ship with a cooler that is adequate for inference workloads, where the CPU is mostly idle.
Cooling for sustained inference: check for throttling, don't guess
A card that is too hot does not crash; it quietly lowers its clocks, and the first sign is a long job that slows down. NVIDIA's driver tells you directly when that happens, so check before you buy fans:
# Temperature, power and the reason clocks are being held back, every second
nvidia-smi -q -d TEMPERATURE,PERFORMANCE -l 1
In the PERFORMANCE block, the "Clocks Event Reasons" list (called "Clocks Throttle Reasons" on older drivers) flags SW Thermal Slowdown or HW Thermal Slowdown when heat is the limit, and SW Power Cap when the card is at its power limit instead. The TEMPERATURE block prints the GPU's own slowdown and shutdown thresholds next to the current reading, so you are comparing against your card's real limits rather than a number from a forum. NVIDIA documents the mechanism in its Dynamic Thermal Management guide.
Two things that check will not show you:
- Memory junction temperature. On GeForce cards nvidia-smi usually reports memory temperature as N/A. On Windows, HWiNFO reads the GDDR6X junction sensor; Tom's Hardware reported that RTX 3080 and 3090 cards appear to throttle their GDDR6X at around a 110°C junction reading. NVIDIA has not published that limit, so treat it as a ceiling to stay well under, not a target. The 24GB used RTX 3090 build is the one to watch, because its GDDR6X sits on both sides of the board.
- Slow decay over a long run. Run a long generation or batch, log tok/s at the start and after 20 minutes, and compare. If speed drops while a Thermal Slowdown flag is active, the fix is airflow (more front intake, a steeper fan curve) before anything exotic.
AI PC vs Mac Comparison
A Mac wins on silence, power draw and setup time; a PC wins on bandwidth per dollar and on CUDA. Apple's figures are from its Mac Studio tech specs and August 2026 announcement; the PC figures are the builds above.
| Budget AI PC (RTX 5060 Ti) | Mid-range PC, used 3090 | High-end PC (RTX 5090) | Mac Studio M5 Max (36GB) | Mac Studio M5 Ultra (96GB) | |
|---|---|---|---|---|---|
| Price (Nov 2, 2026) | ~$1,970 | ~$1,570 + the card | ~$5,800 | From $2,499 | From $5,499 |
| Model memory | 16GB VRAM | 24GB VRAM | 32GB VRAM | 36GB unified (about 27GB usable by the GPU by default) | 96GB unified |
| Bandwidth | 448GB/s | 936GB/s | 1,792GB/s | 460GB/s (614GB/s on the 40-core GPU) | 1.2TB/s |
| Largest class | 14B, 20B MoE | 27B-32B | 32B long context | 27B | 70B |
| Ceiling on llama3.1:8b (4.9GB) | ~91 tok/s | ~191 tok/s | ~366 tok/s | ~94 tok/s | ~245 tok/s |
| Ceiling on qwen3.6:27b (17GB) | does not fit | ~55 tok/s | ~105 tok/s | ~27 tok/s | ~71 tok/s |
| Ceiling on llama3.3:70b (43GB) | does not fit | does not fit | does not fit | does not fit | ~28 tok/s |
| Power (published) | 180W card | 350W card | 575W card | 480W maximum continuous for the line | 480W maximum continuous |
| Noise | Fans | Fans | Fans, lots | Near silent | Near silent |
| CUDA | Yes | Yes | Yes | No (Metal, MLX) | No (Metal, MLX) |
| Upgrade the GPU later | Yes | Yes | Yes | No | No |
The M5 Max at 36GB is the cheapest machine on this table that holds a 27B, and it does it silently; the used-3090 PC does the same thing at twice the ceiling, with a fan, and with CUDA for everything that is not an LLM. The M5 Ultra is the only single box here that holds a 70B, and the best local AI server page is where that comparison belongs. The Mac mini is the cheaper Apple route (M6 at 153-170GB/s, M5 Pro at 307GB/s, up to 64GB); its prices were not read on this check, so see the Apple Silicon AI guide for configurations.
What Models Can You Run?
Every speed below is the card's published bandwidth divided by the Ollama file size: a ceiling, not a measurement. Real output lands below it; on the one machine this site has measured, 7-8B models reached roughly 80 percent of their ceiling (table below). Use the model recommender for a personalised list.
Budget build (16GB, RTX 5060 Ti, 448GB/s)
| Model (Ollama tag) | Task | File size | Bandwidth ceiling |
|---|---|---|---|
| llama3.1:8b | Chat, coding | 4.9GB | ~91 tok/s |
| qwen3:14b | Chat, reasoning | 9.3GB | ~48 tok/s |
| qwen2.5-coder:14b | Coding | 9.0GB | ~50 tok/s |
| deepseek-r1:14b | Reasoning, maths | 9.0GB | ~50 tok/s |
| gpt-oss:20b | Chat, agents (MoE, ~3.6B active) | 14GB | Decodes far faster than a dense 20B would |
| FLUX (image) | Image generation | ~12GB | Not bandwidth-bound; fits |
Mid-range build (16GB RTX 5070 Ti at 896GB/s, or 24GB used RTX 3090 at 936GB/s)
| Model (Ollama tag) | Task | File size | 5070 Ti ceiling | Used 3090 ceiling |
|---|---|---|---|---|
| llama3.1:8b | Chat, coding | 4.9GB | ~183 tok/s | ~191 tok/s |
| qwen3:14b | Chat, reasoning | 9.3GB | ~96 tok/s | ~101 tok/s |
| gpt-oss:20b | Chat, agents (MoE) | 14GB | fits | fits |
| qwen3.6:27b (q4_K_M) | The current best single-GPU model | 17GB | does not fit | ~55 tok/s |
| qwen2.5-coder:32b | Coding | 20GB | does not fit | ~47 tok/s |
| deepseek-r1:32b | Reasoning | 20GB | does not fit | ~47 tok/s |
High-end build (32GB, RTX 5090, 1,792GB/s)
| Model (Ollama tag) | Task | File size | Bandwidth ceiling |
|---|---|---|---|
| llama3.1:8b | Chat, coding | 4.9GB | ~366 tok/s |
| qwen3.6:27b (q4_K_M) | Everything | 17GB | ~105 tok/s |
| qwen2.5-coder:32b | Coding | 20GB | ~90 tok/s |
| qwen3:32b | Chat, reasoning | 20GB | ~90 tok/s |
| qwen3.6:35b-a3b | MoE, fast | 23GB | fits with room |
| llama3.3:70b | Expert tasks | 43GB | does not fit: ~11GB spills to system RAM, slow |
The one measured table
The table rendered below this article is the only set of measured tokens-per-second numbers on this site: our own runs of the benchmark harness on an Apple M3 Pro laptop with 18GB of unified memory, Ollama 0.12.3, 14–15 October 2026. It is here for one reason: to show how far below the bandwidth ceiling real output lands on a machine we actually own, so you can read every ceiling above with the right discount in mind. It says nothing about the PCs on this page, which we have not measured.
Assembly & Software Setup
Hardware assembly
- Install the CPU, cooler and RAM on the motherboard before it goes in the case, and enable the memory's EXPO profile on the first boot.
- The GPU goes in the top PCIe x16 slot, closest to the CPU.
- Connect every power connector the card has. The 5070 Ti, 5080 and 5090 use a 12V-2x6 cable; plug it fully home. The 3090 wants two or three 8-pin leads on separate cables, not a daisy chain.
- Give it airflow. Inference holds the card at 80-95 percent for minutes; three intake fans and a clear path to the exhaust.
- Cable-manage around the card so nothing blocks its fans.
Software stack
# 1. Install Ubuntu 24.04 LTS (recommended) or Windows 11
# Ubuntu is the easier path for CUDA, Docker and the command line
# 2. Install the NVIDIA driver (use the current recommended version from ubuntu-drivers)
sudo ubuntu-drivers install
nvidia-smi # Verify the card is seen
# 3. Install Ollama
curl -fsSL https://ollama.com/install.sh | sh
# 4. Pull your first model (4.9GB)
ollama run llama3.1
# 5. Install Open WebUI for a chat interface
docker run -d -p 3000:8080 \
-e OLLAMA_BASE_URL=http://host.docker.internal:11434 \
ghcr.io/open-webui/open-webui:main
For Windows, see the Ollama Windows guide. The complete Ollama reference is the Ollama guide, and the chat interface is in the Open WebUI setup guide. Before you buy any card, confirm it is on Ollama's GPU list.
FAQ
Got the hardware sorted? Now build on it.
You know what to buy — the courses show you what to actually run, fine-tune, and ship on it. First chapter free, no card.
Decide before you spend a thousand pounds
The AI Hardware course sizes your build properly — VRAM ladder, real bottlenecks, budget builds — and Pick the Right Model tells you what to run on it.
Liked this? 25 full AI courses are waiting.
From fundamentals to RAG, agents, MCP servers, voice AI, and production deployment with real GitHub repos. First chapter free, every course.
Build Real AI on Your Machine
RAG, agents, NLP, vision, and MLOps - chapters across 25 courses that take you from reading about AI to building AI.
Want the structured version?
Hands-on courses on local AI, from $8.99 a month. The first chapter of each is free.
Keep going
- PILLARLocal AI Hardware Requirements (2026): Complete Guide
- Best Mac for Local AI: Every Apple Silicon Chip Ranked M1–M6
- Ollama System Requirements: Minimum RAM, GPU, Disk Size
- Run an LLM on Your Phone: Offline AI on Android & iPhone
- Best Ollama Models for 8GB, 12GB, 16GB & 24GB VRAM Table
- RX 7900 XTX for AI: LLM Performance, ROCm and Price
- AI Hardware Requirements: CPU, GPU and RAM for Beginners
- AI RAM Requirements 2026: How Much for 7B, 13B, 70B Models?
- Best NPU for AI 2026: Intel vs Qualcomm vs AMD vs Apple
- Best Ollama Models for 8GB RAM and VRAM: 12 Local Picks
Comments (0)
No comments yet. Be the first to share your thoughts!
