AI Server Build Under $1,500: The Parts List and What 24GB Fits

Short answer: about $1,500 buys a used 24GB RTX 3090, a Ryzen 7 7700, 64GB of DDR5 and a 2TB NVMe — and 24GB of VRAM comfortably holds any model up to roughly 32B parameters at Q4. A 70B does not fit, at any usable quantisation, without offloading to system RAM and taking a large speed hit.

A serious local AI rig used to mean a workstation card and a Threadripper. That is no longer true. Used 24GB GPUs sit under $750, DDR5 has fallen to commodity prices, and a midrange Ryzen is more CPU than inference ever asks for. Below is the parts list, the arithmetic behind what each tier of VRAM can hold, assembly and BIOS notes, and the software stack. Prices are street prices, not quotes, and they move.

What does a $1,500 AI server parts list look like?

The GPU is the only line where used pricing swings wildly — a 3090 that sells for $720 one month goes for $620 the next. Watch the used market for a few weeks before committing and the total lands under $1,500 without changing a single part.

What actually fits in 24GB of VRAM?

This is the question the whole build turns on, and it is arithmetic rather than opinion. At Q4_K_M a model needs roughly 0.6 GB of VRAM per billion parameters — a 4-bit weight plus the per-block scale metadata k-quants carry works out near 4.8 bits, and 4.8 ÷ 8 = 0.6 bytes. Add roughly 1-2 GB on top for the KV cache and runtime overhead.

So the honest ceiling for this build is a 32B at Q4 with a normal context window, or a 27B with a generous one. That covers essentially every open model people reach for daily.

A 70B is the line this build does not cross. At Q4_K_M the weights alone are around 42 GB — 18 GB more than the card holds. Ollama will still run it by keeping some layers in system RAM, but every token then drags those layers across PCIe, and throughput collapses relative to an all-in-VRAM run. If 70B at full speed is the requirement, the answer is 48 GB of pooled VRAM (two used 3090s), which is the best local AI server build, not this one.

If quantization levels are new to you, GGUF vs GPTQ vs AWQ explains where the 0.6 GB figure comes from and when to step up to Q5 or Q6.

Correction (August 2026): this page previously headlined "Runs Llama 70B at 18 tok/s" and published a table of tokens-per-second figures. A 70B at Q4_K_M is ~42 GB, so on a 24 GB card it can only run with layers offloaded to system RAM. The throughput figures were not independently sourced and have been removed rather than restated; the section below replaces them with a calculated upper bound you can check yourself.

Table of Contents

1. Design Goals
2. The Parts List, With Reasoning
3. Why a Used 3090 Beats a New 4070
4. Assembly Walkthrough
5. BIOS and Linux Tuning
6. Software Stack
7. How Fast Will It Be?
8. Running Costs
9. Mistakes That Cost People Money
10. FAQ

---

Design Goals

Before picking parts, write down what you want. For this build:

1. Run large models locally without a workstation card. Target: comfortable headroom for 32B-class models on one 24GB GPU.
2. Handle a few concurrent users. A household or a small team, which means some parallelism.
3. Quiet enough to live near an office.
4. Low idle power. It runs 24/7, so idle wattage matters more than peak.
5. Upgrade headroom. Room to add a second GPU later.
6. Under $1,500 all-in. Hard cap.

Explicitly not optimised for: gaming, video editing, ECC memory, or future-proofing CPU compute. Inference is overwhelmingly GPU-bound and memory-bandwidth-bound; the CPU stops mattering once you have eight modern cores.

Smaller budget? Budget local AI machine covers the $400-700 tier. Bigger budget? Mac Studio vs PC build is the right comparison.

---

The Parts List, With Reasoning

GPU: Used RTX 3090 Founders Edition 24 GB — ~$720

The single most important part. VRAM capacity is the gating factor on what you can run at all. Every dollar saved elsewhere should go here.

• Why used 3090, not 4070 Ti / 4080 / 4090? A 4070 Ti has 12 GB, a 4080 has 16 GB — both below the 19.2 GB a 32B at Q4 needs. A 4090 has the same 24 GB as a 3090 and costs roughly two and a half times as much used. The 3090 is the cheapest 24 GB card on the market, and that is the whole argument.
• Why the Founders Edition? Its flow-through cooler exhausts heat out of the chassis rather than recirculating it. Partner cards vary a lot here; check reviews for the specific model.
• Buying tips. Buy where returns exist. Avoid ex-mining cards — sustained thermal cycling is hard on the GDDR6X memory modules, and 3090s are known for running those modules hot under load. The used GPU buying guide has the full vetting checklist.

CPU: AMD Ryzen 7 7700 — ~$268

Eight Zen 4 cores at a 65 W TDP, per the AMD product page. Plenty for inference orchestration and any non-LLM workload sharing the box.

• Why not the 7700X? Higher boost clocks, but a 105 W TDP against the 7700's 65 W. On a machine that never turns off, the 40 W delta is the thing you are actually buying.
• Why not the 7600? Six cores is fine for the LLM itself but tight if you also run RAG indexing, embedding generation, or other co-located services.
• Why AM5? Long socket support, DDR5 only, and an upgrade path to later chips without replacing the board.

Motherboard: ASRock B650M PG Lightning — ~$135

B650M is the cheap, sane middle. AM5 socket, two M.2 slots, four DIMM slots, one full x16 PCIe slot for the GPU. Nothing fancy, nothing missing.

• Why mATX? The build is single-GPU and does not need extra PCIe slots. mATX cases are smaller and cheaper.
• Why not X670E? Roughly double the price for extra lanes and beefier VRMs this build never uses. B650 VRMs are not remotely stressed by a 65 W chip.

RAM: G.Skill Flare X5 64 GB DDR5-6000 CL30 — ~$185

64 GB is the sweet spot. Enough to hold a 70B in system RAM if you ever want to try the offload path, and enough for embedding pipelines and a Postgres alongside.

• Why DDR5-6000 CL30? The AM5 sweet spot. Faster kits cost considerably more and barely move LLM throughput, because the model lives on the GPU.
• Why not 128 GB? Because the models you will actually run live in VRAM. Saving that money and putting it toward the GPU is the better trade every time.

Storage: WD Black SN850X 2 TB NVMe — ~$145

Models are large. A 70B at Q5_K_M is around 52 GB by the same arithmetic as above (70 × 0.74). A handful of those plus the OS plus image-generation checkpoints fills 1 TB faster than people expect.

• Why Gen4 rather than SATA? Load time, and the gap is large. A 42 GB model read at a Gen4 drive's ~7,000 MB/s sequential rating takes about 6 seconds; the same file over SATA III, capped at 600 MB/s, takes over a minute. You pay that every cold start.
• Why not Gen5? Gen5 drives run hot and cost more, and sequential model loading does not reward the extra bandwidth much.

PSU: Corsair RM850e 850 W 80+ Gold — ~$115

Add up the board power: the RTX 3090 is a 350 W card, the CPU is 65 W, and drives, fans and board overhead account for perhaps 50 W. That is ~465 W of steady draw, leaving an 850 W unit at roughly 55% load — near the efficiency sweet spot and with room for a second GPU later.

• Why not 750 W? The 3090 is well documented as producing large sub-millisecond transient spikes far above its 350 W rating. A tightly-sized PSU can trip over-current protection on those and reboot the machine mid-run. This is the wrong component to save $40 on.
• Why Gold rather than Platinum? A percentage point or two of efficiency at this load level does not repay the price difference in any reasonable timeframe.

Case: Fractal Design Pop Air Mid-Tower — ~$80

Mesh front, decent stock fans, fits a 320 mm GPU. Any mesh-front mid-tower with clearance for a triple-slot card works equally well.

CPU Cooler: Thermalright Peerless Assassin 120 SE — ~$35

A dual-tower air cooler with far more capacity than a 65 W chip needs, which is exactly why it stays quiet — the fans never have to spin up.

For broader hardware orientation, see the AI hardware requirements complete guide and the canonical local AI hardware requirements guide.

---

Why a Used 3090 Beats a New 4070

This decision drives the whole build, and it is counter-intuitive: a newer card with a newer architecture loses here.

Two things fall out of that table. The 3090 wins on bandwidth per dollar by a wide margin, and bandwidth is the dominant factor for single-stream inference throughput on any model that fits in VRAM. And more importantly, capacity is binary: a model either fits or it does not. A 12 GB card running a 32B is not slower, it is unable.

The 4080 Super is a fine card, but at $1,000 it takes two thirds of the budget and still holds less than a $720 used 3090.

For a deeper apples-to-apples comparison, see RTX 4060 vs RTX 3060 for AI.

---

Assembly Walkthrough

Standard PC assembly with two AI-specific notes. Budget 90 minutes if you have built before, three to four hours if this is your first.

1. CPU and cooler first, outside the case

Drop the 7700 into the AM5 socket. Apply a rice-grain of thermal paste. Mount the cooler per its instructions — AM5 mounting on the Peerless Assassin needs no backplate swap.

2. RAM into A2 and B2 slots

The far slot from the CPU on each side. That is the standard dual-channel configuration. Push firmly until both clips snap; half-seated DIMMs are the number one cause of "won't POST".

3. NVMe into the M.2 slot closest to the CPU

That slot has direct CPU lanes. Use the included thermal pad and heatsink.

4. Mount the motherboard

I/O shield first if it is separate. Standoffs are pre-installed on most modern cases. Do not over-tighten.

5. PSU and cable routing

Modular PSU — run only what you need: 24-pin ATX, EPS 8-pin for the CPU, two PCIe cables for the GPU, SATA for fans. Route behind the tray before plugging anything in.

6. GPU last

Seat the card in the top x16 slot until the latch clicks. Critical for 3090s: the card is heavy and will sag. Use the support bracket that ships with the FE, or a cheap anti-sag brace. Long-term sag stresses the PCIe slot retainer.

Use two separate PCIe cables from the PSU for the two 8-pin connectors, not one cable with a pigtail. A 350 W sustained draw split across a single cable's conductors sits at the edge of its rating.

7. Front panel cables

Tiny pins, motherboard manual required. Power switch, reset, power LED, drive LED. Easy to get wrong, easy to check.

8. First boot

Plug the monitor into the GPU's DisplayPort, not the motherboard's — you want to confirm the card initialises. Power on; you should get the board splash and then UEFI.

No POST? Re-seat the RAM in A2/B2 first. Still nothing? Clear CMOS with the board jumper.

---

BIOS and Linux Tuning

BIOS settings

• EXPO profile: enable. Without it, DDR5-6000 modules run at the JEDEC 4800 MT/s default. Free bandwidth, one toggle.
• Resizable BAR: enable. Helps CUDA workloads that move large buffers.
• PCIe ASPM: disable on the GPU slot. Active state power management can introduce latency spikes during sustained inference.
• Secure Boot: disable initially. It makes NVIDIA driver installation far simpler. Re-enable with signed modules later if your environment requires it.

OS choice

Ubuntu Server 24.04 LTS. A headless server install avoids spending a couple of gigabytes of RAM on a desktop environment you will never open, and the NVIDIA driver path on Ubuntu is the best-documented one.

NVIDIA driver install

sudo apt update
sudo apt install ubuntu-drivers-common
sudo ubuntu-drivers install nvidia:550   # or latest stable

sudo reboot

# Verify
nvidia-smi

You should see the 3090 reporting 24576 MiB total memory.

Linux tuning

# Pin the GPU to max performance state
sudo nvidia-smi -pm 1

# Persist across reboots
sudo nvidia-persistenced --user nvidia-persistenced

# Optional: cap board power if you are thermally constrained
# sudo nvidia-smi -pl 300   # default is 350W on the 3090

# CPU governor: low idle, ramps under load
echo 'GOVERNOR="schedutil"' | sudo tee -a /etc/default/cpufrequtils

# Raise file limits for large models and many connections
echo "* soft nofile 65535" | sudo tee -a /etc/security/limits.conf
echo "* hard nofile 65535" | sudo tee -a /etc/security/limits.conf

Check your own thermals

Used 3090s are worth watching, because the GDDR6X memory modules sit under the same cooler as the core and the thermal pads may be years old. Run a sustained load and watch both sensors:

nvidia-smi --query-gpu=temperature.gpu,temperature.memory --format=csv -l 5

If the memory junction temperature climbs toward its throttle point while the core stays cool, worn thermal pads are the usual explanation and replacement pads are a cheap fix. AI workstation cooling covers the procedure.

---

Software Stack

# Ollama via the official install script
curl -fsSL https://ollama.com/install.sh | sh

# Verify GPU detection and see your own throughput
ollama serve &
ollama run qwen2.5:32b --verbose

--verbose prints the eval rate for your machine, and ollama ps shows whether the model landed 100% on GPU or spilled to CPU. Those two commands answer nearly every "why is this slow" question without anyone else's benchmark.

For multi-user concurrency, swap Ollama for vLLM — much higher batched throughput, considerably harder to operate. For routing in front of either, LiteLLM as an AI gateway is the pattern.

A model set that fits 24GB

ollama pull qwen2.5:7b-instruct-q4_K_M         # daily driver, ~4.7GB
ollama pull qwen2.5-coder:7b-instruct-q4_K_M   # coding, ~4.7GB
ollama pull qwen2.5:32b-instruct-q4_K_M        # heavy reasoning, ~19GB
ollama pull deepseek-r1:7b-qwen-distill-q4_K_M # math/reasoning, ~4.7GB
ollama pull bge-m3                             # embeddings, ~1.2GB

That set is roughly 35 GB on disk. The 32B is the one that uses most of the card; the 7B-class models can share what is left, which is what the headroom column in the fit table was for.

Reverse proxy with Caddy for TLS and auth

ai.lan {
  basicauth {
    you JDJhJDE0$encrypted_hash
  }
  reverse_proxy 127.0.0.1:11434
}

For the deeper hardening pattern, see Ollama production deployment.

---

How Fast Will It Be?

You can put a hard ceiling on generation speed without benchmarking anything, because single-stream token generation is memory-bandwidth bound: the GPU reads essentially every weight once per token. So:

throughput ceiling (tokens/sec) = memory bandwidth (GB/s) ÷ model size (GB)

The RTX 3090's memory bandwidth is 936 GB/s per NVIDIA's specification page. Applying the formula:

Read these as upper bounds, not forecasts. Attention computation, the KV cache, sampling and framework overhead all take a cut, and real output lands well below the ceiling — the smaller the model, the further below, because tiny models stop being bandwidth-bound and start being overhead-bound. What the numbers are genuinely good for is sanity-checking: any claim that this card runs a 32B at 90 tok/s is arithmetically impossible, and anything in the tens of tokens per second for a 32B is plausible.

Run ollama run <model> --verbose on your own build for the number that actually applies to you.

---

Running Costs

Power is the only meaningful recurring cost, and it is calculable from vendor TDP figures rather than guessed:

Full-load draw = 350 W (GPU board power) + 65 W (CPU TDP) + ~50 W (board, drives, fans)
               ≈ 465 W

Cost per hour at $0.16/kWh (US average) = 0.465 kW × $0.16 = $0.074/hour

That is a ceiling — it assumes the GPU is pinned at full board power the entire time, which almost never happens outside sustained batch work. Actual cost depends entirely on duty cycle:

Idle draw sits far below the loaded figure but varies with board, drives and driver power state, so measure it on your own machine with a wall meter rather than trusting anyone's published number.

Against subscriptions, the comparison that matters is simple: a $20/month assistant seat is $240 a year, so a household running two of them recovers the hardware in something like three years — and a small team running four or five recovers it in under two. Below that, buy the subscription; the hardware case is about privacy and control, not savings.

---

Mistakes That Cost People Money

The recurring themes in build threads and support requests for this class of machine:

1. Buying an ex-mining 3090. Years of sustained thermal cycling is hardest on the GDDR6X modules, and degraded memory shows up as instability under long inference runs rather than as an obvious fault at purchase. Prefer cards with a plausible non-mining history and a return window.

2. Under-sizing the PSU. The 3090's transient spikes are the issue, not its 350 W rating. A tight unit trips OCP and reboots the machine mid-run — an intermittent fault that is miserable to diagnose.

3. Skipping the GPU support bracket. A 3090 is over a kilogram of cantilevered card. Sag is slow and then sudden, and it stresses the slot retainer.

4. Installing a desktop environment on a headless server. It costs you RAM and boot time for a GUI nobody logs into. Ubuntu Server is the right image.

5. Forgetting to enable EXPO. DDR5-6000 modules default to 4800 MT/s. It is one BIOS toggle and it is free.

6. Sharing the GPU between inference and a transcoder. Ollama saturating memory bandwidth and a media server transcoding on the same card make each other worse. One workload per GPU.

7. Not budgeting for a UPS. A power blip during a long fine-tune or a model write can corrupt files. A basic line-interactive UPS is cheap insurance on a machine that runs unattended.

---

Frequently Asked Questions

Will this build take two GPUs later? Yes, with a caveat. The B650M PG Lightning has one full x16 slot and one x4. A second 3090 in the x4 slot runs at PCIe 4.0 x4 — acceptable for inference, slower for anything with heavy cross-GPU traffic. If you know you want dual GPUs from day one, spend roughly $50 more on a B650 ATX board with x8/x8 bifurcation.

Why not a Threadripper or a used Xeon for more PCIe lanes? More lanes are nice; the price is not. A used Threadripper Pro board and CPU starts around $900, which forces a downgrade on either the GPU or the RAM to stay under budget. For single-GPU and likely-single-GPU-future builds, AM5 is the better value. Threadripper starts making sense above roughly $2,500.

Can it fine-tune, or only run inference? QLoRA fine-tuning of 7B-13B models is comfortable in 24 GB. Full 70B fine-tuning needs at least 48 GB and realistically 80 GB plus DeepSpeed. Rent cloud A100 or H100 time for that rather than buying into it.

Is it quiet enough for an office? Air-cooled, the 3090's fans are the loudest component under sustained load — everything else in this list is chosen to be near-silent. Whether that is acceptable depends on the room and on the specific card's cooler; Founders Edition and triple-fan partner cards behave quite differently. If it matters, put it in a ventilated closet.

Does Windows work? It does. Linux is the better choice for a dedicated always-on inference box — lighter, better documented for NVIDIA drivers, no desktop overhead — and WSL2 sits in between at the cost of another layer to debug. If the machine is also your daily desktop, Windows is fine.

How does it compare to a Mac Studio? Different trade. A Mac Studio with 64 GB of unified memory holds a 70B at Q4 (~42 GB) entirely in memory, which this build cannot — that is the Mac's real advantage, not raw speed. Against it: roughly twice the price, no upgrade path, and no CUDA. Mac Studio vs PC build works through both sides.

What if I find a 3090 under $500? Buy it. Two of them is 48 GB of pooled VRAM, which is the configuration that unlocks 70B at full speed. The 3090 remains the strongest dollar-per-gigabyte-of-VRAM card available for anything that fits in 24 GB.

Will the 7700 bottleneck the 3090? No. Inference is GPU and memory-bandwidth bound; the CPU schedules work and feeds the GPU. Eight Zen 4 cores are more than that job requires, and money moved from the CPU to the GPU buys real capability while money moved the other way buys none.

---

Conclusion

For under $1,500 you get a machine that runs any model up to about 32B at Q4 entirely in VRAM, supports a small team, and costs roughly $100 a year to power at a realistic duty cycle. The single most important decision is the GPU, and the single most important piece of advice is to buy a used 24 GB card rather than a new 12 GB or 16 GB one. Capacity is binary; speed is a gradient.

Next steps are software: confirm your box clears the Ollama system requirements, then Ollama production deployment, AI gateway with LiteLLM, and adding local AI to an existing app. For picking models, AI on 16GB RAM covers the smaller tier and best local AI models covers the full landscape.

Want the parts list updated as prices move and new GPUs drop? Subscribe to the LocalAIMaster newsletter.