Homelab AI Server Build: Used RTX 3090 Budget Guide

Published on April 10, 2026 • 22 min read

A used RTX 3090 puts 24GB of VRAM in a desktop for roughly the price of a mid-range new card, which is why it keeps showing up in homelab AI builds. This guide prices out three complete servers around that card — $960, $1,200 and $2,300 as specced below — then walks through assembly, Ubuntu Server, Ollama, remote access, and the power and break-even arithmetic so you can decide whether owning the hardware beats renting it.

One thing to be straight about up front: the throughput figures in this guide are arithmetic ceilings derived from published memory bandwidth, not stopwatch numbers. We do not own this hardware and nothing here was benchmarked on a 3090 by us. Ceilings are still useful — they tell you what a card physically cannot exceed, which is what you actually need when sizing a build.

---

What you get from this guide:

• Complete bill of materials at $800, $1,500, and $2,500 price points
• Step-by-step assembly, OS installation, and software stack
• Power consumption arithmetic and noise reduction tactics
• Tailscale remote access so you can query your models from anywhere
• Real cost comparison showing when a homelab beats cloud GPU rental

If you need help choosing a GPU before you start, the complete GPU buying guide for AI covers every current option. For understanding VRAM and RAM requirements in detail, see the AI hardware requirements guide.

Table of Contents

1. Why Build a Dedicated AI Server
2. The Used RTX 3090 Value Proposition
3. Bill of Materials: Three Tiers
4. Step-by-Step Assembly
5. Ubuntu Server Installation
6. Ollama and Open WebUI Setup
7. Tailscale Remote Access
8. Power Consumption Analysis
9. Noise Management
10. Cloud GPU Cost Comparison

---

Why Build a Dedicated AI Server

Running AI on your daily driver machine is fine for experimentation, but it falls apart when you need persistent inference. Every time you close your laptop or reboot for updates, your models unload. Every time you run a large model, your other applications crawl.

A dedicated server solves this permanently:

Always-on inference. Your models stay loaded in VRAM around the clock. That removes the cold-load penalty entirely: a 20GB model has to be pulled off the SSD and written into VRAM before the first token, and on a warm server it is already there. Anyone on your network can query models instantly.

No resource contention. Your workstation stays snappy for actual work. No more choosing between running a 13B model and having enough RAM for your browser tabs.

Headless efficiency. Without a desktop environment, Ubuntu Server dedicates all resources to inference. You gain 1-2GB of RAM and measurable GPU overhead by dropping the display server entirely.

Learning infrastructure. Building and maintaining a server teaches networking, Linux administration, and service management. These skills transfer directly to production AI deployment.

The counterargument is cloud GPU rental. At $0.80/hr for an A100 on Lambda Labs, you can rent serious compute on demand. But if you run models more than 3-4 hours per day, dedicated hardware pays for itself within roughly a year against the mainstream providers. The full break-even math is in the cost comparison section. The other middle path, if you already have storage hardware humming in the rack, is running AI on a QNAP or TrueNAS NAS before committing to a second box.

---

The Used RTX 3090 Value Proposition

The RTX 3090 is the single best value proposition in AI hardware right now. Here is why:

24GB VRAM for $500-600. New GPUs with 24GB VRAM (RTX 4090, RTX 5090) cost $1,600-$2,000+. A used 3090 delivers the same VRAM capacity for a third of the price. VRAM is the hard constraint for model size, not compute speed.

What 24GB VRAM actually runs:

• 7B models at full FP16 precision (14GB) with room for KV cache
• 13B models at Q4_K_M quantization (7.4GB) with generous context
• 33B models at Q4_K_M quantization (18.5GB) at 2K context
• 70B models only with CPU offload — even Q2_K lands around 26GB, above the card's 24GB

What the 3090's bandwidth allows (arithmetic ceiling, not a benchmark):

Token generation on a local LLM is memory-bandwidth bound. Every token requires reading the whole weight set out of VRAM once, so the hard ceiling is simply card bandwidth ÷ bytes of weights. NVIDIA rates the RTX 3090 at 936 GB/s. Q4_K_M costs roughly 0.6GB per billion parameters, which gives:

Read those as speed limits, not speedometers. Real output lands well below the ceiling once sampling, attention over a growing KV cache, and framework overhead are counted — but nothing you do to the software will push a 33B past ~47 tok/s on this card, and that is the number that matters when you are deciding what to buy.

Buying a used 3090 safely:

Check eBay completed listings, not current asking prices. As of early 2026, the 3090 Founders Edition sells for $480-550. Third-party cards (EVGA FTW3, MSI Suprim X) go for $500-620.

Avoid cards with modified BIOS or flashed firmware. Ask the seller if the card was used for mining. Mining cards that ran at controlled temperatures (under 85C memory junction) and stable power limits are actually fine. The danger is cards that ran with thermal pad modifications at extreme temperatures.

Test any used card immediately. Run nvidia-smi to check for ECC errors, then run a sustained load for 30 minutes. If it survives without artifacts or crashes, you have a good card.

---

Bill of Materials: Three Tiers

Tier 1: Budget Build ($800)

This build runs 7B-13B models comfortably and handles 33B at reduced quality.

Prices reflect Q1 2026 US market. You can cut this to $800 by hunting deals on the CPU and buying an open-box PSU.

Tier 2: Recommended Build ($1,500)

The build this guide is written around. Handles 70B quantized models with system RAM offloading, or two mid-size models resident at once.

64GB system RAM matters for CPU offloading. When a 70B model exceeds your 24GB VRAM, layers spill to system RAM. More RAM means more offloaded layers before you hit swap.

Tier 3: Performance Build ($2,500)

For running multiple models simultaneously or handling dual GPUs in the future.

Two 3090s give you 48GB of aggregate VRAM. Ollama does not natively split across GPUs for a single model, but you can run two separate models simultaneously, one per card. For tensor parallelism across GPUs, use vLLM or llama.cpp directly.

Important PSU note: A single RTX 3090 can spike to 450W during load. A dual-GPU build needs 1500W minimum. Do not skimp on the PSU. An underpowered unit causes random shutdowns that corrupt models and filesystems.

---

Step-by-Step Assembly

Pre-Build Checklist

Before you start:

• Ground yourself (touch the PSU case while plugged in but switched off)
• Clear a large, clean, non-carpeted workspace
• Have a Phillips #2 screwdriver and zip ties ready
• Keep the motherboard box as an anti-static workspace

Assembly Order

Step 1: CPU Installation

1. Open motherboard CPU socket lever
2. Align the golden triangle on CPU with socket triangle
3. Drop CPU straight down (zero force needed on AMD)
4. Close lever firmly — some resistance is normal

Step 2: RAM Installation

1. Open RAM slot clips (use slots A2 and B2 for dual channel)
2. Align notch on RAM stick with slot key
3. Press down firmly until both clips snap shut
4. Verify: both clips locked, RAM seated flush

Step 3: NVMe SSD

1. Remove M.2 heatsink screw and heatsink
2. Insert NVMe at 30-degree angle into M.2 slot
3. Press down flat and secure with standoff screw
4. Replace heatsink

Step 4: Motherboard into Case

1. Install I/O shield (press from inside until all tabs click)
2. Align motherboard standoffs with case holes
3. Secure with 9 screws (hand-tight plus quarter turn)
4. Do NOT overtighten — you will crack the PCB

Step 5: PSU Installation

1. Mount PSU with fan facing down (if case has bottom vent)
2. Secure with 4 screws from case rear
3. Route cables through back panel cable management holes
4. Connect: 24-pin ATX, 8-pin CPU, SATA (if needed)

Step 6: GPU Installation

1. Remove 2-3 PCIe slot brackets from case rear
2. Open PCIe x16 slot retention clip
3. Align GPU with slot and press down firmly until clip locks
4. Secure GPU bracket with screws
5. Connect 2x 8-pin (or 3x 8-pin) PCIe power cables
6. CRITICAL: Use separate PCIe cables from PSU, not daisy-chain

The RTX 3090 is a massive card. The Founders Edition is 313mm long and weighs 2.2kg. Use a GPU support bracket or 3D-printed sag preventer. Sagging puts stress on the PCIe slot and can cause intermittent contact issues over months.

Step 7: Cable Management and Fans

1. Route all cables behind motherboard tray
2. Install intake fans on front panel (blowing in)
3. Ensure exhaust through rear and top
4. Zip tie loose cables away from fans
5. Goal: clear airflow path from front intake → GPU → rear exhaust

---

Ubuntu Server Installation

Why Ubuntu Server (Not Desktop)

Ubuntu Desktop wastes 800MB-1.2GB RAM on GNOME and display services you will never use on a headless AI box. Ubuntu Server boots to a terminal, uses ~350MB RAM at idle, and includes everything you need for AI workloads.

Installation

# Download Ubuntu Server 24.04 LTS
# Flash to USB with balenaEtcher or:
sudo dd if=ubuntu-24.04-live-server-amd64.iso of=/dev/sdX bs=4M status=progress

# Boot from USB, follow installer:
# 1. Language: English
# 2. Network: Configure static IP (recommended for servers)
# 3. Storage: Use entire disk with LVM
# 4. Profile: Create your user account
# 5. SSH: Enable OpenSSH server (important!)
# 6. Snaps: Skip everything

Post-Install Configuration

# Update everything first
sudo apt update && sudo apt upgrade -y

# Install essential packages
sudo apt install -y build-essential git curl wget htop btop nvtop \
  net-tools openssh-server ufw fail2ban

# Set static IP (if not done during install)
sudo nano /etc/netplan/00-installer-config.yaml
# Example config:
# network:
#   ethernets:
#     enp3s0:
#       dhcp4: no
#       addresses: [192.168.1.100/24]
#       routes:
#         - to: default
#           via: 192.168.1.1
#       nameservers:
#         addresses: [1.1.1.1, 8.8.8.8]

sudo netplan apply

NVIDIA Driver Installation

This is where most people trip up. Do NOT install drivers from the NVIDIA website directly. Use Ubuntu's built-in package manager.

# Add NVIDIA driver PPA
sudo add-apt-repository ppa:graphics-drivers/ppa -y
sudo apt update

# Check recommended driver
ubuntu-drivers devices
# Look for "nvidia-driver-560" or similar marked "recommended"

# Install the recommended driver
sudo apt install -y nvidia-driver-560

# Reboot
sudo reboot

# Verify after reboot
nvidia-smi
# Should show RTX 3090 with 24576 MiB memory, driver 560.xx

Expected output:

+-----------------------------------------------------------------------------+
| NVIDIA-SMI 560.35.03    Driver Version: 560.35.03    CUDA Version: 12.6    |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence  | Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf  Pwr:Usage  |  Memory-Usage        | GPU-Util  Compute M.  |
|===============================+======================+======================|
|   0  NVIDIA GeForce ...  On   | 00000000:01:00.0 Off |                  N/A |
|  0%   32C    P8     22W       |      1MiB / 24576MiB |      0%      Default |
+-------------------------------+----------------------+----------------------+

Docker Installation

# Install Docker
curl -fsSL https://get.docker.com | sudo sh
sudo usermod -aG docker $USER

# Install NVIDIA Container Toolkit
curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | \
  sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg
curl -s -L https://nvidia.github.io/libnvidia-container/stable/deb/nvidia-container-toolkit.list | \
  sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#g' | \
  sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list
sudo apt update
sudo apt install -y nvidia-container-toolkit
sudo nvidia-ctk runtime configure --runtime=docker
sudo systemctl restart docker

# Verify GPU in Docker
docker run --rm --gpus all nvidia/cuda:12.6.0-base-ubuntu24.04 nvidia-smi

---

Ollama and Open WebUI Setup

Install Ollama

# One-line install
curl -fsSL https://ollama.com/install.sh | sh

# Enable and start service
sudo systemctl enable ollama
sudo systemctl start ollama

# Verify
ollama --version

Configure Ollama for Network Access

By default, Ollama only listens on localhost. For a server, you want it accessible from your LAN.

# Edit the systemd service
sudo systemctl edit ollama

# Add these lines between the comment blocks:
[Service]
Environment="OLLAMA_HOST=0.0.0.0:11434"
Environment="OLLAMA_ORIGINS=*"
Environment="OLLAMA_KEEP_ALIVE=24h"

# Reload and restart
sudo systemctl daemon-reload
sudo systemctl restart ollama

# Verify it is listening on all interfaces
ss -tlnp | grep 11434
# Should show 0.0.0.0:11434

The OLLAMA_KEEP_ALIVE=24h setting keeps models loaded in VRAM for 24 hours after last request. On a dedicated server, this means near-instant responses throughout the day.

Pull Models

# Essential models for a 24GB card
ollama pull llama3.2:7b          # 4.7GB - daily driver
ollama pull qwen2.5-coder:14b   # 9.1GB - code generation
ollama pull mistral:7b           # 4.1GB - fast general purpose
ollama pull llama3.1:70b-q2_K   # ~26GB - exceeds 24GB, offloads to RAM

# Check what is loaded
ollama ps

Deploy Open WebUI with Docker

For a complete walkthrough of Open WebUI features and configuration, see the Ollama + Open WebUI Docker setup guide.

# Run Open WebUI with GPU support
docker run -d \
  --name open-webui \
  --restart always \
  --gpus all \
  -p 3000:8080 \
  -v open-webui:/app/backend/data \
  -e OLLAMA_BASE_URL=http://host.docker.internal:11434 \
  --add-host=host.docker.internal:host-gateway \
  ghcr.io/open-webui/open-webui:main

# Access from any device on your network:
# http://192.168.1.100:3000

Auto-Start on Boot

# Ollama already configured as systemd service above
# Docker containers with --restart always auto-start

# Verify both survive a reboot
sudo reboot
# After reboot:
systemctl status ollama        # Should show "active (running)"
docker ps                      # Should show open-webui running

---

Tailscale Remote Access

Tailscale creates a WireGuard VPN mesh network. You install it on your server and your laptop/phone, and they can reach each other from anywhere without opening firewall ports or configuring port forwarding.

# Install Tailscale on the server
curl -fsSL https://tailscale.com/install.sh | sh
sudo tailscale up

# It prints a URL — open it in your browser to authenticate
# Your server gets a Tailscale IP like 100.x.y.z

# Now install Tailscale on your laptop/phone and authenticate
# Access your AI server from anywhere:
# http://100.x.y.z:3000  (Open WebUI)
# http://100.x.y.z:11434 (Ollama API directly)

Secure the Server

# Configure UFW firewall
sudo ufw default deny incoming
sudo ufw default allow outgoing
sudo ufw allow ssh
sudo ufw allow from 192.168.1.0/24 to any port 3000   # Open WebUI (LAN only)
sudo ufw allow from 192.168.1.0/24 to any port 11434  # Ollama API (LAN only)
sudo ufw enable

# Tailscale traffic bypasses UFW automatically
# So remote access via Tailscale still works

---

Power Consumption Analysis

We have not put a meter on this build, so rather than quote invented wattages, here is the arithmetic you can run against your own numbers — plus the vendor-rated ceilings that bound the answer.

The rated figures for the Tier 2 parts: NVIDIA lists the RTX 3090 at 350W board power, and AMD rates the Ryzen 7 5700X at a 65W TDP. Add board, drives and fans and roughly 450W is the sane upper bound for the whole machine under heavy inference — which is why the PSU guidance above sits where it does. That is a ceiling, not a typical draw.

Electricity cost is then just duty cycle. At the US average of $0.16/kWh, one watt held continuously for a month (730 hours) costs about $0.12:

Working the realistic case. Four hours a day at the 450W ceiling is 450 × 4 × 30.44 ÷ 1000 = 54.8 kWh/month, or $8.77. For the other twenty hours the box idles, and idle draw is the one number you have to get off your own meter — every 10W of idle draw adds 10 × 20 × 30.44 ÷ 1000 = 6.1 kWh, about $0.97/month. A machine that idles around 60W therefore lands near $14-15/month all-in. That is roughly one month of ChatGPT Plus ($20) for unlimited queries.

To get your own idle figure without buying anything, nvidia-smi --query-gpu=power.draw --format=csv reports the card's draw directly; a $25 wall meter covers the rest of the system.

Power Optimization

# Cap the board power limit: 350W -> 280W is a 20% cut
# Install nvidia-settings (even on headless server)
sudo apt install -y nvidia-settings

# Set power limit (default 350W, reduce to 280W)
sudo nvidia-smi -pl 280

# Make persistent across reboots
echo 'sudo nvidia-smi -pl 280' | sudo tee /etc/rc.local
sudo chmod +x /etc/rc.local

# Verify
nvidia-smi -q -d POWER | grep "Power Limit"
# Should show "Current Power Limit: 280.00 W"

Why this costs less speed than it looks like it should: a power cap throttles the GPU core, but token generation is bound by how fast weights stream out of GDDR6X, and memory bandwidth is set by the memory clock rather than the core power budget. Decode throughput is therefore far less sensitive to a power limit than prompt processing, which is genuinely compute-bound and takes the larger hit. Verify on your own card — nvidia-smi will tell you if the memory clock dropped too.

The saving side is straightforward arithmetic. Going from 350W to 280W is a 70W reduction. Sustained four hours a day that is 70 × 4 × 30.44 ÷ 1000 = 8.5 kWh/month, about $1.36 at $0.16/kWh, or roughly $16 a year. On a box that pins the GPU around the clock, the same 70W is 51.1 kWh/month — about $98 a year.

---

Noise Management

A 3090 running sustained inference is not a quiet card. We have not put a meter on one, and dBA figures vary so much by board partner, case and room that borrowed numbers are close to useless anyway — check the specific model you are buying against a reviewer who publishes noise-normalised measurements. What is safe to say: at stock fan curves it will be clearly audible on a desk during long generations, and the fix is almost entirely mechanical rather than acoustic.

Fan Curve Optimization

# Install fan control
sudo apt install -y python3-pip
pip3 install nvidia-ml-py3

# Create custom fan curve script
cat << 'SCRIPT' > ~/fan_curve.sh
#!/bin/bash
# Aggressive cooling at low RPM: quiet but effective
GPU_TEMP=$(nvidia-smi --query-gpu=temperature.gpu --format=csv,noheader)

if [ "$GPU_TEMP" -lt 40 ]; then
  nvidia-settings -a "[gpu:0]/GPUFanControlState=1" -a "[fan:0]/GPUTargetFanSpeed=25"
elif [ "$GPU_TEMP" -lt 60 ]; then
  nvidia-settings -a "[gpu:0]/GPUFanControlState=1" -a "[fan:0]/GPUTargetFanSpeed=40"
elif [ "$GPU_TEMP" -lt 75 ]; then
  nvidia-settings -a "[gpu:0]/GPUFanControlState=1" -a "[fan:0]/GPUTargetFanSpeed=60"
else
  nvidia-settings -a "[gpu:0]/GPUFanControlState=1" -a "[fan:0]/GPUTargetFanSpeed=85"
fi
SCRIPT
chmod +x ~/fan_curve.sh

# Run every 30 seconds via cron
(crontab -l 2>/dev/null; echo "* * * * * ~/fan_curve.sh") | crontab -
(crontab -l 2>/dev/null; echo "* * * * * sleep 30 && ~/fan_curve.sh") | crontab -

Physical Noise Reduction

1. Case selection matters most. The Fractal Meshify 2 Compact has sound-dampening panels while maintaining airflow. Avoid cases marketed as "silent" that choke airflow and cause thermal throttling.

2. Replace stock case fans with Noctua NF-A14 (140mm) or Arctic P14 PWM. These move the same air at half the noise of cheap bundled fans.

3. Rubber anti-vibration mounts on all fans and the GPU bracket. A $3 pack of rubber washers eliminates case resonance.

4. Location. A closet, basement shelf, or under-desk cabinet makes more noise difference than any hardware mod. Run a long Ethernet cable if needed.

---

Cloud GPU Cost Comparison

Here is the break-even math, assuming you build the Tier 2 system for $1,200.

Cloud GPU costs (early 2026):

Homelab monthly cost: electricity only. From the power section, four hours a day of heavy inference is about $8.77/month, plus roughly $0.97/month for every 10W the machine draws while idling the other twenty hours. A box idling near 60W lands at about $15/month, which is the figure the table below assumes — substitute your own idle draw if you have measured it.

Break-even timeline (hardware cost $1,200 ÷ monthly saving):

If you compare against Lambda or AWS (the services most people actually use), the homelab pays for itself in 11-16 months. After that, you are running AI inference essentially for free, minus electricity.

But the real value is not just cost. It is zero cold-start latency, no upload of private data, no usage caps, and the ability to run inference at 3 AM without worrying about a billing surprise.

---

Monitoring Your Server

# Install monitoring stack
sudo apt install -y nvtop btop

# GPU monitoring (real-time)
nvtop
# Shows GPU utilization, VRAM usage, temperature, power draw, per-process

# System monitoring
btop
# Shows CPU, RAM, disk, network in a beautiful TUI

# Create a simple health check script
cat << 'HEALTH' > ~/healthcheck.sh
#!/bin/bash
GPU_TEMP=$(nvidia-smi --query-gpu=temperature.gpu --format=csv,noheader)
GPU_MEM=$(nvidia-smi --query-gpu=memory.used --format=csv,noheader)
GPU_UTIL=$(nvidia-smi --query-gpu=utilization.gpu --format=csv,noheader)
OLLAMA_STATUS=$(systemctl is-active ollama)
WEBUI_STATUS=$(docker inspect -f '{{.State.Running}}' open-webui 2>/dev/null || echo "not found")

echo "=== AI Server Health ==="
echo "GPU Temp: ${GPU_TEMP}C"
echo "GPU Memory: ${GPU_MEM}"
echo "GPU Utilization: ${GPU_UTIL}"
echo "Ollama: ${OLLAMA_STATUS}"
echo "Open WebUI: ${WEBUI_STATUS}"
echo "Uptime: $(uptime -p)"
HEALTH
chmod +x ~/healthcheck.sh

---

Maintenance Schedule

Weekly:

• Check nvidia-smi for ECC errors (any non-zero value means hardware degradation)
• Update Ollama: sudo ollama update
• Check Docker container logs: docker logs open-webui --tail 50

Monthly:

• Run sudo apt update && sudo apt upgrade for security patches
• Check SSD health: sudo smartctl -a /dev/nvme0n1
• Compressed air blast to remove dust (open case, blow front to back)
• Review power consumption with a wall meter and feed it back into the cost math above

Quarterly:

• Repaste GPU thermal compound if temps have risen more than 5C from baseline
• Check all cable connections are secure
• Test UPS battery if you use one

---

Next Steps

Your homelab AI server is running. Here is where to go from here:

1. Optimize your model selection. The best GPUs for AI guide covers GPU-specific model recommendations, and the VRAM requirements guide helps you understand exactly what fits on your 24GB card.

2. Set up a proper web interface. Follow the Ollama + Open WebUI Docker setup guide for multi-user accounts, conversation history, and model switching.

3. Consider multi-GPU scaling. If 24GB VRAM is not enough, adding a second 3090 is far cheaper than buying a single 48GB card.

---

Frequently Asked Questions

Can a used RTX 3090 from a crypto miner be trusted for AI workloads?

Yes. Mining runs GPUs at constant, moderate temperatures with stable power draw. This is actually less stressful than gaming, which cycles temperatures rapidly. The main risk is GDDR6X memory degradation from prolonged high junction temperatures, but this is rare. Test for 30 minutes under load before committing.

Is 24GB VRAM enough for serious AI work in 2026?

For local inference, absolutely. 24GB runs every model up to 33B at high quality and 70B at reduced quality. The only scenario where 24GB falls short is fine-tuning large models or running 70B+ at full precision, both of which require 48GB+ regardless.

Should I buy one RTX 3090 or two RTX 3060 12GB cards?

One 3090. Two 3060s give you 24GB total but you cannot combine VRAM across cards for a single model in Ollama. The 3090 also has higher memory bandwidth (936 GB/s vs 360 GB/s) which directly affects token generation speed.

How loud is this build during inference?

Loud enough to notice on a desk at stock settings, and easy to tame. The three levers that matter are an airflow-first case, quality 140mm fans, and a 280W power limit on the GPU — less heat to move means lower fan RPM for the same temperature. We have not measured this build with a sound meter, so treat any specific dBA number you see for a 3090 as board- and case-specific. Moving the machine to a closet or cabinet does more than any hardware change.

Can I run this server on a UPS?

Yes, and you should. An APC Back-UPS 1500VA ($180) provides 8-12 minutes of runtime during a power outage, enough for a clean shutdown. Configure apcupsd on Ubuntu for automatic shutdown when battery hits 20%.

What about running this on a Raspberry Pi instead?

A Raspberry Pi 5 with 8GB RAM can run tiny models (1-3B) but nothing useful for production work. The Pi has no GPU acceleration for inference. It is good for learning, not for a real AI server. See the AI hardware requirements guide for minimum specs.

---

Conclusion

A homelab AI server built around a used RTX 3090 is the most cost-effective way to get serious about local AI. For roughly the price of 15 months of cloud GPU rental, you own hardware that runs 24/7 with no recurring fees beyond electricity.

The build itself takes an afternoon. The software stack (Ubuntu Server, Ollama, Open WebUI, Tailscale) takes another hour. After that, you have a private AI inference server accessible from anywhere, running any open-weight model that fits in 24GB of VRAM.

Start with the Tier 2 build if you can swing $1,200-1,500. The 64GB of system RAM gives you headroom for CPU offloading and future expansion. If you are budget-constrained, the Tier 1 build at $800-950 still runs circles around any cloud API for sustained daily use.

---

Need help choosing the right GPU for your build? Check our GPU comparison guide for a card-by-card comparison, or visit the hardware requirements guide for sizing your build to your workload.