AI on Steam Deck: Run Local LLMs on Valve Handheld (2026 Guide)

Published on April 23, 2026 - 18 min read

Quick Start: Run Phi-3 on Your Steam Deck in 4 Minutes

The Steam Deck ships with an AMD Van Gogh APU (Zen 2 CPU + RDNA 2 iGPU) and 16GB of LPDDR5 unified memory. That is enough to run quantized 7B models at usable speeds. Boot into Desktop Mode, open Konsole, and run:

1. passwd - set a sudo password (SteamOS does not ship with one)
2. curl -fsSL https://ollama.com/install.sh | sh - installer auto-detects ROCm absence and falls back to CPU+Vulkan
3. ollama pull phi3:mini - 2.2GB Q4 quant, fits easily in RAM
4. ollama run phi3:mini - start chatting

That is it. You now have a fully offline LLM running on a 7-inch handheld that fits in a backpack pocket.

---

What this guide covers:

• Installing Ollama on SteamOS without breaking the immutable root filesystem
• Working around the read-only /usr partition that survives every system update
• Throughput ceilings for Phi-3 Mini, Llama 3.2 3B, Mistral 7B and Gemma 2 9B, derived from the Deck's memory bandwidth
• Battery, thermals, and TDP tuning for sustained inference
• Switching between gaming and AI workloads without conflicts

A note on the numbers below. We do not own a Steam Deck and nothing in this guide was timed on one. Where you see a tokens-per-second figure it is an arithmetic ceiling — the Deck's published memory bandwidth divided by the size of the model's weights — clearly labelled as such. Ceilings are the honest, useful half of the question: they tell you what the hardware physically cannot exceed, which is exactly what you need when deciding whether a model is worth pulling onto a handheld.

The Steam Deck is an underrated AI device. Most reviews compare it to the ROG Ally and Legion Go for gaming and stop there, even though Valve's handheld has 16GB of unified memory shared between CPU and GPU - the same architectural trick that makes Apple Silicon punch above its weight. If you already own a Deck, you own a portable, fanless-when-idle LLM workstation.

For broader hardware decisions, pair this guide with the AI hardware requirements deep dive and the AMD vs NVIDIA vs Intel AI GPU buyer's guide. For privacy-sensitive deployments where the handheld never touches Wi-Fi, the air-gapped AI deployment guide covers the full offline workflow.

Table of Contents

1. Why the Steam Deck is Surprisingly Good for AI
2. Hardware Reality Check
3. SteamOS Filesystem Quirks
4. Installing Ollama Correctly
5. Model Selection for 16GB Unified Memory
6. Throughput Ceilings Across All Major Models
7. TDP, Battery, and Thermal Tuning
8. Open WebUI on the Deck
9. Switching Between Gaming and AI
10. Pitfalls and Fixes

---

Why the Steam Deck is Surprisingly Good for AI

Three architectural facts make the Deck competitive for local inference at its price point.

Unified memory. The Van Gogh APU shares 16GB of LPDDR5-5500 between CPU and GPU. Unlike a discrete-GPU laptop where the iGPU is starved by an 8GB or 12GB VRAM ceiling, the Deck can dedicate up to 12GB to a model and still leave 4GB for SteamOS and Plasma. That is the same trick Apple uses on M-series Macs.

Decent memory bandwidth. LPDDR5-5500 across a 128-bit bus works out to 5,500 MT/s × 16 bytes ≈ 88 GB/s. Apple publishes 100 GB/s for the M2, so the LCD Deck sits within about 12% of an M2 Air on the one spec that governs token generation, and well ahead of any dual-channel DDR4 desktop. Decoding a quantized model is bandwidth-bound, so this matters more than raw FLOPS.

Thermal headroom. The Deck's 15W default TDP can be pushed to 20W in handheld mode and effectively uncapped (45W package power) when docked with active cooling. Because decode is bandwidth-bound rather than compute-bound, inference does not need the top of that range the way a game does.

The catch: AMD does not officially support ROCm on Van Gogh (RDNA 2 mobile). You will run inference on CPU with Vulkan compute as a partial accelerator, not on GPU compute the way you would on an RX 7900 XTX. That is less fatal than it sounds, because the bottleneck is the memory bus rather than the shader array — but it does mean four Zen 2 cores have to drive that bus, and CPU-side inference does not get close to saturating it. Expect real output meaningfully below the ceilings in the table further down.

Hardware Reality Check

The OLED model runs LPDDR5-6400 rather than 5500, which on the same 128-bit bus is 102.4 GB/s against 88 GB/s — 16% more bandwidth, and therefore a 16% higher arithmetic ceiling on token generation. Whether you see all 16% depends on how close the CPU can get to saturating the bus in the first place, which on four Zen 2 cores is "not very". The OLED also carries a larger 50 Wh battery against the LCD's 40 Wh, a flat 25% more runtime at the same power draw. Not a reason to upgrade if you have an LCD, but if you are buying new for AI specifically, OLED wins.

The 64GB eMMC base model is a non-starter. A single 7B Q4 model is 4.4GB. You need at least the 256GB SSD model, and 512GB or 1TB is more practical once you start collecting Phi, Llama, Mistral, and embedding models.

SteamOS Filesystem Quirks

SteamOS 3 is built on an immutable Arch Linux base. The root filesystem (/usr, /etc, /var partially) is read-only and gets atomically replaced on every system update. Anything you install via pacman -S will get wiped the next time Valve pushes an update. This trips up almost everyone who tries to install AI tools the conventional way.

Three workable approaches:

1. Install into /home (recommended). /home is on a separate writable partition. The Ollama installer puts binaries in /usr/local/bin by default, which gets wiped. Override the install location to ~/.local/bin and add it to PATH.

2. Use Distrobox. Run a full Arch or Ubuntu container inside SteamOS with persistent storage. Heavy but bulletproof. Recommended if you want a complete dev environment.

3. Disable the read-only filesystem. Run sudo steamos-readonly disable. Works, but updates will conflict and you may need to re-disable after each major SteamOS release. Not recommended for a daily-driver Deck.

Approach 1 is the right default for Ollama itself. Reach for approach 2 when you need a build toolchain for Python-based tools (llama.cpp, embedding model fine-tuning, and so on).

Installing Ollama Correctly

Boot into Desktop Mode (Power button > Switch to Desktop). Open Konsole.

# Set sudo password if you have not already
passwd

# Make /home/deck/.local/bin exist and be on PATH
mkdir -p ~/.local/bin
echo 'export PATH="$HOME/.local/bin:$PATH"' >> ~/.bashrc
echo 'export OLLAMA_MODELS="$HOME/.ollama/models"' >> ~/.bashrc
source ~/.bashrc

# Download Ollama binary directly (skip the systemd unit installation)
curl -L https://ollama.com/download/ollama-linux-amd64 -o ~/.local/bin/ollama
chmod +x ~/.local/bin/ollama

# Verify
ollama --version

This puts Ollama in your home directory. SteamOS updates will not touch it.

To run it as a background service that starts with Desktop Mode, create a user systemd unit:

mkdir -p ~/.config/systemd/user
cat > ~/.config/systemd/user/ollama.service <<'EOF'
[Unit]
Description=Ollama Service (user)
After=network-online.target

[Service]
Type=simple
ExecStart=%h/.local/bin/ollama serve
Environment="OLLAMA_MODELS=%h/.ollama/models"
Environment="OLLAMA_HOST=127.0.0.1:11434"
Restart=on-failure
RestartSec=3

[Install]
WantedBy=default.target
EOF

systemctl --user daemon-reload
systemctl --user enable --now ollama.service

Now pull your first model:

ollama pull phi3:mini       # 2.2 GB, fastest
ollama pull llama3.2:3b     # 2.0 GB, best quality-to-speed ratio
ollama pull mistral:7b      # 4.1 GB, GPT-3.5-class quality

Models go into ~/.ollama/models/. If you have an SD card and want to keep models off the internal SSD, mount the card and symlink:

# Assuming your SD card is mounted at /run/media/deck/SDCARD
mkdir -p /run/media/deck/SDCARD/ollama-models
mv ~/.ollama/models /run/media/deck/SDCARD/ollama-models/
ln -s /run/media/deck/SDCARD/ollama-models ~/.ollama/models

Note that SD card read speeds (around 100 MB/s for a class A2 card) are far slower than the internal NVMe, so first-load time goes up in proportion — a 4.1GB model read at 100 MB/s takes about 41 seconds just to reach memory. Inference speed once loaded is unaffected, because the entire model lives in RAM.

Model Selection for 16GB Unified Memory

The Deck has 16GB total, shared between CPU and GPU. SteamOS Desktop Mode, Plasma and a browser take a few GB between them before you load anything — check yours with free -h rather than trusting a number from a guide, since it depends heavily on how many Firefox tabs you keep open. Budget on the order of 11-12GB for a model plus its context and you will not be surprised.

Phi-3 Mini is the answer for most people. It is fast enough to feel interactive on a handheld and Microsoft trained it specifically for tool use and structured output. Llama 3.2 3B beats it on general writing quality. Mistral 7B is the upgrade pick when you need actual reasoning depth and can live with a noticeably slower first token.

Throughput Ceilings Across All Major Models

These are not stopwatch numbers and we have not run them on a Deck. They are the arithmetic ceiling: decoding one token requires reading the entire weight set out of memory once, so the hard limit is memory bandwidth divided by model file size. Nothing — no quantization trick, no runtime flag — gets you past these figures on this hardware.

Bandwidth: 88 GB/s on the LCD Deck (LPDDR5-5500 on a 128-bit bus), 102.4 GB/s on the OLED (LPDDR5-6400). File sizes are the published Q4_K_M sizes from the Ollama library.

Read the ceiling as a speed limit, not a speedometer. Two things push real output well below it on a Deck specifically. First, inference here runs on four Zen 2 cores rather than a GPU with ROCm, and a CPU-side decode path does not come close to saturating the memory bus. Second, sampling, the growing KV cache and framework overhead all cost time the arithmetic ignores. The practical value of the table is the ordering and the ratios: a 9B will never be much more than a third the speed of a 3B on this hardware, because it is roughly three times the bytes per token.

What the ceiling does settle cleanly is where the wall is. Anything at or below about 2.2 GB has ceiling headroom to stay interactive on a handheld. Above roughly 5 GB the ceiling itself is already in single-digit-to-teens territory before any real-world overhead, which is why 13B-class models are not a serious proposition on a Deck no matter how you quantize them.

TDP, Battery, and Thermal Tuning

Token generation is bandwidth-bound, and the TDP slider does not change memory bandwidth. That is the whole reason TDP tuning is worth doing on a Deck: past the point where the cores can keep the memory controller fed, extra watts buy you very little throughput and cost you runtime directly.

Battery arithmetic makes the trade concrete. The TDP slider caps APU package power, so the runtime it buys is just capacity divided by draw. On the OLED's 50 Wh pack, 12W is 50 ÷ 12 ≈ 4.2 hours of APU budget; 15W is 3.3 hours. On the LCD's 40 Wh pack the same figures are 3.3 and 2.7 hours. Real runtime is shorter because the display, Wi-Fi and storage draw on top of the APU cap — treat these as ceilings, and dropping from 15W to 10W is a third more runtime by construction.

Hold the QAM button (the three-dot button) to bring up the performance overlay in Game Mode, or use PowerStation from the Discover store in Desktop Mode.

Start at the low end and only raise it if generation feels slow — the arithmetic above says the extra watts mostly buy prompt-processing speed, not token throughput.

Set the GPU clock to 800 MHz. RDNA 2 idle clocks save power without affecting CPU-bound inference. Disable the SMT-aware scheduler tweak if you see Konsole report only 4 cores at full load - Ollama benefits from all 8 threads.

For thermal monitoring while inferencing, run watch -n 1 sensors in a second terminal. The Deck's APU thermal limit is 100C, and a TDP-capped inference workload should sit far below it — if you are near the limit, drop the TDP.

Open WebUI on the Deck

A ChatGPT-style interface running locally on the Deck is genuinely useful when you dock it as a desktop. Open WebUI runs in a single Docker container.

# Install Docker (Distrobox approach is cleaner; this works for quick setup)
sudo steamos-readonly disable
sudo pacman-key --init
sudo pacman -Sy docker
sudo systemctl enable --now docker
sudo usermod -aG docker deck
# Re-enable readonly after install:
sudo steamos-readonly enable

# Pull and run Open WebUI
docker run -d -p 3000:8080 \
  --add-host=host.docker.internal:host-gateway \
  -v open-webui:/app/backend/data \
  --name open-webui \
  --restart always \
  ghcr.io/open-webui/open-webui:main

Visit http://localhost:3000 in Firefox. Add Ollama at http://host.docker.internal:11434 in the admin panel. You now have a polished web UI for chatting, document RAG, and image input on a model that lives entirely on your Deck.

For RAG over your own documents on the Deck, see the Open WebUI setup guide and the best Ollama models for 16GB systems.

Switching Between Gaming and AI

The Deck's 16GB cannot host a AAA game and a 7B model at the same time. But you do not have to choose at install time - just at runtime.

# Stop Ollama before launching demanding games
systemctl --user stop ollama.service

# Restart it for a coding session later
systemctl --user start ollama.service

For lighter games (Stardew Valley, Hades, Slay the Spire) you can usually keep Phi-3 Mini loaded - the weights are 2.2GB and the runtime adds a little on top, which still leaves the bulk of 16GB for SteamOS and a 2D game. The memory budget is the thing to check, not the CPU: a resident model that is not generating costs you RAM, not cycles.

For modding and AI together (asking the model to write Lua for Stardew Valley scripts while you play), this dual-stack capability is genuinely useful.

Pitfalls and Fixes

SteamOS update wiped my install. You installed into /usr instead of /home. Re-install with the binary download method into ~/.local/bin as shown above.

ollama: command not found. PATH not exported. Run echo 'export PATH="$HOME/.local/bin:$PATH"' >> ~/.bashrc && source ~/.bashrc.

Out of memory pulling Mistral 7B. Likely you have a heavy browser tab eating RAM. Close Firefox tabs or run ollama pull mistral:7b-instruct-q4_K_S for the smaller quant.

Token generation pauses every 30 seconds. SteamOS aggressive memory compaction kicks in. Disable it temporarily: echo 0 | sudo tee /proc/sys/vm/compaction_proactiveness.

Fan ramps to max during inference. TDP is too high. Drop to 12W in PowerStation - less heat to shed means a lower fan curve for the same temperature.

Model pulls are slow on Steam Deck Wi-Fi. Download time is just file size over link speed: a 4.1GB model is about 33 gigabits, so a link that actually sustains 400 Mbps needs roughly 82 seconds, and a congested 100 Mbps link needs five and a half minutes. A USB-C ethernet adapter when docked removes the variable entirely.

ROCm install instructions online do not work. AMD does not support ROCm on Van Gogh (RDNA 2 mobile). Stop trying. Vulkan compute via llama.cpp is the closest you get to GPU offload, and Ollama already uses it where helpful.

---

Frequently Asked Questions

Q: Can the Steam Deck actually replace a laptop for AI work?

A: For inference at small model sizes, it is in the same class as a thin-and-light without a discrete GPU - the LCD Deck's 88 GB/s of memory bandwidth is within about 12% of the 100 GB/s Apple publishes for the M2, and bandwidth is what governs token generation. For training or fine-tuning, no - you need a discrete GPU with CUDA or ROCm support.

Q: Does it work in Game Mode or only Desktop Mode?

A: Ollama runs as a background service that works in both modes. You cannot launch a chat window in Game Mode without leaving it, but you can use the API endpoint at localhost:11434 from any app you add to Steam, including a Chromium shortcut to Open WebUI.

Q: How much battery does an LLM session actually drain?

A: Work it out from the TDP cap, because that is what the slider controls. Thirty minutes at 12W is 6 Wh of APU draw, which is 12% of the OLED's 50 Wh pack and 15% of the LCD's 40 Wh. At 15W it is 7.5 Wh, so 15% and 19% respectively. Those are floors - the screen, Wi-Fi and storage draw on top of the APU budget, so plan on meaningfully more.

Q: Is the Steam Deck OLED worth the upgrade for AI specifically?

A: The OLED's LPDDR5-6400 gives 102.4 GB/s against the LCD's 88 GB/s, a 16% higher arithmetic ceiling on token generation, and its 50 Wh battery is 25% larger than the LCD's 40 Wh. Add a much better screen for reading model output. If you already own an LCD model, the upgrade is hard to justify on AI alone. If you are buying new and AI is a primary use case, OLED is the right pick.

Q: Will SteamOS updates break my Ollama install?

A: Not if you follow the home-directory install method in this guide. Updates only replace the immutable root partition. Anything in /home survives untouched.

Q: Can I use the AMD GPU for inference?

A: Partially. Ollama uses Vulkan compute on RDNA 2 for some operations. Full ROCm acceleration is not supported by AMD on Van Gogh hardware. Do not expect a transformation either way: CPU and iGPU share the same 88 GB/s memory bus, and that bus is the ceiling on token generation regardless of which one is doing the arithmetic.

Q: What is the most useful real-world workflow on a Deck for AI?

A: Travel coding assistant. Pair Phi-3 Mini with Continue.dev configured for Ollama, code on the Deck in VS Code (Distrobox or Flatpak), and you have a fully offline coding setup that fits in a jacket pocket.

Q: Can I run Stable Diffusion on the Deck?

A: SD 1.5 at 512x512 fits in memory and will run on the Vulkan backend, but slowly - diffusion is compute-bound rather than bandwidth-bound, and 8 RDNA 2 compute units at a 15W package cap is not much compute. SDXL is impractical on memory grounds. For image generation, see the Stable Diffusion local setup guide for desktop-class options.

---

Conclusion

The Steam Deck is a 2022 handheld console with a 4-year-old AMD APU. It also happens to be one of the most cost-effective ways to own a portable, fully-offline LLM workstation. For roughly $400 used, you get 16GB of unified memory, an 8-thread APU, a 7-inch screen, a battery big enough for multi-hour sessions at a low TDP cap, and enough memory bandwidth - 88 GB/s, against the 100 GB/s Apple publishes for the M2 - to make small quantized models genuinely usable.

The trick is respecting SteamOS's immutable design: keep your install in /home, use systemd user units, and treat the read-only root partition as a feature, not a bug. Once the install survives system updates, you have a Linux box that fits in a backpack pocket and runs LLMs anywhere.

Next steps: pick the right model for your workflow with the best Ollama models guide, then drop the Deck into a dock and add Open WebUI for a polished interface.

---

Want more handheld and edge AI guides? Join the LocalAIMaster newsletter for weekly hardware guides and model breakdowns.