Ollama on QNAP & TrueNAS: Turn Your NAS Into an AI Server

Short answer: yes — any x86 NAS with 8 GB of RAM can host Ollama in Docker, and the thing that decides how fast it feels is memory bandwidth, not core count. A Celeron- or Atom-class NAS is limited to 3B-class models. A Core i5 NAS is a comfortable single-user 7B box. Adding a 12 GB NVIDIA card to a TrueNAS SCALE machine is the step that makes it fast enough for a whole household to use daily.

Both platforms get there through Docker: QNAP via Container Station, TrueNAS SCALE via its Apps custom-YAML path. TrueNAS is the easier of the two for GPU passthrough, because underneath the UI you have a real Debian shell.

Quick Start: Ollama on Your NAS in 7 Minutes

If you have a QNAP with Container Station or TrueNAS SCALE with Apps enabled, this is the fastest path:

# SSH into the NAS
ssh admin@nas.local

# Pull and run Ollama (CPU mode, works on any x86 NAS)
docker run -d \
  --name ollama \
  --restart unless-stopped \
  -v /share/Container/ollama:/root/.ollama \
  -p 11434:11434 \
  ollama/ollama

# Pull a small model
docker exec ollama ollama pull llama3.2:3b

# Test from another machine on the LAN
curl http://nas.local:11434/api/generate \
  -d '{"model":"llama3.2:3b","prompt":"hello"}'

That is the bare minimum. The rest of this guide covers the part that turns "it works" into "the family uses it daily": GPU passthrough, Open WebUI, surviving firmware updates, and exposing it to clients without putting an unauthenticated model endpoint on the public internet.

Table of Contents

1. Is your NAS powerful enough?
2. How fast will it actually be?
3. How do you install Ollama on QNAP?
4. How do you install Ollama on TrueNAS SCALE?
5. How do you pass a GPU through on TrueNAS?
6. How do you add a ChatGPT-style interface?
7. How do you keep it running after a firmware update?
8. Should you expose it beyond the LAN?
9. What goes wrong on a NAS?
10. FAQ

---

Is Your NAS Powerful Enough to Run a Local LLM?

Token generation on a CPU is a memory-bandwidth problem. Every token requires reading the whole active model out of RAM, so the useful question is not "how many cores" but "how many GB per second can this box move".

Those bandwidth figures are arithmetic, not measurements — see the next section for how they are derived and what they do and do not tell you.

If your NAS is a 4-core ARM unit (Synology DS220+, QNAP TS-233 and similar), do not bother. Buy a small x86 box instead and let the NAS keep doing what it is good at. For options, see our best mini PC for Ollama roundup.

If your NAS is x86 with 8 GB or more, keep reading.

---

How Fast Will a NAS Actually Be?

Nobody publishes tok/s numbers for the exact NAS you own, so the honest thing to do is compute the ceiling yourself. Two pieces of arithmetic get you most of the way.

Step 1 — how big is the model on disk and in memory?

size at Q4_K_M (GB) ~= 0.6 x parameters in billions

So a 3B is roughly 1.8 GB, a 7B roughly 4.2 GB, an 8B roughly 4.8 GB, a 13B roughly 7.8 GB. Add roughly 1-2 GB more for the KV cache and runtime at ordinary context lengths.

Step 2 — what is the throughput ceiling?

tokens/sec ceiling = memory bandwidth (GB/s) / model size in memory (GB)

Dual-channel DDR4-3200 gives 3200 MT/s x 8 bytes x 2 channels = 51.2 GB/s. An RTX 3060 12 GB has a 192-bit bus at 15 Gbps, which is 192 / 8 x 15 = 360 GB/s. Divide and you get:

Read those as "no faster than", never as a forecast. The ceiling assumes the hardware saturates its memory bus and nothing else is competing for it. On a NAS, neither is true: a low-power Celeron cannot drive its own memory controller flat out, and ZFS ARC, SMB and a Plex scan are all pulling on the same bus. Real output lands well below the ceiling — the gap is largest on weak CPUs and smallest on a dedicated GPU.

Two things follow immediately. First, a model that does not fit in VRAM falls back to the system-RAM row of that table, which is why a 12 GB card and a 7B model is such a common pairing. Second, the fix for "too slow" is almost always a smaller model or a faster memory path, not more cores. The same arithmetic is explained at length in our GPU memory bandwidth guide.

To measure your own box rather than estimate it, run a real completion with timing on:

docker exec -it ollama ollama run qwen2.5:7b-instruct-q4_K_M \
  "Write a haiku about backups" --verbose

The --verbose flag prints eval rate at the end. That number is yours, for your config, and it beats any figure published for different hardware.

---

How Do You Install Ollama on QNAP Container Station?

QNAP's Container Station is essentially a managed Docker. The CLI is the fastest path; the GUI works but is harder to script.

Step 1: Install Container Station

App Center → Container Station → Install. Reboot if prompted; some firmware versions require it.

Step 2: Enable SSH and connect

Control Panel → Network & File Services → Telnet/SSH → Allow SSH. Default port 22. Connect as the admin user:

ssh admin@your-qnap.local

Step 3: Create persistent storage

Use the Container shared folder created by Container Station, with a subfolder for Ollama:

mkdir -p /share/Container/ollama
chmod 755 /share/Container/ollama

Step 4: Run Ollama

docker run -d \
  --name ollama \
  --restart unless-stopped \
  -v /share/Container/ollama:/root/.ollama \
  -p 11434:11434 \
  -e OLLAMA_NUM_PARALLEL=2 \
  -e OLLAMA_KEEP_ALIVE=24h \
  -e OLLAMA_HOST=0.0.0.0:11434 \
  ollama/ollama

OLLAMA_HOST=0.0.0.0 is the important bit. Without it Ollama binds only to the container's localhost and no LAN client can reach it.

Step 5: Pull a model sized to your CPU

# Celeron / Atom: keep it small
docker exec -it ollama ollama pull llama3.2:3b

# Core i5+ / Ryzen: 7B is realistic
docker exec -it ollama ollama pull qwen2.5:7b-instruct-q4_K_M

# Test
docker exec -it ollama ollama run llama3.2:3b "Say hello in 10 words"

Step 6: Verify LAN access

From any other device on the network:

curl http://your-qnap.local:11434/api/tags

You should see JSON listing the model you pulled.

---

How Do You Install Ollama on TrueNAS SCALE?

TrueNAS SCALE is Debian-based and has had first-class Docker support since the Electric Eel release. The path is cleaner than QNAP because there is a real Linux shell underneath. The official SCALE documentation is the reference if your version's UI differs from the steps below.

Step 1: Enable Apps and Docker

Apps → Configure Pool. Pick a pool with at least 50 GB free.

Step 2: Create a dataset for Ollama

Storage → Pools → tank → Add Dataset. Name: ollama. ZFS settings: recordsize=128K, atime=off. Then:

sudo chown -R apps:apps /mnt/tank/ollama

Step 3: Deploy via custom-app YAML

Apps → Discover Apps → Custom App → Install via YAML:

services:
  ollama:
    image: ollama/ollama:latest
    container_name: ollama
    restart: unless-stopped
    ports:
      - "11434:11434"
    volumes:
      - /mnt/tank/ollama:/root/.ollama
    environment:
      - OLLAMA_HOST=0.0.0.0:11434
      - OLLAMA_NUM_PARALLEL=2
      - OLLAMA_KEEP_ALIVE=24h

Step 4: Confirm and pull a model

sudo docker exec -it ollama ollama pull qwen2.5:7b-instruct-q4_K_M

That completes the CPU-only setup. For the bandwidth jump described earlier, you want GPU passthrough.

---

How Do You Pass a GPU Through on TrueNAS?

A 12 GB NVIDIA card moves a 7B model from the ~51 GB/s system-RAM row of the table above to the ~360 GB/s VRAM row. That is the whole reason to do it. NVIDIA cards work through the official container toolkit; AMD works through ROCm but with noticeably more friction, so NVIDIA is the pragmatic pick for a NAS.

Used-card prices move constantly, so check current listings rather than trusting any figure in an article — our used GPU buying guide covers what to look for.

Step 1: Fit the card

Power down. Add the GPU to the PCIe slot. Three things to verify before you buy, because many pre-built NAS chassis fail at least one: a real x16 electrical link (some units expose x4), a spare 8-pin PCIe power connector on the PSU, and physical clearance for a dual-slot card.

Step 2: Verify TrueNAS sees the GPU

sudo lspci | grep -i nvidia
# Should show: ... NVIDIA Corporation GA106 [GeForce RTX 3060 12GB]

Step 3: Install NVIDIA drivers

In SCALE 24.10+ this is a checkbox. System Settings → Advanced → Isolated GPU Devices → confirm the GPU is NOT isolated (SCALE needs to use it). Then System → General Settings → enable the NVIDIA driver.

Older SCALE versions may need:

sudo apt update
sudo apt install nvidia-driver nvidia-container-toolkit
sudo systemctl restart docker
nvidia-smi   # should now print GPU info

The NVIDIA Container Toolkit install guide is the canonical source when the packaged version misbehaves.

Step 4: Update the compose file

services:
  ollama:
    image: ollama/ollama:latest
    container_name: ollama
    restart: unless-stopped
    ports:
      - "11434:11434"
    volumes:
      - /mnt/tank/ollama:/root/.ollama
    environment:
      - OLLAMA_HOST=0.0.0.0:11434
      - OLLAMA_NUM_PARALLEL=2
      - OLLAMA_KEEP_ALIVE=24h
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: 1
              capabilities: [gpu]

Redeploy, then confirm the model is actually on the GPU:

sudo docker exec -it ollama ollama ps
# PROCESSOR should read 100% GPU, and SIZE should be under your VRAM

If ollama ps reports 100% CPU, the container is not seeing the card — usually a driver-version mismatch or a missing nvidia-container-toolkit. If it reports a split, the model did not fit and part of it is in system RAM, which drops you back to the slow row of the bandwidth table. Pick a smaller model or a tighter quant.

---

How Do You Add a ChatGPT-Style Interface?

The raw API is fine for scripts, but nobody in the household will curl a JSON endpoint. Add Open WebUI.

services:
  open-webui:
    image: ghcr.io/open-webui/open-webui:main
    container_name: open-webui
    restart: unless-stopped
    ports:
      - "3000:8080"
    volumes:
      - /mnt/tank/open-webui:/app/backend/data
    environment:
      - OLLAMA_BASE_URL=http://ollama:11434
      - WEBUI_AUTH=true
    depends_on:
      - ollama

On QNAP, swap /mnt/tank/open-webui for /share/Container/open-webui and create the directory first.

The network-mode gotcha

If open-webui and ollama live in separate Container Station projects they cannot resolve each other by container name. Use http://host.docker.internal:11434 or the NAS's LAN IP. The cleaner fix is to keep both services in one compose file so they share a Docker network.

First login

Browse to http://your-nas.local:3000. The first account created becomes the admin. Turn off open registration in Settings → Authentication so only invited users can sign up. Our Open WebUI complete guide covers multi-user RAG and per-user document libraries.

---

How Do You Keep It Running After a Firmware Update?

Uptime is the whole reason to host this on a NAS instead of a laptop. Make sure the containers come back after firmware updates and power events.

QNAP

# Verify autostart
docker inspect ollama | grep -A 1 RestartPolicy
# Should show: "Name": "unless-stopped"

# Test by simulating a crash
docker kill ollama
# Wait 5 seconds
docker ps | grep ollama   # should be running again

QNAP firmware updates sometimes restart Container Station, which restarts Docker. unless-stopped handles that.

TrueNAS SCALE

SCALE's Apps engine restarts containers on its own. Major version upgrades occasionally require custom YAML apps to be re-applied, so keep the YAML in a git repo and re-deploy after every upgrade rather than reconstructing it from memory.

Health monitoring

A cron job that pings the model catches the silent failures:

# /share/Container/ollama-healthcheck.sh
#!/bin/bash
RESPONSE=$(curl -s -m 10 http://localhost:11434/api/tags || echo "FAIL")
if [[ "$RESPONSE" == "FAIL" ]]; then
  docker restart ollama
  echo "$(date): Ollama restarted" >> /share/Container/ollama-restart.log
fi

Schedule it every 5 minutes from the NAS's cron UI.

---

Should You Expose It Beyond the LAN?

Decide this deliberately. For most readers the right answer is LAN-only or VPN.

LAN-only (the sane default)

That is what the configs above give you. http://nas.local:11434 answers from anywhere on your home network and nowhere else.

VPN access (for remote)

WireGuard or Tailscale. Both NAS platforms run Tailscale natively. Install it, accept the magic-DNS hostname, and a phone on cellular reaches http://nas.tailnet:3000 without anything being exposed publicly.

Public exposure (not recommended)

If you genuinely need it, do not port-forward 11434. Put a reverse proxy (Caddy or nginx) in front, terminate TLS with a real certificate, require auth, and rate-limit. Ollama's API has no authentication of its own — treat it as infrastructure that must be fronted. The patterns in our securing Ollama guide apply unchanged on a NAS, and Ollama production deployment covers the reverse-proxy side.

---

What Goes Wrong on a NAS?

1. Firmware updates resetting the Docker network. QNAP upgrades can revert Container Station's bridge or firewall rules. After every QTS update, re-check that http://nas.local:11434 still answers from a LAN client before assuming Ollama itself broke.

2. A tiny ZFS recordsize on the model dataset. The 128K default is right for model weights. Set it to 4K because "my apps are small" and a 4.8 GB model file is stored as roughly 1.2 million records instead of about 37,000 — an enormous amount of extra metadata work on every cold load. Use 128K or 1M for the Ollama dataset.

3. atime left on. Default datasets update access time, so every model load triggers a metadata write. Set atime=off on the Ollama dataset; there is no reason to track read times on model weights.

4. Container Station's default bridge network. QNAP's bridge adds an iptables NAT hop. If a capable CPU still looks CPU-bound, try host networking for the Ollama container.

5. SCALE Apps engine resets on major upgrades. Cobia → Dragonfish-class jumps have wiped custom apps before. Keep the YAML in version control.

6. Docker permissions on QNAP. docker as the ordinary admin user often fails with permission errors. Prefix with sudo where allowed, or use Container Station's web SSH, which has the right privileges.

7. Airflow over the PCIe slot. Pre-built NAS chassis are designed for drives, not for a card drawing its full rated power for hours. NVIDIA rates the RTX 3060 12 GB at 170 W. A chassis with no airflow across that slot will heat-soak, and the first thing you notice is usually storage errors rather than GPU errors. Add a case fan before running daily AI workloads.

8. Assuming inference is free to run. Do the arithmetic instead: a part rated at 170 W held flat for a year is 170 W x 8,760 h = 1,489 kWh. At the US residential average published by the EIA, that is a real line item. Inference is bursty, so multiply by your actual duty cycle and your own rate — but do multiply.

---

Frequently Asked Questions

Can I run AI on a 4-core ARM Synology like a DS220+?

Technically yes, practically no. Those units pair slow ARM cores with single-channel low-clocked memory, so both halves of the bandwidth equation work against you, and most run so little RAM that even a 3B is a squeeze. A small x86 mini PC is the better answer.

Does ZFS on QuTS hero or TrueNAS SCALE help AI performance?

It helps cold starts, not inference. The ARC cache keeps recently read model files in RAM, so a reload can skip the disk entirely. Once weights are resident in RAM or VRAM, the filesystem is out of the loop and only the bandwidth arithmetic matters.

Will an RTX 3060 work in any NAS?

Three checks, and off-the-shelf units routinely fail one: a PCIe x16 slot with a real x16 electrical link, a PSU with a spare 8-pin GPU power connector, and physical clearance for a dual-slot card. Confirm all three against your specific chassis before buying a card.

Can I run multiple GPUs in TrueNAS SCALE for bigger models?

Yes — set count: 2 in the YAML and Ollama will split a model across the pair when neither card holds it alone. Splitting adds cross-GPU traffic that a single card does not pay, so one 24 GB card beats two 12 GB cards for any model that fits the larger one.

How does this compare to using a Mac as the AI server?

Apple Silicon's unified memory gives a Mac far more bandwidth than NAS system RAM and lets large models fit without a discrete card, in a quiet low-power box. A NAS plus a used GPU is usually cheaper for 7B-13B work, especially if you already own the NAS. If AI is the primary workload and you are starting from nothing, our Mac local AI setup guide covers the alternative.

Will running AI slow down file serving?

They contend less than you would expect: inference on a GPU is bound by VRAM bandwidth while SMB and NFS are bound by network and disks. CPU-only inference is the case to watch, because then both are pulling on the same memory bus. Avoid scheduling heavy write workloads such as Time Machine backups against the same pool during sustained inference.

Can the NAS GPU do Ollama and Plex transcoding at once?

Plex hardware transcoding uses NVENC, a separate engine from the CUDA cores Ollama uses, so concurrent operation is possible in principle. In practice they share VRAM and memory bandwidth, so a sustained inference load can starve a transcode. If both matter, use two GPUs or pick one.

What about Unraid instead of TrueNAS?

Unraid has mature GPU passthrough and a large container community; the setup is the same Docker plus nvidia-container-toolkit pattern, and the bandwidth arithmetic is identical. Pick whichever platform you already prefer for storage.

---

Conclusion

A NAS earns its place by running quietly, all the time, with redundant storage. Ollama and Open WebUI turn it into the always-on private AI endpoint for a household or homelab. On a Celeron unit that means a 3B toy. On a real x86 NAS it means a usable single-user 7B. Add a 12 GB GPU and the bandwidth arithmetic moves by roughly 7x, which is the difference between "technically working" and "people actually use it".

Ready to go further? Ollama production deployment and AI gateway with LiteLLM cover putting the NAS behind a real reverse proxy with auth and routing, and AI on 16GB RAM helps you match a model to the memory you actually have.

Want more weird-but-practical local AI deployments delivered weekly? Subscribe to the LocalAIMaster newsletter.