Air-Gapped AI Deployment: Install Ollama With No Internet

Published April 23, 2026 · Updated August 23, 2026 · 22 min read

Short answer: yes, Ollama and Open WebUI run with zero network access, and the whole job is four steps. Build a bundle of binaries, models and packages on an internet-connected machine; hash and sign it; carry it across on write-protected media; then install from those local files instead of the usual curl | sh script. Everything after that is verification discipline and an update cadence you actually keep.

The part that trips people up is not Ollama. It is every other tool in the stack quietly reaching for the internet on first run - Open WebUI downloading an embedding model, ComfyUI cloning custom nodes from GitHub, n8n pulling community nodes from npm. Those failures happen after you have already disconnected, which is the worst time to find them.

---

Is your network actually air-gapped, or just firewalled?

"Air-gapped" gets used loosely. For this guide the definition is: no routable network path between the AI environment and any internet-connected system at any time. Not "we use a firewall." Not "we use a VPN." Physically isolated, with sneakernet (USB or a one-way data diode) as the only transfer mechanism.

If your environment matches that - SCIF, classified network, regulated medical infrastructure, OT/SCADA isolation, financial trading vault, or a customer's internal no-egress enclave - keep reading. If you have a private network with controlled internet access, the local AI privacy guide covers a lighter posture that gets most of the same protection for a fraction of the operational cost.

What this guide covers:

• Reference architecture for air-gapped AI environments
• Building an offline bundle that installs without touching the internet
• Verifying model integrity through SHA-256 chains and signed manifests
• Sneakernet workflow for models, container images and updates
• Which local AI tools survive the loss of egress, and which ones break
• Certificate management without ACME or an external CA
• Audit evidence for FedRAMP High, ISO 27001, HIPAA and SOC 2 reviews
• Update cadence and the real cost of staleness

Ollama, Open WebUI, llama.cpp and most of the local-AI ecosystem run fine in air-gap. The hard parts are operational: keeping the install reproducible, the models verifiable, the updates timely, and the audit trail intact. If you have never installed Ollama on a normal networked box, start with the complete Ollama guide and come back - the offline path assumes you know what the normal path does.

Table of Contents

1. Is your network actually air-gapped?
2. What does the threat model forbid?
3. What does the architecture look like?
4. What goes in the offline bundle?
5. How do you prove the bundle arrived intact?
6. How do you install Ollama with no internet?
7. How do you prove a model was not tampered with?
8. How do you run Open WebUI offline?
9. Which AI tools actually work air-gapped?
10. How do you get HTTPS with no Let's Encrypt?
11. How do you audit an air-gapped AI system?
12. How do you keep an offline stack updated?
13. What breaks first?

---

What does the air-gap threat model forbid?

A real air-gapped environment assumes that any data leaving the network is a security breach. That rules out:

• Egress to model providers (Hugging Face, the Ollama registry, GitHub releases)
• DNS lookups to internet hosts
• NTP from the public pool (use a local stratum-1 source instead)
• ICMP responses to traceroute, in some classifications
• Operating system and application telemetry of any kind
• Unlogged USB transfers - every one is logged, signed and audited

What you control:

• Hardware acquisition through an audited supply chain
• A low side workstation with internet that downloads artifacts
• A high side environment that is the actual air-gapped network
• A controlled transfer mechanism (USB, optical disc, one-way data diode)

The low side / high side terminology comes from defense networks. The same pattern applies to financial vault networks, OT/SCADA isolation and regulated healthcare environments.

What does an air-gapped AI deployment look like?

A canonical air-gapped AI deployment has four logical zones:

Zone 1 - Low side staging. Internet-connected workstation used to download artifacts. Never touches the air-gap network. Hosts the artifact mirror you assemble.

Zone 2 - Transfer media. Write-once or write-protected media (DVD-R, signed USB stick, one-way diode). Transports vetted artifacts from Zone 1 to Zone 3.

Zone 3 - High side ingest. Air-gapped workstation that receives artifacts from Zone 2, validates signatures, computes hashes and registers them in an internal artifact repository.

Zone 4 - Production AI nodes. Inference servers running Ollama. Pull from the internal artifact repo only. Never see Zone 1 or Zone 2 directly.

[Internet] -> [Zone 1: Low Side] -> [Zone 2: USB/DVD] -> [Zone 3: Ingest] -> [Zone 4: Production]
                                       |
                                       +-- one-way only, audited

This shape is not mandated by name in any framework, but it maps cleanly onto the information-flow and boundary-protection controls that high-baseline programmes inherit - specifically AC-4 (Information Flow Enforcement) and SC-7 (Boundary Protection) in NIST SP 800-53 Rev. 5. Less-strict environments often collapse Zones 3 and 4. The separation of low side from high side is the one that is not negotiable.

What goes in the offline bundle, and how do you build it?

Everything the air-gap network needs must be assembled into a single bundle on the low side:

Build the bundle on a freshly imaged Linux workstation. Commands below assume Ubuntu 22.04 LTS on the low side.

# Set up working directory
mkdir -p ~/airgap-bundle/{binaries,images,models,packages,signatures,manifest}
cd ~/airgap-bundle

# Pull the Ollama release tarball (this is the artifact the official
# manual-install docs use - not a bare binary)
curl -L https://github.com/ollama/ollama/releases/latest/download/ollama-linux-amd64.tgz \
  -o binaries/ollama-linux-amd64.tgz
sha256sum binaries/ollama-linux-amd64.tgz > signatures/ollama.sha256

# Pull Open WebUI image and save as a tarball
docker pull ghcr.io/open-webui/open-webui:main
docker save ghcr.io/open-webui/open-webui:main \
  -o images/open-webui.tar
sha256sum images/open-webui.tar > signatures/open-webui.sha256

# Pull models via Ollama (running locally on the low side)
ollama pull llama3.1:8b
ollama pull phi3:mini
ollama pull nomic-embed-text

# Copy model blobs from ~/.ollama/models to bundle
cp -r ~/.ollama/models models/
find models -type f -exec sha256sum {} \; > signatures/models.sha256

# Pull distro packages for offline apt
apt-get download nginx-full ca-certificates apparmor apparmor-utils
mv *.deb packages/

# Generate the bundle manifest
cat > manifest/bundle.json <<EOF
{
  "bundle_version": "$(date +%Y%m%d)",
  "creator": "$(whoami)",
  "creation_date": "$(date -u +%Y-%m-%dT%H:%M:%SZ)",
  "low_side_host": "$(hostname)",
  "artifacts": {
    "ollama_tarball": "$(sha256sum binaries/ollama-linux-amd64.tgz | cut -d' ' -f1)",
    "open_webui_image": "$(sha256sum images/open-webui.tar | cut -d' ' -f1)",
    "models_directory_hash": "$(find models -type f -exec sha256sum {} \; | sort | sha256sum | cut -d' ' -f1)"
  }
}
EOF

# Sign the manifest with your operator key
gpg --armor --detach-sign manifest/bundle.json

# Final tarball
tar czf airgap-bundle-$(date +%Y%m%d).tar.gz \
  binaries/ images/ models/ packages/ signatures/ manifest/
sha256sum airgap-bundle-*.tar.gz > airgap-bundle-final.sha256

The result is one signed tarball plus a SHA-256 hash. Both go on the transfer media, and the hash also travels out-of-band (phone call, printed sheet, separate courier) so the receiving operator has something independent to compare against.

Sizing it. Add up the tag sizes published on ollama.com/library rather than guessing. At the time of writing that page lists llama3.1:8b at 4.9 GB, phi3:mini at 2.2 GB and nomic-embed-text at 274 MB, so 4.9 + 2.2 + 0.27 puts the model portion at roughly 7.4 GB. For the container image, run docker images after the docker save and read the real number - it moves tag to tag. Add your OS package set on top and you have the disc size you need to plan for.

How do you move the bundle across and prove it arrived intact?

The transfer step is the most-audited part of any air-gap workflow. Three mechanisms, in declining order of assurance:

One-way data diode. Hardware that physically permits data flow in a single direction. Vendors such as Owl Cyber Defense and Forcepoint sell these by quote rather than list price, and the integration work is usually a larger line item than the appliance. Required for some classified environments; out of scope for most readers.

Signed write-once media (DVD-R, BD-R). Burned on the low side, hash-verified on the high side, unmodifiable after burn. Cheap and auditable, but slow: the Blu-ray 1x data rate is 4.5 MB/s, so a 6x burn moves about 27 MB/s and a full 25 GB disc takes roughly 25,000 / 27 = about 15 minutes of write time, before verification. Best practical option for most organisations.

Vetted USB with hardware write-protection. A stick with a physical read-only switch (Kanguru FlashTrust, Apricorn Aegis) flipped after writing on the low side, then logged in and out of the facility. Faster than optical, far cheaper than a diode.

The receiving procedure on the high side is where the audit evidence is actually created:

# Mount the transfer media read-only
sudo mount -o ro /dev/sr0 /mnt/transfer    # for optical
# or
sudo mount -o ro /dev/sdc1 /mnt/transfer   # for USB

# Verify outer hash matches what was provided out-of-band
sha256sum /mnt/transfer/airgap-bundle-*.tar.gz

# Verify GPG signature on bundle manifest
cd /tmp
mkdir verify && cd verify
tar xzf /mnt/transfer/airgap-bundle-*.tar.gz
gpg --verify manifest/bundle.json.asc manifest/bundle.json

# Confirm individual artifact hashes against manifest
sha256sum -c signatures/*.sha256

# If all checks pass, copy to internal artifact repo
sudo cp -r * /srv/airgap-repo/$(date +%Y%m%d)/

# Log the transfer
echo "$(date -u +%Y-%m-%dT%H:%M:%SZ) BUNDLE_INGEST $(whoami) bundle_$(date +%Y%m%d) sha256=$(sha256sum /mnt/transfer/airgap-bundle-*.tar.gz | cut -d' ' -f1)" \
  | sudo tee -a /var/log/airgap-transfers.log

Every action gets logged with timestamp, operator and artifact hash. That log is the first thing an auditor will ask for, and it is the only artifact that proves the chain was unbroken.

How do you install Ollama with no internet?

The convenience installer (curl -fsSL https://ollama.com/install.sh | sh) cannot work in air-gap - it fetches the script and then the release tarball from ollama.com. The official Linux docs also document a manual install, and that is the path you take offline: unpack the release tarball into /usr and write your own service unit.

# Create system user
sudo useradd --system --shell /bin/false --home /var/lib/ollama --create-home ollama

# Unpack the release tarball (this is the manual-install path from the
# official docs, minus the download step)
sudo tar -C /usr -xzf /srv/airgap-repo/latest/binaries/ollama-linux-amd64.tgz

# Confirm the binary landed and runs
/usr/bin/ollama --version

# Create data directory
sudo mkdir -p /var/lib/ollama
sudo chown ollama:ollama /var/lib/ollama

# Copy verified model blobs into Ollama's data dir
sudo cp -r /srv/airgap-repo/latest/models/* /var/lib/ollama/
sudo chown -R ollama:ollama /var/lib/ollama

# Create systemd unit
sudo tee /etc/systemd/system/ollama.service <<'EOF'
[Unit]
Description=Ollama AI Service
After=network-online.target

[Service]
Type=simple
User=ollama
Group=ollama
ExecStart=/usr/bin/ollama serve
Environment="OLLAMA_MODELS=/var/lib/ollama/models"
Environment="OLLAMA_HOST=127.0.0.1:11434"
Environment="OLLAMA_KEEP_ALIVE=30m"
# Hardening
NoNewPrivileges=yes
PrivateTmp=yes
ProtectSystem=strict
ProtectHome=yes
ReadWritePaths=/var/lib/ollama
Restart=on-failure
RestartSec=3

[Install]
WantedBy=multi-user.target
EOF

sudo systemctl daemon-reload
sudo systemctl enable --now ollama.service

# Verify: the models you copied should be listed
sudo systemctl status ollama
ollama list

Two settings matter more than the rest. OLLAMA_MODELS must point at the directory you copied blobs into, or ollama list comes back empty and looks like the copy failed. OLLAMA_HOST=127.0.0.1:11434 keeps the API on loopback so the reverse proxy is the only way in - the full manual-install and environment-variable reference lives in the official Ollama Linux documentation.

How do you prove a model file was not tampered with?

In a regulated environment you have to show every model file came from a verified source and was unchanged in transit. The chain:

1. On the low side. Ollama stores each model layer as a file under models/blobs/ named sha256-<digest>, and the manifest under models/manifests/registry.ollama.ai/library/ references those digests. The filename is the expected hash.
2. In the bundle. signatures/models.sha256 records a computed hash for every blob file.
3. On transfer media. The outer tarball hash matches the value delivered out-of-band.
4. On high-side ingest. sha256sum -c signatures/models.sha256 validates every blob.
5. On the production node. Re-run the same check after copying into /var/lib/ollama.
6. At any time afterwards. Because the digest is in the filename, re-verification is a one-liner - no external service required.

That last property is the useful one: you can re-prove integrity months later, on an isolated box, with nothing but sha256sum.

# Re-verify every blob against the digest embedded in its own filename
cd /var/lib/ollama/models/blobs
for f in sha256-*; do
  actual=$(sha256sum "$f" | cut -d' ' -f1)
  expected=${f#sha256-}
  [ "$actual" = "$expected" ] || echo "MISMATCH: $f"
done

For the strictest environments, add a signing step: after high-side validation, your organisation's PKI signs the model manifest, and production nodes verify both the upstream digest and your own signature before loading.

# On the high side, after validating the bundle
cd /srv/airgap-repo/latest/models

# Sign the manifest with your operator/CISO key
gpg --output models.sig --detach-sig --armor \
  manifests/registry.ollama.ai/library/llama3.1/8b

# Store the signature alongside the manifest on the production node
sudo cp models.sig /var/lib/ollama/models/manifests/registry.ollama.ai/library/llama3.1/8b.sig

That gives you a chain any auditor can replay from scratch.

How do you run Open WebUI offline without it phoning home?

Open WebUI ships as a container image. Loading it is easy; stopping it from reaching the internet on first boot is the part people miss. By default it downloads a sentence-transformers embedding model from Hugging Face for its RAG feature, and that happens the first time a user uploads a document - long after you disconnected.

# Load image from the bundle
sudo docker load -i /srv/airgap-repo/latest/images/open-webui.tar
sudo docker images | grep open-webui

# Run with auth on and every outbound path closed
sudo docker run -d \
  --name open-webui \
  --restart always \
  -p 127.0.0.1:3000:8080 \
  -v open-webui-data:/app/backend/data \
  -e OLLAMA_BASE_URL=http://host.docker.internal:11434 \
  -e WEBUI_AUTH=true \
  -e ENABLE_SIGNUP=false \
  -e ENABLE_OPENAI_API=false \
  -e DEFAULT_USER_ROLE=user \
  -e HF_HUB_OFFLINE=1 \
  -e ANONYMIZED_TELEMETRY=false \
  -e DO_NOT_TRACK=true \
  -e SCARF_NO_ANALYTICS=true \
  -e RAG_EMBEDDING_ENGINE=ollama \
  -e RAG_EMBEDDING_MODEL=nomic-embed-text \
  --add-host=host.docker.internal:host-gateway \
  ghcr.io/open-webui/open-webui:main

What each of the air-gap-specific variables buys you:

• HF_HUB_OFFLINE=1 stops the Hugging Face client from attempting any download - it fails fast instead of hanging on a DNS timeout
• RAG_EMBEDDING_ENGINE=ollama plus RAG_EMBEDDING_MODEL=nomic-embed-text routes embeddings to the model you already sneakernetted, so the Hugging Face download is never needed
• ANONYMIZED_TELEMETRY=false, DO_NOT_TRACK=true and SCARF_NO_ANALYTICS=true turn off the analytics paths
• ENABLE_OPENAI_API=false removes any code path that could reach OpenAI
• ENABLE_SIGNUP=false means only admin-created accounts work

These names come straight from the Open WebUI environment configuration reference; check it against the tag you actually bundled, because the list grows every few releases. After first start, read docker logs open-webui and confirm there are no outbound connection attempts before you sign off on the deployment.

For SSO, RBAC and audit-log integration, the Open WebUI setup guide covers the hardening this section skips.

Which local AI tools actually work air-gapped?

Ollama is the easy one. The rest of the stack varies enormously, and the failure mode is almost always the same: a first-run download that nobody noticed because the build machine had internet. This is the table to check before you cut the cable.

Deeper setup walkthroughs for the ones people ask about most: n8n with Ollama for local automation, Continue.dev with Ollama for local code assistance, the ComfyUI complete guide, the faster-whisper transcription guide, and the AnythingLLM setup guide. Run each one on a networked machine first, watch what it downloads, and put that in your bundle.

How do you get HTTPS certificates with no Let's Encrypt?

Let's Encrypt and public ACME require internet access. In air-gap you run an internal CA. Three workable approaches:

1. Step-CA (smallstep). A modern internal CA with an ACME-compatible interface that runs inside the air-gap network. Issues certs to internal hosts through the same ACME flow Let's Encrypt uses publicly, just with your internal root as the trust anchor.

2. cfssl or Easy-RSA. Simpler, script-driven PKI. Better for static infrastructure where certificates change rarely.

3. Active Directory Certificate Services. If the air-gap network already runs AD, ADCS issues certificates through Group Policy. Adds Windows dependencies but integrates cleanly with AD-joined hosts.

Step-CA quick setup:

# Install Step-CA from packages in your bundle
sudo dpkg -i /srv/airgap-repo/latest/packages/step-ca*.deb step-cli*.deb

# Initialize a new CA (one-time, on the CA host)
step ca init \
  --name "AcmeCorp Air-Gap CA" \
  --dns ca.airgap.internal \
  --address ":443" \
  --provisioner admin@airgap.internal

# Run the CA
sudo systemctl enable --now step-ca

# On the AI server, request a cert
step ca certificate ai.airgap.internal \
  /etc/nginx/ssl/ai.crt /etc/nginx/ssl/ai.key \
  --provisioner admin@airgap.internal

Distribute the CA root through your normal endpoint management (GPO, MDM, manual install). Once endpoints trust the internal root, internal HTTPS behaves exactly like public HTTPS.

How do you run a compliance audit on an air-gapped AI system?

An auditor reviewing an air-gapped AI deployment is checking two things: that nothing left the boundary, and that you can reconstruct who asked the model what. Neither is provable after the fact, so the logging has to exist from day one.

The minimum event set:

Retention periods are set by your framework and your data classification, not by any of these tools - do not copy a number out of a blog post into a policy document. The one figure with a clear statutory source is HIPAA's documentation requirement, which runs six years from creation or last effective date under 45 CFR 164.316(b)(2)(i). Everything else on that table is a policy decision your compliance team owns.

Two practical notes. Open WebUI stores conversations in SQLite by default; for anything audited, move to PostgreSQL with WAL archiving so the transcript survives a corrupted file, and disable user-initiated deletion so a record cannot vanish mid-retention. And decide up front whether prompts are records - if they are, the AI system inherits every discovery and retention obligation attached to that data class.

For schema design and retention strategy, the local AI audit trail guide goes considerably deeper, and role-based access control for local AI covers the permissions half of the same review.

How do you keep an air-gapped AI stack updated?

The hardest operational reality: an air-gapped network runs old software. The install was current the day the bundle was built and degrades from there.

A workable cadence to write into your SOP:

Each update follows the same path: build a new bundle on the low side, transfer through Zone 2, validate on Zone 3, deploy to Zone 4. The lag between upstream release and high-side deployment is a target you set and then measure yourself against - pick a routine-patch window and a shorter emergency window, publish both, and track the misses. An air-gap programme without a stated lag target does not have a slow update process; it has an unmeasured one.

Set user expectations against your own bundle cadence rather than against whatever model launched last week. On an air gap, your stack is exactly as new as your last transfer, and pretending otherwise is how you end up with an unapproved USB stick in a production server.

What breaks first on an air-gapped install?

Bundle hash mismatch on the high side. Bit rot on optical media is real. Burn two copies and verify both; the second disc costs less than a second trip through the facility's transfer process.

Ollama still reaches for the network. Ollama performs a background version check. There is no documented environment variable that disables it, so block it at the host firewall and confirm with ss -tnp that nothing is trying to establish an outbound connection. (OLLAMA_NOPRUNE=1 is a real variable, but it prevents blob pruning at startup - it is not a telemetry switch.)

ollama list is empty after copying models. Ollama expects manifests/ and blobs/ under whatever OLLAMA_MODELS points at. Check that the path in the unit file matches where you actually copied files, and that the ollama user owns them. ollama pull will not save you here - there is no registry to pull from.

Open WebUI shows update or telemetry activity. Set the variables in the Open WebUI section above, block egress at the host firewall as a backstop, and verify with container logs rather than trusting the settings UI.

TLS certs expire and break access overnight. Use Step-CA ACME for automated renewal. Manual certificate management always fails on the timescale of an air-gap deployment, usually while the person who issued them is on leave.

Audit logs fill the disk. Configure logrotate on day one. Models and conversations grow fast; audit logs grow slowly and never get pruned, which is worse.

A container cannot verify an internal TLS connection. It needs your CA root in its own trust store. Mount the cert into the container at /etc/ssl/certs/ca-certificates.crt or rebuild the image with the root baked in.

A new model will not load on an old runtime. Newer architectures need newer llama.cpp support, which means newer Ollama. Schedule the runtime update ahead of the model update or you will get stuck halfway through a bundle with nothing to roll back to.

Sneakernet gets too slow at scale. Past roughly 50 GB per bundle, optical media becomes painful - at the 27 MB/s figure above that is four full BD-R discs and over an hour of burn time. A vetted USB-C SSD with hardware write protection is the usual next step.

---

Frequently Asked Questions

Can I run Ollama with no internet at all, after install?

Yes. Once Ollama is installed and the model blobs are in place, inference needs no network. The only network-dependent operations are pulling new models and the background version check, and both are handled by the bundle process plus a firewall rule.

How big is a typical air-gap bundle?

Add up the tag sizes published on ollama.com/library rather than guessing. llama3.1:8b (4.9 GB) plus phi3:mini (2.2 GB) plus nomic-embed-text (274 MB) is about 7.4 GB of models. Add the Open WebUI image - read the real size from docker images after saving it - and your OS package set on top.

What about FedRAMP and DoD STIG compliance?

Standard Linux STIG hardening applies: disable root SSH, enforce password policy, run AIDE for file integrity. Neither Ollama nor Open WebUI has a FedRAMP-authorised build, so you are providing compensating controls - audit logging, network segmentation, endpoint hardening - and documenting them as such.

Can an air-gapped Ollama receive models without sneakernet?

Only through a one-way data diode. There is no other mechanism that preserves the air-gap property. Plenty of organisations sidestep the question entirely by building RAG over LAN-internal data sources and rarely changing the base model.

How do I update an air-gapped GPU driver safely?

Same bundle process: download the NVIDIA .run installer or distro driver package on the low side, hash and sign it, sneakernet it across. Test on a non-production node first - driver updates carry more rollback risk than anything else in the stack, and a failed one takes the GPU with it.

Is Hugging Face usable in air-gap?

The web UI is not, but the files are. Download model files on the low side and transport them; GGUF-quantised models load in Ollama through a custom Modelfile. Plan your model strategy around formats that are already quantised so you are not compiling anything on the high side.

What about OS telemetry on RHEL or Ubuntu?

Both have phone-home features that fail loudly when blocked, which generates noise in exactly the logs you need to be clean. Configure the offline subscription manager on RHEL, or disable Ubuntu Pro features. Rocky Linux and Debian are common choices for air-gap specifically because the telemetry surface is smaller.

Can I run image generation or transcription air-gapped?

Yes, with the same bundle pattern - but check the tool table above first. ComfyUI needs its custom nodes and checkpoints staged in advance because the node manager clones from GitHub at install time, and faster-whisper needs its weights in the cache before you set HF_HUB_OFFLINE=1. Checkpoints run 5-10 GB each, so size your transfer media accordingly.

---

Conclusion

Air-gapped AI deployment is an operational discipline more than a technical one. The software works: Ollama, Open WebUI and the rest of the local stack run cleanly without internet. The work is in the bundle process, the verification chain, the audit evidence and the rhythm of quarterly transfers.

If you are still deciding whether to air-gap at all, the honest answer is: only if your threat model genuinely requires it. The cost in update lag, operational overhead and user-experience degradation is real and permanent. If what you need is strong privacy rather than true isolation, the local AI privacy guide covers a lighter posture, and the AI hardware requirements guide will tell you what the machine needs to be either way.

If you do need the real thing - SCIF, classified network, regulated medical infrastructure, financial vault - the architecture above maps onto the controls auditors actually ask about, and every step of it produces evidence you can hand over.

---

Want more security and compliance-focused AI guides? Join the LocalAIMaster newsletter for weekly deployment patterns and audit walkthroughs.