Run LLMs on Google Colab's Free GPU: Ollama on a T4 for $0

Yes — Google Colab's free tier will run real LLMs today: you get an NVIDIA T4 with 16GB of VRAM, Ollama installs with one command, and llama3.1:8b (a 4.9GB download) fits with room to spare. Total cost: $0, no card on file. Five notebook cells take you from a blank page to a working model, and one more cell gives you a public HTTPS API you can hit from your laptop — no ngrok account even required if you use a Cloudflare quick tunnel.

The catch is all in Google's fine print, and this guide quotes it rather than hand-waving: free notebooks run "at most 12 hours, depending on availability and your usage patterns," Colab "restricts access to expensive resources like GPUs in its free tier," and the VM's disk is wiped when your session ends. Colab is a brilliant test bench and a terrible home. We will set it up properly, measure it honestly, and be clear about the moment you have outgrown it.

If you landed here because you do not own a usable GPU: this page, running an LLM on your phone, and running one in your browser are the three genuinely-free ways in. Colab is the most capable of the three.

---

What $0 Buys You: A 16GB Datacenter GPU

The free-tier GPU is typically an NVIDIA T4: 16GB of GDDR6, 320+ GB/s of memory bandwidth, 2,560 CUDA cores, and 65 FP16 TFLOPS, per NVIDIA's spec sheet. It is a 2018-era Turing inference card — slow by 2026 standards, but its 16GB of VRAM is the same memory budget as a 16GB desktop card, and for LLM inference, memory is what decides which models you can run at all.

Two Google-stated caveats before you build anything on this. First, the hardware is not a contract: "the types of GPUs and TPUs that are available in Colab vary over time," per the Colab FAQ — the T4 is what free users typically see, not what Google promises. Second, the limits are deliberately unpublished: "Colab does not publish these limits, in part because they can vary over time." Some days at peak hours you simply will not get a GPU. That is the deal at $0.

Worth saying plainly: this is not piracy or a loophole. Running an interactive Ollama session on a Colab GPU is exactly the interactive compute Colab exists for. The line you should not cross is treating it as a free 24/7 production server — Colab's FAQ is explicit that it "prioritizes interactive compute," and idle runtimes get reclaimed.

---

The Five-Cell Setup

Open colab.research.google.com, create a notebook, set Runtime → Change runtime type → T4 GPU, then run these five cells. Budget about five minutes, most of it the model download.

Cell 1 — confirm you actually got a GPU (if this errors, you are on a CPU runtime or the free pool is empty right now):

!nvidia-smi

You should see a Tesla T4 with roughly 15GB of memory free. Cell 2 — install Ollama with its official Linux installer (this is the exact command from ollama.com's Linux download page; Colab VMs run as root, so no sudo needed):

!curl -fsSL https://ollama.com/install.sh | sh

Cell 3 — start the Ollama server in the background. Notebook cells block while a command runs, so detach it and give it a moment to bind to port 11434:

!nohup ollama serve > ollama.log 2>&1 &
!sleep 3 && tail -2 ollama.log

Cell 4 — pull a model that fits comfortably. The download runs over Google's datacenter connection, not your home broadband:

!ollama pull llama3.1:8b

Cell 5 — talk to it, with --verbose so Ollama prints real timing stats at the end:

!ollama run llama3.1:8b --verbose "Explain what a KV cache is in two sentences."

That is the whole local loop. The --verbose flag matters: it prints an eval rate in tokens per second after every response, which turns "how fast is Colab?" from a forum argument into a number you measured yourself on today's hardware. If any of these steps feel like magic, our complete Ollama guide explains what each piece is doing — everything there applies inside Colab unchanged, because a Colab VM is just Ubuntu with a GPU.

One workflow tip that saves real time: keep the pull cell and the run cell separate. When Colab disconnects you (it will), Runtime → Run all rebuilds the whole stack in one click, and the only slow part is re-downloading the model.

---

Which Models Actually Fit in 16GB

The sweet spot is 7-8B models at the default 4-bit quantization — 4.4 to 5.2GB downloads that leave two-thirds of the T4's VRAM for context. 12-14B models fit too. 27B does not. Download sizes below are from the Ollama model library, August 2026:

Remember the download size is not the whole VRAM story: the KV cache grows with your context length, so a 9.3GB model with a long conversation can push against 16GB in a way a 5GB model never will. Our Ollama RAM/VRAM table covers the sizing math in detail, and since the T4's budget is identical to any 16GB desktop card, our best Ollama models for 16GB VRAM picks apply to Colab verbatim — that page is effectively the model menu for this notebook.

A note on deepseek-r1: it fits and it works, but reasoning models spend hundreds of tokens thinking before they answer. On a fast local card that is fine; on a T4's decode speed it turns a two-second answer into a coffee break. For interactive Colab use, a non-reasoning 8B is the better experience.

---

How Fast Is It, Honestly

We are not going to quote you a tokens-per-second benchmark we did not run on your session — Colab hardware allocation varies, so measure your own with --verbose. What we can give you is the physics: decode speed is memory-bandwidth-bound, and the T4's 320GB/s puts a hard ceiling of roughly 65 forward passes per second on a 4.9GB model (320 ÷ 4.9). Real-world throughput always lands well under that ceiling.

That arithmetic is the useful frame for expectations. Enthusiast-class desktop cards carry two to three times the T4's bandwidth — a used RTX 3090's 936GB/s nearly triples it — so the same llama3.1:8b decodes correspondingly faster at home. Same model, same quality, different patience budget. (The caveat cuts the other way too: a couple of budget 16GB cards ship T4-class bandwidth, so a card upgrade is not automatically a speed upgrade.) On the T4, an 8B model is genuinely usable for chat, summarization, and code questions; it is not the speed you would build a snappy product demo on.

Three practical observations from the shape of the hardware:

• First response after a pull is the slowest — the model loads from disk into VRAM. Subsequent prompts skip that.
• Prompt processing is much faster than generation — the T4's tensor cores chew through your input in parallel; the token-by-token decode is where the bandwidth ceiling bites. Long prompts, short answers is the T4-friendly usage pattern.
• Smaller models buy speed linearly — qwen3:4b at 2.5GB roughly doubles the theoretical decode ceiling versus an 8B. If your task survives a 4B model, use one.

Run the cell-5 --verbose prompt once per session and you will know your actual number in thirty seconds — more trustworthy than any table we could print here.

---

Getting a Public API URL Out of Colab

Ollama is now serving on the Colab VM's localhost:11434 — a tunnel makes it reachable from your laptop. The zero-signup route is a Cloudflare quick tunnel; the commands below, including the crucial host-header flag, are straight from Ollama's official FAQ.

Ollama rejects requests whose Host header does not look local, which is why every naive Colab tunnel guide ends in mysterious 403s. Both official fixes rewrite the header. Cell 6, the no-account version:

!wget -q https://github.com/cloudflare/cloudflared/releases/latest/download/cloudflared-linux-amd64.deb
!dpkg -i cloudflared-linux-amd64.deb
!cloudflared tunnel --url http://localhost:11434 --http-host-header="localhost:11434"

This cell keeps running — that is correct, it IS the tunnel. Cloudflare prints a random https://something.trycloudflare.com URL into the cell output; that is your API endpoint, no Cloudflare account required. From your own machine:

curl https://YOUR-SUBDOMAIN.trycloudflare.com/api/generate -d '{
  "model": "llama3.1:8b",
  "prompt": "Say hello from a datacenter.",
  "stream": false
}'

Cloudflare documents two quick-tunnel limits worth knowing: a cap of 200 in-flight requests, and no server-sent events — and SSE is what OpenAI-style streaming uses, so if you point a chat UI at Ollama's OpenAI-compatible /v1 endpoint through a quick tunnel, use non-streaming mode. If streaming matters, the alternative from the same Ollama FAQ is ngrok, which needs a free account and authtoken:

ngrok http 11434 --host-header="localhost:11434"

This serve-from-Colab pattern is a small, active ecosystem, not our invention — amin-tehrani/ollama-colab runs Colab-hosted Ollama "exposed as a public API endpoint via Ngrok" (its words), and Luxadevi/Ollama-Colab-Integration (129 stars) wraps the same idea with Cloudflared tunneling and a web UI. Reading either notebook is a good way to cross-check everything on this page against independent sources.

Two security sentences you should actually read. Anyone who has your tunnel URL can run generations on your GPU quota and pull models onto your VM, so treat the URL like a password and let it die with the session — and everything in our guide to securing Ollama applies double the moment an endpoint leaves localhost. Also remember your prompts now transit Google's VM and a tunnel provider: fine for experiments, wrong for anything sensitive — which is ultimately a local-hardware argument.

---

The Honest Limitations

Colab free is a loaner, not a server. Every constraint below is either Google's own published wording or labeled as community experience — none of it is a dealbreaker for learning, all of it is a dealbreaker for depending on it.

• Sessions end, full stop. "In the version of Colab that is free of charge notebooks can run for at most 12 hours, depending on availability and your usage patterns" — Google's FAQ, verbatim. That "depending on" clause means 12 hours is the ceiling, not a promise.
• Idle means disconnected. Google confirms runtimes "will time out if you are idle" and does not say when. Structure your notebook so one Runtime → Run all rebuilds everything, and losing a session costs you a model download, not an afternoon.
• Nothing persists. The VM disk is wiped between sessions, so your 4.9GB model re-downloads every single time. There is no supported way around this that is worth the complexity — just re-run the pull cell.
• GPU access is rationed. Colab "restricts access to expensive resources like GPUs in its free tier," and usage limits "fluctuate" by design. Hit your quota and you may see CPU-only runtimes for a while. There is no free-tier appeal process.
• Quick tunnels have hard edges. 200 in-flight requests, no SSE, random URL each session — per Cloudflare's docs. Fine for one user; wrong for sharing with a team.
• It is not yours. The FAQ's phrase is "Colab prioritizes interactive compute." Running a notebook you are actively using is the intended case. Leaving a tunnel up as your app's free backend is how accounts meet the usage-limit hammer.

If you read that list and thought "fine, I just want to test models before buying anything" — perfect, that is exactly the job this setup is best at. If you read it and winced, the next section is for you.

---

Outgrown Colab? What a Real GPU Costs

The moment you are restarting Colab sessions daily, re-downloading the same model, and planning your day around a 12-hour timer, the free tier has done its job: it proved local models handle your work. The exit is not a bigger cloud bill — it is a one-time purchase of the same VRAM you have been borrowing.

The math is unusually clean because you already know your requirements. Everything you ran on the T4 fits in 16GB — and the 8B models that felt best probably fit in 12GB. That makes the entry point cheap: a used RTX 3060 12GB runs the same llama3.1:8b and qwen3:8b you just tested, at home, with no session timer, no re-downloads, no tunnel, and no prompts leaving your machine — our RTX 3060 model guide shows exactly what it handles. If you want the full 16GB you had on the T4 (with several times the memory bandwidth), the 16GB VRAM picks page doubles as a shopping brief.

For the wider map — what each VRAM tier unlocks, what boxes cost in the current market, and when a mini-PC beats a GPU — start at our hardware hub. The honest pitch is not that Colab is bad; it is that Colab charges you in time and friction what a GPU charges you once in dollars. Borrow the datacenter card until you know what you need. Then own the card.

---

Sources

• Google Colab FAQ — free-tier runtime limits, GPU availability, and idle-timeout wording (all quotes above)
• NVIDIA T4 product page — VRAM, bandwidth, core counts, TFLOPS, power draw
• Ollama Linux download and Ollama official FAQ — install command, ngrok and Cloudflare tunnel commands with host-header flags
• Ollama model library — all model download sizes (checked August 2026)
• Cloudflare TryCloudflare docs — quick-tunnel limits (200 in-flight requests, no SSE) and cloudflared downloads
• GitHub: amin-tehrani/ollama-colab, Luxadevi/Ollama-Colab-Integration — the active Colab-Ollama serving ecosystem

---

FAQ