Local AI Benchmarks: Measured Ollama Speeds
By Barsha Mohapatra · measurements run with the same harness on every device; the raw harness log behind every row is kept on record.
This is every generation speed we have measured ourselves: 43 rows covering 20 Ollama model tags on two devices, measured 14 October – 3 November 2026. The fastest is llama3.2:3b at 103.0 tok/s on the NVIDIA L4 (24 GB). Real runs landed at 52–90% of the memory-bandwidth ceiling, which is why every speed on this site that we did not measure is labelled as a ceiling. Download the rows as JSON or CSV and reuse them under CC BY 4.0 with a credit link.
Devices in the dataset
- Apple M3 Pro (18 GB) (150 GB/s): 13 models, our own machine, Ollama 0.40.13 / 0.12.3, 14 October – 2 November 2026. What runs on it
- NVIDIA L4 (24 GB) (300 GB/s): 17 models, rented cloud VM, Ollama 0.40.13, 3 November 2026.
We add rented GPUs one card at a time. A rented-GPU row is published only after a second, independent host with the same card reproduces it. Until a card has rows here, its page shows labelled bandwidth ceilings only, for example the GPU guides and the VRAM calculator.
Every measured row
One row per model, device and run date. Re-runs are added as new rows, never written over old ones, so a speed change between Ollama versions stays visible. Share is the measured median as a percentage of the ceiling.
| Device | Model (Ollama tag) | Quant | File | tok/s | Low–high | Ceiling | Share | Ollama | Date |
|---|---|---|---|---|---|---|---|---|---|
| Apple M3 Pro (18 GB) | llama3.2:3b | Q4_K_M | 2.02 GB | 55.6 | 53.4–55.8 | 74.3 | 75% | 0.40.13 | 2026-11-02 |
| Apple M3 Pro (18 GB) | llama3.2:3b | Q4_K_M | 2.02 GB | 55.3 | 54.8–55.8 | 74.3 | 74% | 0.40.13 | 2026-11-02 |
| Apple M3 Pro (18 GB) | qwen2.5-coder:3b-base | Q4_K_M | 1.93 GB | 55.0 | 53.8–56.6 | 77.7 | 71% | 0.40.13 | 2026-11-02 |
| Apple M3 Pro (18 GB) | llama3.2:3b | Q4_K_M | 2.02 GB | 55.0 | 54.9–55.1 | 74.3 | 74% | 0.12.3 | 2026-10-14 |
| Apple M3 Pro (18 GB) | qwen2.5-coder:3b-base | Q4_K_M | 1.93 GB | 54.0 | 53.6–54.3 | 77.7 | 69% | 0.12.3 | 2026-10-14 |
| Apple M3 Pro (18 GB) | phi4-mini | Q4_K_M | 2.49 GB | 46.3 | 46.1–46.4 | 60.2 | 77% | 0.12.3 | 2026-10-14 |
| Apple M3 Pro (18 GB) | qwen3:4b | Q4_K_M | 2.50 GB | 45.1 | 43.8–45.3 | 60.0 | 75% | 0.40.13 | 2026-11-02 |
| Apple M3 Pro (18 GB) | qwen3:4b | Q4_K_M | 2.50 GB | 44.7 | 44.1–44.8 | 60.0 | 75% | 0.40.13 | 2026-11-02 |
| Apple M3 Pro (18 GB) | phi4-mini:3.8b | Q4_K_M | 2.49 GB | 43.7 | 42.8–45.2 | 60.2 | 73% | 0.40.13 | 2026-11-02 |
| Apple M3 Pro (18 GB) | qwen3:4b | Q4_K_M | 2.50 GB | 43.2 | 42.4–43.2 | 60.0 | 72% | 0.12.3 | 2026-10-14 |
| Apple M3 Pro (18 GB) | qwen2.5-coder:7b | Q4_K_M | 4.68 GB | 29.0 | 28.9–29.0 | 32.1 | 90% | 0.40.13 | 2026-11-02 |
| Apple M3 Pro (18 GB) | mistral:7b | Q4_K_M | 4.37 GB | 29.0 | 29.0–29.0 | 34.3 | 84% | 0.40.13 | 2026-11-02 |
| Apple M3 Pro (18 GB) | mistral:7b | Q4_K_M | 4.37 GB | 28.5 | 28.4–28.6 | 34.3 | 83% | 0.12.3 | 2026-10-15 |
| Apple M3 Pro (18 GB) | qwen2.5-coder:7b | Q4_K_M | 4.68 GB | 28.0 | 27.8–28.2 | 32.1 | 87% | 0.12.3 | 2026-10-15 |
| Apple M3 Pro (18 GB) | llama3.1:8b | Q4_K_M | 4.92 GB | 27.2 | 26.9–27.4 | 30.5 | 89% | 0.40.13 | 2026-11-02 |
| Apple M3 Pro (18 GB) | llama3.1:8b | Q4_K_M | 4.92 GB | 26.8 | 26.5–27.2 | 30.5 | 88% | 0.40.13 | 2026-11-02 |
| Apple M3 Pro (18 GB) | llama3.1:8b | Q4_K_M | 4.92 GB | 26.8 | 26.7–26.9 | 30.5 | 88% | 0.12.3 | 2026-10-14 |
| Apple M3 Pro (18 GB) | deepseek-r1:8b | Q4_K_M | 5.23 GB | 25.4 | 25.4–25.5 | 28.7 | 89% | 0.40.13 | 2026-11-02 |
| Apple M3 Pro (18 GB) | qwen3:8b | Q4_K_M | 5.23 GB | 25.3 | 25.2–25.3 | 28.7 | 88% | 0.40.13 | 2026-11-02 |
| Apple M3 Pro (18 GB) | qwen3:8b | Q4_K_M | 5.23 GB | 25.2 | 25.1–25.2 | 28.7 | 88% | 0.40.13 | 2026-11-02 |
| Apple M3 Pro (18 GB) | qwen3:8b | Q4_K_M | 5.23 GB | 24.9 | 24.8–24.9 | 28.7 | 87% | 0.12.3 | 2026-10-15 |
| Apple M3 Pro (18 GB) | deepseek-r1:8b | Q4_K_M | 5.23 GB | 24.8 | 24.6–25.0 | 28.7 | 86% | 0.12.3 | 2026-10-15 |
| Apple M3 Pro (18 GB) | gemma3:12b | Q4_K_M | 7.31 GB | 16.7 | 16.1–16.8 | 20.5 | 81% | 0.40.13 | 2026-11-02 |
| Apple M3 Pro (18 GB) | deepseek-r1:14b | Q4_K_M | 8.99 GB | 14.7 | 14.6–14.8 | 16.7 | 88% | 0.40.13 | 2026-11-02 |
| Apple M3 Pro (18 GB) | qwen3:14b | Q4_K_M | 9.28 GB | 14.5 | 14.4–14.5 | 16.2 | 90% | 0.40.13 | 2026-11-02 |
| Apple M3 Pro (18 GB) | qwen2.5-coder:14b | Q4_K_M | 8.99 GB | 14.5 | 14.4–14.6 | 16.7 | 87% | 0.40.13 | 2026-11-02 |
| NVIDIA L4 (24 GB) | llama3.2:3b | Q4_K_M | 2.02 GB | 103.0 | 102.9–103.4 | 148.5 | 69% | 0.40.13 | 2026-11-03 |
| NVIDIA L4 (24 GB) | qwen3:4b | Q4_K_M | 2.50 GB | 81.3 | 81.0–81.7 | 120.0 | 68% | 0.40.13 | 2026-11-03 |
| NVIDIA L4 (24 GB) | qwen3.5:4b | Q4_K_M | 2.65 GB | 59.3 | 59.1–59.5 | 113.2 | 52% | 0.40.13 | 2026-11-03 |
| NVIDIA L4 (24 GB) | mistral:7b | Q4_K_M | 4.37 GB | 53.2 | 53.2–53.4 | 68.6 | 77% | 0.40.13 | 2026-11-03 |
| NVIDIA L4 (24 GB) | qwen2.5-coder:7b | Q4_K_M | 4.68 GB | 52.0 | 51.9–52.2 | 64.1 | 81% | 0.40.13 | 2026-11-03 |
| NVIDIA L4 (24 GB) | llama3.1:8b | Q4_K_M | 4.92 GB | 49.2 | 49.2–49.4 | 61.0 | 81% | 0.40.13 | 2026-11-03 |
| NVIDIA L4 (24 GB) | qwen3:8b | Q4_K_M | 5.23 GB | 46.2 | 46.1–46.3 | 57.4 | 81% | 0.40.13 | 2026-11-03 |
| NVIDIA L4 (24 GB) | deepseek-r1:8b | Q4_K_M | 5.23 GB | 46.2 | 46.0–46.3 | 57.4 | 81% | 0.40.13 | 2026-11-03 |
| NVIDIA L4 (24 GB) | qwen3.5:9b | Q4_K_M | 5.63 GB | 37.1 | 37.0–37.1 | 53.3 | 70% | 0.40.13 | 2026-11-03 |
| NVIDIA L4 (24 GB) | gemma3:12b | Q4_K_M | 7.31 GB | 30.3 | 30.2–30.3 | 41.0 | 74% | 0.40.13 | 2026-11-03 |
| NVIDIA L4 (24 GB) | qwen2.5-coder:14b | Q4_K_M | 8.99 GB | 26.8 | 26.7–26.9 | 33.4 | 80% | 0.40.13 | 2026-11-03 |
| NVIDIA L4 (24 GB) | deepseek-r1:14b | Q4_K_M | 8.99 GB | 26.8 | 26.7–26.8 | 33.4 | 80% | 0.40.13 | 2026-11-03 |
| NVIDIA L4 (24 GB) | qwen3:14b | Q4_K_M | 9.28 GB | 26.5 | 26.5–26.5 | 32.3 | 82% | 0.40.13 | 2026-11-03 |
| NVIDIA L4 (24 GB) | mistral-small3.2:24b | Q4_K_M | 15.18 GB | 17.1 | 17.1–17.2 | 19.8 | 87% | 0.40.13 | 2026-11-03 |
| NVIDIA L4 (24 GB) | gemma3:27b | Q4_K_M | 16.56 GB | 14.1 | 14.1–14.1 | 18.1 | 78% | 0.40.13 | 2026-11-03 |
| NVIDIA L4 (24 GB) | qwen2.5-coder:32b | Q4_K_M | 19.85 GB | 12.3 | 12.3–12.3 | 15.1 | 81% | 0.40.13 | 2026-11-03 |
| NVIDIA L4 (24 GB) | deepseek-r1:32b | Q4_K_M | 19.85 GB | 12.3 | 12.3–12.3 | 15.1 | 81% | 0.40.13 | 2026-11-03 |
tok/s = median generation rate of the valid runs. Ceiling = the device's vendor-listed memory bandwidth ÷ the model's file size. Thinking models (qwen3, deepseek-r1) are timed over every token they emit, reasoning included. The full rows, with run counts, OS, driver, host and raw-log path, are in the downloads.
How each row is measured
- The harness.
benchmark-ollama.py(one Python 3 file, standard library only, MIT licence) sends the same prompt to Ollama's HTTP API, streams the reply to time the first token, and computes each run's rate from the raweval_countandeval_durationin Ollama's final message, not from the terminal output (an early version that parsedollama run --verboseprinted an impossible rate because of escape codes; the October 2026 rows came from the first version, which did not stream). Output is capped at 512 tokens with a fixed seed. The prompt for every row: “Explain how attention works in a transformer model. Give a clear, technical answer of roughly 200 words.” - Runs and the median. Each model gets 7 runs. The first is discarded as warm-up, any run that generated fewer than 30 tokens is discarded rather than averaged, and the row reports the median of what is left (6 valid runs in the current rows) with the slowest and fastest valid run beside it.
- Validity gates. A row needs at least 3 valid runs. Its median must not exceed 1.05× the bandwidth ceiling, because a dense model reads every weight once per generated token and cannot outrun the memory; a row that does is rejected as a measurement error, not published. The ceiling itself must match the device's bandwidth divided by the file size, so a row cannot pass by carrying a bigger ceiling. The one honest way past the ceiling is a model that drafts several tokens per step (multi-token prediction): its runs are real but not comparable, so it is held back (list below) rather than published.
- Hosts that must agree. The Apple Silicon rows come from our own machine. A rented GPU is a machine we do not control (power limits, drivers, neighbours on the host), so a rented-GPU row stays unpublished until a second, independent host with the same card (a different physical GPU) reproduces it within 10%. A sweep only counts if the machine was quiet: the script records the load, power source and any other loaded model, and a sweep with a warning verifies nothing. Our first NVIDIA L4 host was timed straight after its model downloads, so its rows stay in the raw logs and the published L4 rows come from the two clean hosts that followed.
- Sizes and builds. Quantisation and file size come from the Ollama registry manifest for the exact tag pulled. Rows record the manifest digest where the harness captured it, so a re-published tag can be told apart; the first sweep did not record digests or a context setting (the harness left
num_ctxat Ollama's default), and those fields are empty in its rows rather than filled in after the fact. - What a row is not. One device, one prompt, generation speed only. It does not include prompt processing, and it falls as the context grows. The current rows were measured on Ollama 0.40.13 / 0.12.3.
Held back: qwen3.5:4b (Apple Silicon), qwen3.5:9b (Apple Silicon), qwen3.6:27b (every device)
qwen3.5:4b, Apple Silicon: On Apple Silicon, Ollama 0.40.13 served an NVFP4 MLX build instead of the Q4_K_M listed in scripts/cloud-bench/tags.txt, and Qwen3.5 has a multi-token-prediction head (text_config.mtp_num_hidden_layers = 1 in https://huggingface.co/Qwen/Qwen3.5-9B/raw/main/config.json). Its M3 Pro run (36.9 tok/s, raw/apple-m3-pro-18gb/2026-11-02-owner-mac-6.json) sat at 98% of the file-size ceiling, which the dense-ceiling method cannot interpret. Not held on NVIDIA: there Ollama serves the Q4_K_M GGUF without MTP layers (no qwen35.nextn_predict_layers key; llama-server logs 'no implementations specified for speculative decoding') and the L4 measured 59.3 tok/s against a 113.2 tok/s ceiling (verified on two hosts, 3 Nov 2026).qwen3.5:9b, Apple Silicon: Same family and Apple build as qwen3.5:4b. Its M3 Pro run measured 35.9 tok/s against a 19.6 tok/s file-size ceiling (raw/apple-m3-pro-18gb/2026-11-02-owner-mac-10.json): faster than one full read of the weights per token allows, consistent with multi-token prediction. Not held on NVIDIA: the Q4_K_M GGUF has no MTP layers there and the L4 measured 37.1 tok/s against a 53.3 tok/s ceiling (verified on two hosts, 3 Nov 2026).qwen3.6:27b, every device: Ollama 0.40.13's Q4_K_M GGUF carries one multi-token-prediction layer (qwen35.nextn_predict_layers = 1, 65 blocks for a 64-layer model) and llama-server creates an MTP draft context for it, so it decodes with self-speculation. On three NVIDIA L4 hosts it measured 25.2, 23.2 and 26.4 tok/s against a 17.8 tok/s file-size ceiling (raw/nvidia-l4-24gb/2026-11-03-cloud-nvidia-l4.json, -2.json, -3.json; the first host's warm-up, before drafts were accepted, ran at 16.4 tok/s). Real, but not comparable with one-read-per-token rows, and MTP could inflate speeds that stay under the ceiling on other devices. Held on every device until an MTP-aware method exists.
Where the rows are used: the measured tables on the Llama 3.1 8B, Mistral 7B and Qwen2.5-Coder 7B pages, the best Ollama models guide, the DeepSeek R1 guide, the Apple Silicon buying guide and every Mac guide. The editorial rules behind them are on our content policy page.
Download and reuse
The data is licensed under Creative Commons Attribution 4.0 International: use it in articles, tools, papers and products, commercial or not, as long as you credit it. The credit line to use, with a link to this page:
Local AI Master (localaimaster.com/benchmarks), Creative Commons Attribution 4.0 International (CC BY 4.0)Online, link the words “Local AI Master” to https://localaimaster.com/benchmarks. If you change or filter the rows, say so. Spotted a row that looks wrong? Tell us through the contact page; corrections are added as new rows and noted in the changelog.
How to reproduce a row
The harness is a single Python 3 file with no dependencies beyond the standard library: download benchmark-ollama.py (MIT licence; it is the same file that produced every row). With Ollama running on the machine you want to measure, close everything else and run:
curl -O https://localaimaster.com/data/benchmark-ollama.pyollama pull llama3.1:8bpython3 benchmark-ollama.py llama3.1:8b --runs 7It prints the environment (accelerator, memory, driver, Ollama version, UTC time), every run, and the median of the valid runs. To check your result against the ceiling, divide your card's memory bandwidth in GB/s by the file size Ollama downloaded: on the NVIDIA L4 (24 GB) that is 300 ÷ 4.92 GB ≈ 61.0 tok/s for llama3.1:8b, which measured 49.2. A median above the ceiling means something in the measurement is wrong, or the model drafts tokens ahead (multi-token prediction, as Qwen3.6 27B does); python3 benchmark-ollama.py --list-devices shows the bandwidth figures the script knows.
Changelog
- 2026-11-03: The measuring script is now a download on this page (MIT licence), the same file that produced every row. The reference table on our guides moved from the 14–15 October M3 Pro run (Ollama 0.12.3) to the 2 November run (Ollama 0.40.13); the October rows stay in the dataset. Qwen3.6 27B is held back on every device because it drafts tokens ahead (multi-token prediction), and Qwen3.5 is held back on Apple Silicon only; see the held-back list under the method.
- 2026-11-03: 17 rows measured on the NVIDIA L4 (24 GB) (rented cloud VM, Ollama 0.40.13): deepseek-r1:14b, deepseek-r1:32b, deepseek-r1:8b, gemma3:12b, gemma3:27b, llama3.1:8b, llama3.2:3b, mistral-small3.2:24b, mistral:7b, qwen2.5-coder:14b, qwen2.5-coder:32b, qwen2.5-coder:7b, qwen3.5:4b, qwen3.5:9b, qwen3:14b, qwen3:4b, qwen3:8b.
- 2026-11-02: Published as an open dataset: one row file in a fixed format, JSON and CSV downloads, CC BY 4.0. The nine M3 Pro rows from 14–15 October were carried over with their raw harness logs. Checking them against those logs, three rows (phi4-mini, llama3.1:8b, qwen3:8b) had the discarded warm-up run as their slowest run; their low value is now the slowest valid run. No median changed.
- 2026-11-02: 17 rows measured on the Apple M3 Pro (18 GB) (our own machine, Ollama 0.40.13): deepseek-r1:14b, deepseek-r1:8b, gemma3:12b, llama3.1:8b, llama3.1:8b, llama3.2:3b, llama3.2:3b, mistral:7b, phi4-mini:3.8b, qwen2.5-coder:14b, qwen2.5-coder:3b-base, qwen2.5-coder:7b, qwen3:14b, qwen3:4b, qwen3:4b, qwen3:8b, qwen3:8b.
- 2026-10-15: 4 rows measured on the Apple M3 Pro (18 GB) (our own machine, Ollama 0.12.3): deepseek-r1:8b, mistral:7b, qwen2.5-coder:7b, qwen3:8b.
- 2026-10-14: 5 rows measured on the Apple M3 Pro (18 GB) (our own machine, Ollama 0.12.3): llama3.1:8b, llama3.2:3b, phi4-mini, qwen2.5-coder:3b-base, qwen3:4b.
Turn these numbers into a working local setup
All 561 chapters — running local models, RAG, agents, fine-tuning — plus the Python Lab and every course added later.

Written by
Barsha Mohapatra
Founder and Editor, Local AI Master · Founder, Little AI Master
Founder of Little AI Master and the editor behind every course and guide on Local AI Master. Years of tutoring students taught her where AI education actually fails.