Local AI Benchmarks: Measured Ollama Speeds

By Barsha Mohapatra · measurements run with the same harness on every device; the raw harness log behind every row is kept on record.

Devices in the dataset

• Apple M3 Pro (18 GB) (150 GB/s): 9 models, our own machine, Ollama 0.12.3, 14–15 October 2026. What runs on it

We plan to add rented consumer GPUs. A rented-GPU row is published only after a second, independent host with the same card reproduces it. Until a card has rows here, its page shows labelled bandwidth ceilings only, for example the GPU guides and the VRAM calculator.

Every measured row

One row per model, device and run date. Re-runs are added as new rows, never written over old ones, so a speed change between Ollama versions stays visible. Share is the measured median as a percentage of the ceiling.

tok/s = median generation rate of the valid runs. Ceiling = the device's vendor-listed memory bandwidth ÷ the model's file size. Thinking models (qwen3, deepseek-r1) are timed over every token they emit, reasoning included. The full rows, with run counts, OS, driver, host and raw-log path, are in the downloads.

How each row is measured

• The harness. scripts/benchmark-ollama.py sends the same prompt to Ollama's HTTP API with streaming off and computes each run's rate from the raw eval_count and eval_duration it returns, not from the terminal output (an early version that parsed ollama run --verbose printed an impossible rate because of escape codes). The prompt for the current rows: “Explain how attention works in a transformer model. Give a clear, technical answer of roughly 200 words.”
• Runs and the median. Each model gets 7 runs. The first is discarded as warm-up, any run that generated fewer than 30 tokens is discarded rather than averaged, and the row reports the median of what is left (6 valid runs in the current rows) with the slowest and fastest valid run beside it.
• Validity gates. A row needs at least 3 valid runs. Its median must not exceed 1.05× the bandwidth ceiling, because a dense model reads every weight once per generated token and cannot outrun the memory; a row that does is rejected as a measurement error, not published. The ceiling itself must match the device's bandwidth divided by the file size, so a row cannot pass by carrying a bigger ceiling.
• Hosts that must agree. The Apple Silicon rows come from our own machine. A rented GPU is a machine we do not control (power limits, drivers, neighbours on the host), so a rented-GPU row stays unpublished until a second, independent host with the same card reproduces it.
• Sizes and builds. Quantisation and file size come from the Ollama registry manifest for the exact tag pulled. Rows record the manifest digest where the harness captured it, so a re-published tag can be told apart; the first sweep did not record digests or a context setting (the harness left num_ctx at Ollama's default), and those fields are empty in its rows rather than filled in after the fact.
• What a row is not. One device, one prompt, generation speed only. It does not include prompt processing, and it falls as the context grows. The current rows were measured on Ollama 0.12.3.

Where the rows are used: the measured tables on the Llama 3.1 8B and Mistral 7B pages, the Apple Silicon buying guide and every Mac guide. The editorial rules behind them are on our content policy page.

Download and reuse

The data is licensed under Creative Commons Attribution 4.0 International: use it in articles, tools, papers and products, commercial or not, as long as you credit it. The credit line to use, with a link to this page:

Online, link the words “Local AI Master” to https://localaimaster.com/benchmarks. If you change or filter the rows, say so. Spotted a row that looks wrong? Tell us through the contact page; corrections are added as new rows and noted in the changelog.

How to reproduce a row

The harness is a single Python 3 file with no dependencies beyond the standard library; the public repository link will be added here when it is published. With Ollama running on the machine you want to measure, close everything else and run:

It prints the environment (accelerator, memory, driver, Ollama version, UTC time), every run, and the median of the valid runs. To check your result against the ceiling, divide your card's memory bandwidth in GB/s by the file size Ollama downloaded: on the Apple M3 Pro (18 GB) that is 150 ÷ 4.92 GB ≈ 30.5 tok/s for llama3.1:8b, which measured 26.8. A median above the ceiling means something in the measurement is wrong.

Changelog

• 2026-11-02: Published as an open dataset: one row file in a fixed format, JSON and CSV downloads, CC BY 4.0. The nine M3 Pro rows from 14–15 October were carried over with their raw harness logs. Checking them against those logs, three rows (phi4-mini, llama3.1:8b, qwen3:8b) had the discarded warm-up run as their slowest run; their low value is now the slowest valid run. No median changed.
• 2026-10-15: 4 rows measured on the Apple M3 Pro (18 GB) (our own machine, Ollama 0.12.3): mistral:7b, qwen2.5-coder:7b, qwen3:8b, deepseek-r1:8b.
• 2026-10-14: 5 rows measured on the Apple M3 Pro (18 GB) (our own machine, Ollama 0.12.3): llama3.2:3b, qwen2.5-coder:3b-base, phi4-mini, qwen3:4b, llama3.1:8b.

Written by

Barsha Mohapatra

Founder and Editor, Local AI Master · Founder, Little AI Master

Founder of Little AI Master and the editor behind every course and guide on Local AI Master. Years of tutoring students taught her where AI education actually fails.

About BarshaLinkedInLittle AI Master