Qwen 2.5 3B: Size, VRAM, Ollama Tags and the Research Licence
Qwen 2.5 3B is Alibaba Cloud's 3-billion-parameter chat model from September 2024. On Ollama, qwen2.5:3b is a 1.9 GB download and needs about 4.3 GB of VRAM with its full 32K context, or about 3.4 GB at an 8K context, so it runs on a 4 GB graphics card and, with no GPU, on a laptop with 8 GB of RAM. One thing sets it apart from the rest of the family: the 3B is under the Qwen Research Licence, which permits non-commercial use only, while the 0.5B, 1.5B, 7B, 14B and 32B are Apache 2.0. This page has not benchmarked it; every figure comes from Alibaba's model card and blog posts, Hugging Face or Ollama.
Updated: October 15, 2026
What hardware does Qwen 2.5 3B need?
A 4 GB card runs the default tag at an 8K context, and the q2_K and q3_K_M tags at the full 32K. A 6 GB card runs everything up to q6_K at full context. An 8 GB card runs q8_0. CPU-only works: the default tag plus a 4K cache is about 2.1 GB, so a machine with 8 GB of RAM holds it with room to spare.
“VRAM needed” is arithmetic, not a measurement: the file size, plus the key-value cache for the whole 32,768-token context, plus 1 GB for compute buffers. The cache comes from config.json: 36 layers × 2 key-value heads × 128 dimensions × 2 (keys and values) × 2 bytes = 36,864 bytes per token, which is 1.21 GB at 32,768 tokens, 0.30 GB at 8,192 and 0.15 GB at Ollama's default 4,096. Two key-value heads against sixteen query heads is why the cache is small for a model with a 32K window.
| Ollama tag | Ollama download | GGUF file (Qwen's repo) | VRAM needed (32K context) | Fits |
|---|---|---|---|---|
3b-instruct-q2_K | 1.3 GB | 1.38 GB | 3.6 GB | 4 GB card |
3b-instruct-q3_K_M | 1.6 GB | 1.72 GB | 3.9 GB | 4 GB card, with little headroom |
3b-instruct-q4_0 | 1.8 GB | 2.00 GB | 4.2 GB | 6 GB card (4 GB at an 8K context) |
3b (default, q4_K_M) | 1.9 GB | 2.10 GB | 4.3 GB | 6 GB card (4 GB at an 8K context) |
3b-instruct-q5_K_M | 2.2 GB | 2.44 GB | 4.7 GB | 6 GB card |
3b-instruct-q6_K | 2.5 GB | 2.79 GB | 5.0 GB | 6 GB card |
3b-instruct-q8_0 | 3.3 GB | 3.62 GB | 5.8 GB | 8 GB card (6 GB at an 8K context) |
3b-instruct-fp16 | 6.2 GB | 6.80 GB | 9.0 GB | 12 GB card |
The GGUF column is Qwen's own quantised files, which run a little larger than the Ollama tags of the same name; the table uses the larger figure so the “needs” number errs on the safe side. In bfloat16 the weights are about 6.2 GB, the same as the fp16 tag, so a 12 GB card runs the model unquantised with its full context.
With no GPU, Ollama loads the file into system RAM. At under 2 GB for the default tag, this is one of the few chat models that is practical on an 8 GB laptop with a browser still open, and it fits a Raspberry Pi 5 with 8 GB. Speed on a processor depends on the machine and has not been measured here; the general picture, and which processors are fast enough for chat, is in running LLMs on CPU only and the best models for 8 GB of RAM.
Where a card is marginal, the lever is context length rather than quantisation: dropping from 32K to 8K saves 0.9 GB, more than the difference between the q4_K_M and q2_K files. The quantisation levels themselves are explained in the quantisation guide.
What Qwen 2.5 3B is
Qwen2.5 is the September 2024 generation of Alibaba Cloud's open-weight language models, released in seven sizes from 0.5B to 72B and announced in the Qwen2.5 blog post on September 19, 2024. The 3B is the middle of the three small sizes. Its official card is at huggingface.co/Qwen/Qwen2.5-3B-Instruct.
| Spec | Qwen2.5-3B-Instruct |
|---|---|
| Parameters | 3.09B total, 2.77B excluding embeddings |
| Release | September 19, 2024 (Hugging Face repository created September 17, 2024) |
| Architecture | Decoder-only transformer, 36 layers, 16 query heads, 2 key-value heads (grouped-query attention), RoPE, SwiGLU, RMSNorm, QKV bias, tied embeddings |
| Context window | 32,768 tokens; generation up to 8,192 tokens |
| Vocabulary | 151,936 tokens |
| Languages | Over 29, including Chinese, English, French, Spanish, Portuguese, German, Italian, Russian, Japanese, Korean, Vietnamese, Thai and Arabic |
| Training data | The Qwen2.5 family was pre-trained on up to 18 trillion tokens (Alibaba does not publish a per-size figure) |
| Tool calling | Yes; the Ollama library page shows the tools badge |
| Licence | Qwen Research Licence Agreement (non-commercial) |
The 3B exists as a base model (Qwen2.5-3B) and an instruction-tuned chat model (Qwen2.5-3B-Instruct). Ollama only publishes the instruct build; qwen2.5:3b and qwen2.5:3b-instruct are the same file. If you need the base checkpoint for fine-tuning, it is on Hugging Face only.
An earlier revision of this page claimed the model “achieves 85-90% of 7B performance”, was “optimised for ARM”, and drew 15 watts. None of that is on the model card, and all of it has been removed. What the card does say is narrower and more useful: a 32K context, tool calling, structured JSON output, and the family's improvements in coding and mathematics, which the benchmark section below puts numbers to.
The Qwen 2.5 3B licence: research only, unlike the other sizes
This is the detail most pages about Qwen 2.5 get wrong, including an earlier revision of this one. The announcement post says: “All our open-source models, except for the 3B and 72B variants, are licensed under Apache 2.0.” The 3B ships with the Qwen Research Licence Agreement dated September 19, 2024, and Hugging Face tags the repository license: other, qwen-research.
The agreement grants the right to use, modify and distribute the weights “FOR NON-COMMERCIAL PURPOSES ONLY”, and defines non-commercial as “for research or evaluation purposes only”. For anything else it says: “If you are commercially using the Materials, you shall request a license from us.” Derivative models must carry a “Built with Qwen” or “Improved using Qwen” notice. There is no user-count threshold of the kind Llama's licence has; the restriction applies to everyone.
In practice: running it at home, in a course, or in an evaluation is fine; shipping it inside a product or using it in a business workflow needs Alibaba's permission. If you want a Qwen of this size for commercial work, the 1.5B and 7B are Apache 2.0, and so is every dense Qwen3 model including the 4B; see the last section. The broader picture is in which local models you can legally use at work.
Qwen 2.5 3B benchmarks, as reported by Alibaba
These are the publisher's figures from the Qwen2.5 LLM post, which prints a small-model table for the instruct models and a separate table for the base models. This site has not reproduced them.
| Benchmark | Qwen2.5-3B-Instruct |
|---|---|
| MMLU-Pro | 43.7 |
| MMLU-redux | 64.4 |
| GPQA | 30.3 |
| MATH | 65.9 |
| GSM8K | 86.7 |
| HumanEval | 74.4 |
| MBPP | 72.7 |
| MultiPL-E | 60.2 |
| LiveCodeBench (2305-2409) | 19.9 |
| LiveBench (0831) | 26.8 |
| IFEval (strict prompt) | 58.2 |
For the base model, the same post gives Qwen2.5-3B an MMLU of 65.6, GSM8K of 79.1 and HumanEval of 42.1.
The instruct numbers are where the 3B stands out: 86.7 on GSM8K and 74.4 on HumanEval are high for a model this size, and Alibaba's own framing is that the 3B “outperforms” the comparison models in its table “in mathematics and coding tasks” despite fewer parameters. The weak rows are the ones that need breadth of knowledge or long reasoning: GPQA at 30.3 and LiveBench at 26.8. Treat it as a capable small assistant for short tasks, not a reference.
Ollama tags and install
ollama pull qwen2.5 with no tag downloads the 7B (4.7 GB), not the 3B. Name the size you want.
# Qwen 2.5 3B instruct, q4_K_M, 1.9 GB (the 3B default)
ollama pull qwen2.5:3b
# Smallest file, 1.3 GB
ollama pull qwen2.5:3b-instruct-q2_K
# Higher-quality quantisation, 3.3 GB
ollama pull qwen2.5:3b-instruct-q8_0
# Chat
ollama run qwen2.5:3b- Every 3B tag lists a 32K context on Ollama, matching the model card. Ollama's own default is 4,096 tokens; see the settings section.
- Tags:
3b-instruct-followed by q2_K, q3_K_S, q3_K_M, q3_K_L, q4_0, q4_1, q4_K_S, q4_K_M, q5_0, q5_1, q5_K_S, q5_K_M, q6_K, q8_0 or fp16. Sizes not in the hardware table: q3_K_S 1.5 GB, q3_K_L 1.7 GB, q4_1 2.0 GB, q4_K_S 1.8 GB, q5_0 2.2 GB, q5_1 2.3 GB, q5_K_S 2.2 GB. - Tool calling is supported. The library page shows the tools badge, so agent frameworks that send tool definitions work with it.
- No base tags. There is no
3b-baseon Ollama.
Full tag list: ollama.com/library/qwen2.5/tags. Once pulled, the model is available to any local tool at http://localhost:11434/api/chat.
Qwen 2.5 model sizes: 0.5B, 1.5B, 3B and 7B
| Model | Licence | Ollama tag | Download | Context |
|---|---|---|---|---|
| Qwen 2.5 0.5B | Apache 2.0 | qwen2.5:0.5b | 398 MB | 32K |
| Qwen 2.5 1.5B | Apache 2.0 | qwen2.5:1.5b | 986 MB | 32K |
| Qwen 2.5 3B | Qwen Research (non-commercial) | qwen2.5:3b | 1.9 GB | 32K |
| Qwen 2.5 7B | Apache 2.0 | qwen2.5:7b | 4.7 GB | 32K |
Sizes and contexts are from the Ollama library page; licences from the Qwen2.5 announcement. The 7B is the smallest size that doubles as a general assistant for longer work, and it is Apache 2.0; the 3B is the largest that fits a 4 GB card. The 14B, 32B and 72B exist too (9.0 GB, 20 GB and 47 GB on Ollama), with the 72B under the separate Qwen licence rather than Apache 2.0.
Running Qwen 2.5 3B on small devices
The 3B size draws people who want a model on a Raspberry Pi, a mini PC or a phone. What can be said from the published numbers, without this site having measured anything:
- Raspberry Pi 5 (8 GB). The 1.9 GB file plus a 4K cache fits in memory with the operating system. Ollama has an ARM64 Linux build, so the install is the standard one-line script. Expect a few words per second at best on the Pi's processor; the Raspberry Pi 5 LLM guide covers which sizes are usable there.
- Mini PCs and old laptops. Anything with 8 GB of RAM runs the default tag CPU-only. An integrated GPU does not change the arithmetic; it is the RAM that matters.
- Phones. Ollama has no phone build. The GGUF files can be loaded by llama.cpp-based phone apps on devices with enough free memory, but an earlier revision of this page printed install commands for tools that do not exist, and they have been removed. If you go this route, treat the q4_K_M file at 2.1 GB plus cache as the memory you need free.
- Apple Silicon. A base MacBook with 8 GB of unified memory runs the default tag through Ollama; 16 GB lets you run the q8_0 file with the full 32K context.
None of these configurations has been timed on this site, and tokens-per-second figures that appear elsewhere for this model should be read with the hardware they were measured on.
Settings that matter, and one that does not exist
An earlier revision of this page told readers to export OLLAMA_EFFICIENCY_MODE=1. That variable does not exist; it is absent from the Ollama FAQ and Ollama ignores it. These are the documented settings worth changing for a small chat model:
# Use more of the model's 32K context (Ollama's default is 4,096 tokens)
OLLAMA_CONTEXT_LENGTH=16384 ollama serve
# or per session
ollama run qwen2.5:3b
>>> /set parameter num_ctx 16384
# Keep it loaded between conversations (default: unloaded after 5 minutes idle)
OLLAMA_KEEP_ALIVE=1h ollama serve- Context. Each extra 1,000 tokens costs about 37 MB of cache on this model, so the full 32K adds only 1.2 GB. On a 6 GB card you can afford it; on a 4 GB card, stop at 8K.
- Cache quantisation.
OLLAMA_KV_CACHE_TYPE=q8_0withOLLAMA_FLASH_ATTENTION=1halves the cache, which matters only if you are running the full 32K on a small card. - Two models at once.
OLLAMA_MAX_LOADED_MODELS=2keeps this model and a second one resident, useful on a 12 GB card where the 3B serves quick questions beside a larger model.
Is Qwen 2.5 3B still worth running?
This section is opinion. Alibaba released Qwen3 on April 29, 2025 with dense models at 0.6B, 1.7B, 4B, 8B, 14B and 32B, all under Apache 2.0. The announcement states that “Qwen3-1.7B/4B/8B/14B/32B-Base performs as well as Qwen2.5-3B/7B/14B/32B/72B-Base, respectively”, which by Alibaba's own account makes the Qwen3 1.7B the replacement for this model at a smaller size, and the Qwen3 4B a step up at a slightly larger one. Both come with a commercial licence the 3B never had. The install is in how to run Qwen3 locally.
The case for still pulling the 2.5 3B is narrow: it answers without a thinking phase, so it is quick for short exchanges, and it is a known quantity in older tutorials and tool configurations. For a new install on a 4 GB card, Qwen3 4B at q4 is about the same size with a better licence. For a different family at this size, Phi-3 Mini 3.8B and Llama 3.2 3B are the usual comparisons, and the Qwen 2.5 Coder 1.5B is the Apache 2.0 choice if the job is code.
Frequently asked questions
How do I download Qwen 2.5 3B?
ollama pull qwen2.5:3b fetches the 1.9 GB instruct build. The weights and Qwen's own GGUF files are on Hugging Face under Qwen/Qwen2.5-3B-Instruct and Qwen/Qwen2.5-3B-Instruct-GGUF.
How much RAM does Qwen 2.5 3B need?
About 2.1 GB for the default tag at a 4K context and about 3.3 GB at the full 32K, by the arithmetic above, so an 8 GB machine is comfortable and a 4 GB graphics card is enough.
Can I use Qwen 2.5 3B commercially?
Not under the licence it ships with. The Qwen Research Licence is for research and evaluation only, and commercial use requires a licence from Alibaba Cloud. The 1.5B and 7B, and all of Qwen3, are Apache 2.0.
Does it support tool calling?
Yes. The Ollama library page shows the tools badge for qwen2.5, and the model card lists structured output and tool use among the family's features.
What languages does it speak?
The card says over 29, naming Chinese, English, French, Spanish, Portuguese, German, Italian, Russian, Japanese, Korean, Vietnamese, Thai and Arabic. Quality in the smaller languages is lower than in English and Chinese, as it is for every small model.
How does it compare with the 7B?
The 7B is 4.7 GB on Ollama, needs an 8 GB card, and is Apache 2.0. Alibaba's tables give it higher scores on every benchmark; the exact rows are on the Qwen 2.5 7B page.
Sources
- Qwen2.5-3B-Instruct model card: parameters, architecture, context, languages
- Qwen Research Licence Agreement
- Qwen2.5 announcement: release date, sizes, which sizes are Apache 2.0
- Qwen2.5 LLM post: the benchmark tables
- Qwen3 announcement: the successor models and their licence
- Ollama tag list for qwen2.5
- Qwen's GGUF files: the file sizes in the hardware table
- Ollama FAQ: default context, environment variables
Build Real AI on Your Machine
RAG, agents, NLP, vision, and MLOps - chapters across 25 courses that take you from reading about AI to building AI.
Related small models
Go from reading about AI to building with AI
25 structured courses. Hands-on projects. Runs on your machine. Start free.

Written by
Barsha Mohapatra
Founder and Editor, Local AI Master · Founder, Little AI Master
Founder of Little AI Master and the editor behind every course and guide on Local AI Master. Years of tutoring students taught her where AI education actually fails.
Related Guides
Continue your local AI journey with these comprehensive guides
- PILLARLocal AI Models Directory: Every Model Compared
- DeepSeek V4 Local Setup: 1M Context, MIT (2026)
- models/llama-3-1-8b
- models/deepseek-coder-v2-236b
- Gemini 3.1 Pro Review: 1M Context, MoE Thinking (2026)
- Bark by Suno AI: Open-Source Text-to-Audio Generation Guide
- StarCoder 2 3B – Technical Analysis & Performance Guide
- Nemotron 3.5 Lightning: 30B Agent Model on 32GB of RAM
- models/codestral-22b
- models/mistral-7b
Grab the AI Starter Kit — career roadmap, cheat sheet, setup guide
No spam. Unsubscribe with one click.
Found your model? Now build something with it.
25 hands-on courses — RAG, agents, fine-tuning — all running locally. First chapter free, no card.