Qwen 2.5 7B
on Ollama
Updated: November 2, 2026
ollama pull qwen2.5:7b # 4.7 GB download (Q4_K_M)
ollama run qwen2.5:7b # needs ~5.7 GB at an 8K context
ollama pull qwen2.5-coder:7b # coding variant, also 4.7 GBQwen2.5 7B is a 4.7 GB download in its default Ollama build (qwen2.5:7b, Q4_K_M) and needs about 5.7 GB of memory at an 8K context (4.7 GB of weights + 0.45 GB of KV cache + runtime overhead), so it fits an 8 GB GPU with room for a 32K context. Install and start it with ollama pull qwen2.5:7b. The model has 7.61B parameters, a 131,072-token context, an Apache 2.0 licence and 29+ languages on its model card. See the full size-and-memory table below; the coding variant qwen2.5-coder:7b is also 4.7 GB.
This page covers the setup questions people search for first (download size, VRAM, the exact tag names), then Qwen's published benchmarks, and finally what replaced the model in 2025.
Complete Implementation Guide
Technical Overview
Implementation
Resources
Technical Specifications
Technical Specifications
Where these come from
Parameter counts, head layout, context and licence are from the Qwen2.5-7B-Instruct model card; hidden size, vocabulary and the 32,768 position limit are from its config.json; the 18-trillion-token figure and the 29-language count are from the Qwen2.5 announcement (September 19, 2024).
Performance Analysis
The table is Qwen's, from the Qwen2.5 LLM post that accompanied the September 2024 release. Every column was run by the Qwen team; we have not re-run it, and the comparison models' own vendors publish different numbers for themselves.
| Benchmark | Qwen2.5-7B-Instruct | Gemma2-9b-IT | Llama3.1-8B-Instruct | Qwen2-7B-Instruct |
|---|---|---|---|---|
| MMLU-Pro | 56.3 | 52.1 | 48.3 | 44.1 |
| MMLU-redux | 75.4 | 72.8 | 67.2 | 67.3 |
| GPQA | 36.4 | 32.8 | 32.8 | 34.3 |
| MATH | 75.5 | 44.3 | 51.9 | 52.9 |
| GSM8K | 91.6 | 76.7 | 84.5 | 85.7 |
| HumanEval | 84.8 | 68.9 | 72.6 | 79.9 |
| MBPP | 79.2 | 74.9 | 69.6 | 67.2 |
| MultiPL-E | 70.4 | 53.4 | 50.7 | 59.1 |
| LiveCodeBench 2305-2409 | 28.7 | 18.9 | 8.3 | 23.9 |
| LiveBench 0831 | 35.9 | 30.6 | 26.7 | 29.2 |
| IFEval strict-prompt | 71.2 | 70.1 | 75.9 | 54.7 |
| Arena-Hard | 52.0 | 41.6 | 27.8 | 25.0 |
| AlignBench v1.1 | 7.33 | 7.05 | 4.75 | 7.13 |
| MT-Bench | 8.75 | 8.49 | 8.23 | 8.26 |
Source: Qwen team, "Qwen2.5-LLM: Extending the boundary of LLMs" (qwenlm.github.io). Base-model rows from the same post: Qwen2.5-7B MMLU 74.2, BBH 70.4, HumanEval 57.9, MBPP 74.9, MATH 49.8, GSM8K 85.4 (vs Llama3-8B 66.6 / 57.7 / 33.5 / 53.9 / 20.5 / 55.3).
What stands out
- Maths and code are the big moves. MATH 75.5 against 52.9 for the previous Qwen2 7B; HumanEval 84.8 against 79.9; MultiPL-E 70.4 against 59.1.
- Instruction following is the one loss. Llama 3.1 8B Instruct scores 75.9 on IFEval strict-prompt to Qwen's 71.2.
- GPQA is modest for all four. 36.4 is the best of the row, but all four are in the low-to-mid 30s.
- No throughput figures here. Tokens per second depend on the GPU, quantization and context; this page does not quote speeds it has not measured. The GPU comparison guide covers measured runs on named cards.
Hardware Requirements
Short answer: an 8 GB GPU runs the default build with a 32K context (about 7 GB). A 12 GB card (RTX 3060 12GB, 4070) runs q8_0 at 32K. Only fp16 needs 24 GB. On CPU or Apple Silicon, 16 GB of RAM is the sensible floor for the default build; 8 GB machines should use q2_K or q3_K_M with a short context, or pick a 3B model instead.
How the numbers are worked out
These are arithmetic, not measurements. Weights are the Ollama file sizes above. The KV cache comes from config.json: 28 layers x 4 KV heads x 128 head dimension (3584 hidden / 28 heads) x 2 for keys and values x 2 bytes at fp16 = 57,344 bytes, or 56 KB per token. That is 0.45 GB at 8K context, 1.75 GB at 32K and 7 GB at the full 131K. Add about 0.5 GB for the runtime and compute buffers. Grouped-query attention with only 4 KV heads is why this 7B model's cache is smaller than a 3.8B model with 8 KV heads.
Memory by Quantization Level
| Quantization | Ollama file | Memory at 8K ctx* | Memory at 32K ctx* | Fits on |
|---|---|---|---|---|
| q2_K | 3.0 GB | ~4.0 GB | ~5.3 GB | 6 GB card (8K); 8 GB card (32K) |
| q3_K_M | 3.8 GB | ~4.8 GB | ~6.1 GB | 6 GB card (8K); 8 GB card (32K) |
| q4_0 | 4.4 GB | ~5.4 GB | ~6.7 GB | 8 GB card |
| q4_K_M (default: 7b, 7b-instruct) | 4.7 GB | ~5.7 GB | ~7.0 GB | 8 GB card; 16 GB RAM on CPU/Mac |
| q5_K_M | 5.4 GB | ~6.4 GB | ~7.7 GB | 8 GB card |
| q6_K | 6.3 GB | ~7.3 GB | ~8.6 GB | 8 GB card (8K); 12 GB card (32K) |
| q8_0 | 8.1 GB | ~9.1 GB | ~10.4 GB | 12 GB card |
| fp16 | 15 GB | ~16 GB | ~17.3 GB | 24 GB card; 32 GB Mac |
*File size + KV cache (56 KB/token) + ~0.5 GB overhead, rounded. Ollama's OLLAMA_KV_CACHE_TYPE=q8_0 halves the cache term. The full 131K context on the default build is about 12.2 GB by the same arithmetic. See quantization explained for what the tags mean, or adjust the context in the VRAM calculator.
By card
- 8 GB (RTX 3050 8GB, 4060, 3070, 4060 Laptop): q4_K_M to 32K; q6_K to about 16K.
- 12 GB (RTX 3060 12GB, 4070, 5070): q8_0 to 32K; q4_K_M to about 100K.
- 16 GB (4060 Ti 16GB, 4080, 5080, 5070 Ti): q8_0 at 128K, or fp16 to about 8K.
- 24 GB (3090, 4090, 5090 32GB): fp16 to 128K.
CPU and Apple Silicon
- 8 GB RAM / 8 GB Mac: q2_K or q3_K_M at 8K, with 2-3 GB left for the OS. Tight.
- 16 GB RAM / 16 GB Mac: q4_K_M to q6_K at 32K.
- 32 GB: any build, any context.
Pick a card on the hardware page.
Installation & Setup
Installation & Setup Guide
System Requirements
- ✓Ollama (any OS) for the one-command route, or Python 3.9+ with pip for Transformers
- ✓8 GB VRAM for the default Q4_K_M build on GPU; 16 GB RAM for CPU or Apple Silicon
- ✓5 GB of free disk for the Ollama build, 15 GB for the Hugging Face safetensors
- ✓CUDA-capable GPU for Transformers acceleration (optional)
- ✓git-lfs if you clone the Hugging Face repository instead of using huggingface-cli
Installation Methods
Ollama (recommended)
# Install Ollama (Linux/macOS); Windows has an installer at ollama.com
curl -fsSL https://ollama.com/install.sh | sh
# Pull and run the default 4.7 GB Q4_K_M build
ollama pull qwen2.5:7b
ollama run qwen2.5:7b
# Raise the context for long documents (KV cache grows 56 KB per token)
ollama run qwen2.5:7b --num_ctx 32768Transformers (from the model card)
pip install torch transformers accelerate
from transformers import AutoModelForCausalLM, AutoTokenizer
model_name = "Qwen/Qwen2.5-7B-Instruct"
model = AutoModelForCausalLM.from_pretrained(
model_name, torch_dtype="auto", device_map="auto"
)
tokenizer = AutoTokenizer.from_pretrained(model_name)
messages = [
{"role": "system", "content": "You are Qwen, created by Alibaba Cloud. You are a helpful assistant."},
{"role": "user", "content": "Give me a short introduction to large language models."},
]
text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer([text], return_tensors="pt").to(model.device)
out = model.generate(**inputs, max_new_tokens=512)
print(tokenizer.decode(out[0][len(inputs.input_ids[0]):], skip_special_tokens=True))Terminal Commands
# Download only (4.7 GB)
ollama pull qwen2.5:7b
# Chat
ollama run qwen2.5:7b
# Show parameters, quantization, context and licence of the installed build
ollama show qwen2.5:7b
# A specific quantization
ollama pull qwen2.5:7b-instruct-q8_0
# One-shot prompt with a bigger context window
ollama run qwen2.5:7b --num_ctx 32768 "Summarise the following text: ..."
# Remove the model when done
ollama rm qwen2.5:7bFor the HTTP API, Modelfiles and tool calling, see the complete Ollama guide and the tool-calling guide; Windows installation is covered in Ollama on Windows.
Practical Applications
Practical Applications
Multilingual chat and translation
The model card lists support for over 29 languages including Chinese, English, French, Spanish, Portuguese, German, Italian, Russian, Japanese, Korean, Vietnamese, Thai and Arabic.
Notes:
- • Same weights for every language
- • Chinese and English are the training-data core
- • AlignBench (Chinese) 7.33 on Qwen's table
Structured output and JSON
Qwen's release post singles out improved generation of structured outputs, especially JSON, and better handling of structured data such as tables.
Notes:
- • JSON mode via the chat template
- • Tool/function calling in Ollama
- • IFEval strict-prompt 71.2
Coding help (or the Coder sibling)
HumanEval 84.8 and MBPP 79.2 on Qwen's table. For repository-scale work the separate qwen2.5-coder:7b (same 4.7 GB) is the better pick.
Notes:
- • Multiple languages (MultiPL-E 70.4)
- • Completion and explanation
- • Coder variant for IDE use
Long-document summarisation
131,072-token context on the card with 8,192-token generation. Past 32K the YaRN setting applies, and the KV cache grows at 56 KB per token.
Notes:
- • Up to 128K input
- • Up to 8K output
- • Plan 7 GB of cache at full context
Research & Documentation
Official Sources
Primary
- Qwen2.5 announcement (September 19, 2024)
Sizes, 18T-token pretraining, language count, context
- Qwen2.5-LLM post with the benchmark tables
The 7B instruct and base tables reproduced above
- Qwen2.5 Technical Report (arXiv 2412.15115)
December 2024; the pre- and post-training detail
Weights and code
- Qwen2.5-7B-Instruct on Hugging Face
Model card, config.json, chat template, YaRN instructions
- QwenLM/Qwen2.5 on GitHub
Deployment notes for vLLM, SGLang, llama.cpp
- qwen2.5 on Ollama
All tags and sizes
Multilingual Capabilities and Chinese Language Support
Chinese and English first
Qwen is built by Alibaba Cloud's Qwen team and Chinese and English are the core of its training data. The model card does not publish a per-language breakdown, so the honest evidence is the Chinese-language benchmark on Qwen's table: AlignBench v1.1 7.33 for Qwen2.5-7B-Instruct against 7.05 for Gemma2-9b-IT and 4.75 for Llama3.1-8B-Instruct. Simplified and Traditional Chinese share one tokenizer and one set of weights.
What that means in practice
For Chinese-to-English and English-to-Chinese work on an 8 GB card, this is the model we would start with among the 2024 generation. We have not run our own translation evaluation and do not claim one.
29+ languages on the card
The model card's list: Chinese, English, French, Spanish, Portuguese, German, Italian, Russian, Japanese, Korean, Vietnamese, Thai, Arabic "and more", for a stated total of over 29. The multilingual coding benchmark on Qwen's table (MultiPL-E, 70.4) covers programming languages, not human ones.
Tokenizer
152,064-entry vocabulary (config.json). Non-Latin scripts cost more tokens per word than English, which matters for the 8,192-token generation cap and the KV-cache arithmetic.
Successor coverage
Qwen3 (April 2025) widened the pretraining corpus to 119 languages and dialects, per the Qwen3 release post. If a language outside the 29 matters to you, start there.
What superseded Qwen2.5 7B
Qwen3, released April 29, 2025 under Apache 2.0, is the direct successor. The release post lists dense models at 0.6B, 1.7B, 4B, 8B, 14B and 32B plus two MoE models, pretraining on about 36 trillion tokens across 119 languages (double Qwen2.5's 18 trillion), and claims that "even a tiny model like Qwen3-4B can rival the performance of Qwen2.5-72B-Instruct". The like-for-like replacement for this page's model is qwen3:8b, a 5.2 GB Ollama download (tag page); qwen3:4b is 2.5 GB.
Reasons to still run Qwen2.5 7B: it has no "thinking" mode to switch off, its 4.7 GB build is half a gigabyte smaller than qwen3:8b, and a large ecosystem of fine-tunes and GGUF conversions targets it. Reasons to move: Qwen3's language coverage and Qwen's own claims about reasoning quality at the same size.
Setup for the successor: How to run Qwen3 locally. For the 2026 generation at a larger size, see Qwen3.8-27B VRAM by quant.
Frequently Asked Questions
What are the minimum hardware requirements for Qwen2.5 7B?
The default Ollama build (qwen2.5:7b, Q4_K_M) is a 4.7 GB file. Add the KV cache (56 KB per token of context, so about 0.45 GB at 8K) and roughly 0.5 GB of runtime overhead and you need about 5.7 GB, which fits an 8 GB GPU. On CPU or Apple Silicon, 16 GB of RAM is the comfortable floor; 8 GB machines can load the q2_K or q3_K_M builds with a short context but leave little for the operating system.
How big is the Qwen2.5 7B download?
It depends on the tag. qwen2.5:7b and qwen2.5:7b-instruct are 4.7 GB (Q4_K_M). The other published builds are 3.0 GB (q2_K), 3.8 GB (q3_K_M), 4.4 GB (q4_0), 5.4 GB (q5_K_M), 6.3 GB (q6_K), 8.1 GB (q8_0) and 15 GB (fp16). The coding variant qwen2.5-coder:7b is also 4.7 GB at its default quantization. The unquantized safetensors on Hugging Face are about 15 GB.
How does Qwen2.5 7B compare to other 7B and 8B models?
On Qwen's own table the 7B instruct model leads Llama 3.1 8B Instruct and Gemma 2 9B IT on 13 of 14 rows; the exception is IFEval strict-prompt, where Llama 3.1 8B scores 75.9 to Qwen's 71.2. The largest gaps are MATH (75.5 vs 51.9 and 44.3) and MultiPL-E (70.4 vs 50.7 and 53.4). Those are Qwen's numbers; Meta and Google publish their own, which are not directly comparable.
Can Qwen2.5 7B run on consumer hardware?
Yes. An RTX 3060 12 GB runs the q8_0 build with a 32K context by the arithmetic on this page; an 8 GB card such as the RTX 4060 runs the default Q4_K_M build with room for a 32K context; a 16 GB Mac runs it on CPU/GPU shared memory. Only the fp16 build needs a 24 GB card.
Does Qwen2.5 7B really have a 128K context?
The model card states a 131,072-token context and 8,192-token generation. The config.json ships with max_position_embeddings set to 32,768; inputs beyond that use YaRN rope scaling, which the card explains how to enable (and warns that static YaRN in vLLM can affect short-text quality). The Ollama build loads with Ollama's default context, so set num_ctx explicitly when you need more. At 128K the KV cache alone is about 7 GB.
What is the difference between qwen2.5:7b and qwen2.5-coder:7b?
qwen2.5:7b is the general instruct model covered on this page. qwen2.5-coder:7b is a separate model from the Qwen2.5-Coder series, continued-pretrained on code; it is the same 4.7 GB at Q4_K_M and has its own page on this site. Use the coder build for completion and repository work, and the general build for chat, writing and multilingual tasks.
Go from reading about AI to building with AI
25 structured courses. Hands-on projects. Runs on your machine. Start free.
Ready to Go Beyond Tutorials?
25 structured courses with hands-on chapters - build RAG chatbots, AI agents, and ML pipelines on your own hardware.

Written by
Barsha Mohapatra
Founder and Editor, Local AI Master · Founder, Little AI Master
Founder of Little AI Master and the editor behind every course and guide on Local AI Master. Years of tutoring students taught her where AI education actually fails.
Compare with Similar Models
Alternatives at nearby sizes
Qwen 2.5 14B
Same family, twice the parameters: ollama pull qwen2.5:14b is a 9.0 GB download at Q4_K_M, before context. The step up if you have a 12 GB or 16 GB card.
→ Qwen picks for 12 GB VRAMLlama 3.1 8B
The comparison column on Qwen's table: ahead on IFEval (75.9 vs 71.2), behind on MATH, HumanEval and MultiPL-E. Meta community licence.
→ Llama 3.1 8B pageMistral 7B
Apache 2.0 like Qwen2.5. On Qwen's base-model table Mistral-7B scores 64.2 MMLU and 29.3 HumanEval to Qwen2.5-7B's 74.2 and 57.9.
→ Mistral 7B pageGemma 7B
Google's first-generation 7B. Its successor Gemma2-9b-IT is the column Qwen compares against (MMLU-Pro 52.1 vs 56.3).
→ Gemma 7B pagePhi-3 Mini
Microsoft's 3.8B option when 4.7 GB is too much; MIT licence. Its successor Phi-4 Mini has its own page.
→ Phi-3 Mini pageQwen 2.5 3B
Same family at 3B for 6 GB cards and 8 GB laptops. Note the different (Qwen Research) licence.
→ Qwen 2.5 3B pageWhich one: on an 8 GB card wanting one general model, Qwen2.5 7B; for code, Qwen 2.5 Coder 7B; for a 2025-or-later model at the same size, qwen3:8b. The model recommender takes your VRAM and returns a shortlist.
Related Guides
Continue your local AI journey with these comprehensive guides
- PILLARLocal AI Hardware Requirements (2026): Complete Guide
- Best Mac for Local AI: Every Apple Silicon Chip Ranked M1–M6
- Ollama System Requirements: Minimum RAM, GPU, Disk Size
- Run an LLM on Your Phone: Offline AI on Android & iPhone
- Run an LLM on Your Laptop's NPU: What Actually Works
- RTX 5060 Ti 16GB for Local AI (2026): Cheapest New 16GB GPU?
- RTX 3060 12GB vs RTX 4060 8GB for AI: Which Wins?
- Apple M5 for Local AI: MacBook Pro M5 Max 128GB LLM Guide
- Homelab AI Server Build: Used RTX 3090 Budget Guide
- How Much VRAM Do You Need for AI Models? (2026)
Grab the AI Starter Kit — career roadmap, cheat sheet, setup guide
No spam. Unsubscribe with one click.
Found your model? Now build something with it.
25 hands-on courses — RAG, agents, fine-tuning — all running locally. First chapter free, no card.