★ Reading this for free? Get 25 structured AI courses + per-chapter AI tutor — the first chapter of every course free, no card.Start free in 30 secondsOr own every course: $149 once
QWEN 2.5 7B OLLAMA GUIDE

Qwen 2.5 7B
on Ollama

Size, VRAM, model tags, benchmarks and setup

Updated: November 2, 2026

ollama pull qwen2.5:7b         # 4.7 GB download (Q4_K_M)
ollama run qwen2.5:7b          # needs ~5.7 GB at an 8K context

ollama pull qwen2.5-coder:7b   # coding variant, also 4.7 GB
KEY SPECIFICATIONS (model card):
7.61B
Parameters
29+
Languages
131K
Context (8K generation)
SHORT ANSWER

Qwen2.5 7B is a 4.7 GB download in its default Ollama build (qwen2.5:7b, Q4_K_M) and needs about 5.7 GB of memory at an 8K context (4.7 GB of weights + 0.45 GB of KV cache + runtime overhead), so it fits an 8 GB GPU with room for a 32K context. Install and start it with ollama pull qwen2.5:7b. The model has 7.61B parameters, a 131,072-token context, an Apache 2.0 licence and 29+ languages on its model card. See the full size-and-memory table below; the coding variant qwen2.5-coder:7b is also 4.7 GB.

This page covers the setup questions people search for first (download size, VRAM, the exact tag names), then Qwen's published benchmarks, and finally what replaced the model in 2025.

Technical Specifications

Technical Specifications

Parameters
7.61B total, 6.53B non-embedding (model card)
Architecture
28 layers; RoPE, SwiGLU, RMSNorm, QKV bias; GQA with 28 query heads and 4 KV heads
Context Window
131,072 tokens input, 8,192 generation; config ships at 32,768 and uses YaRN beyond that
Vocabulary
152,064 tokens; input and output embeddings are separate (tie_word_embeddings: false)
Training
Pretrained on up to 18 trillion tokens (Qwen2.5 announcement), then supervised and preference post-training
Languages
Over 29 languages listed on the model card
Licence
Apache 2.0 (the 3B and 72B siblings use different Qwen licences)
Disk
~15 GB in bf16 safetensors; 4.7 GB as the default Ollama Q4_K_M build

Where these come from

Parameter counts, head layout, context and licence are from the Qwen2.5-7B-Instruct model card; hidden size, vocabulary and the 32,768 position limit are from its config.json; the 18-trillion-token figure and the 29-language count are from the Qwen2.5 announcement (September 19, 2024).

Ollama tags and download sizes

Every 7B tag on the qwen2.5 library page is the instruct model. qwen2.5:7b, qwen2.5:7b-instruct and qwen2.5:7b-instruct-q4_K_M are the same 4.7 GB file; qwen2.5:latest also resolves to the 7B Q4_K_M build.

TagDownloadPull command
qwen2.5:7b (= 7b-instruct, latest)4.7 GBollama pull qwen2.5:7b
qwen2.5:7b-instruct-q2_K3.0 GBollama pull qwen2.5:7b-instruct-q2_K
qwen2.5:7b-instruct-q3_K_M3.8 GBollama pull qwen2.5:7b-instruct-q3_K_M
qwen2.5:7b-instruct-q4_04.4 GBollama pull qwen2.5:7b-instruct-q4_0
qwen2.5:7b-instruct-q4_K_M4.7 GBollama pull qwen2.5:7b-instruct-q4_K_M
qwen2.5:7b-instruct-q5_K_M5.4 GBollama pull qwen2.5:7b-instruct-q5_K_M
qwen2.5:7b-instruct-q6_K6.3 GBollama pull qwen2.5:7b-instruct-q6_K
qwen2.5:7b-instruct-q8_08.1 GBollama pull qwen2.5:7b-instruct-q8_0
qwen2.5:7b-instruct-fp1615 GBollama pull qwen2.5:7b-instruct-fp16
qwen2.5-coder:7b (separate model)4.7 GBollama pull qwen2.5-coder:7b

Sizes as listed on ollama.com (qwen2.5 and qwen2.5-coder tag pages). The coder model is covered on its own page: Qwen 2.5 Coder 7B.

Performance Analysis

The table is Qwen's, from the Qwen2.5 LLM post that accompanied the September 2024 release. Every column was run by the Qwen team; we have not re-run it, and the comparison models' own vendors publish different numbers for themselves.

BenchmarkQwen2.5-7B-InstructGemma2-9b-ITLlama3.1-8B-InstructQwen2-7B-Instruct
MMLU-Pro56.352.148.344.1
MMLU-redux75.472.867.267.3
GPQA36.432.832.834.3
MATH75.544.351.952.9
GSM8K91.676.784.585.7
HumanEval84.868.972.679.9
MBPP79.274.969.667.2
MultiPL-E70.453.450.759.1
LiveCodeBench 2305-240928.718.98.323.9
LiveBench 083135.930.626.729.2
IFEval strict-prompt71.270.175.954.7
Arena-Hard52.041.627.825.0
AlignBench v1.17.337.054.757.13
MT-Bench8.758.498.238.26

Source: Qwen team, "Qwen2.5-LLM: Extending the boundary of LLMs" (qwenlm.github.io). Base-model rows from the same post: Qwen2.5-7B MMLU 74.2, BBH 70.4, HumanEval 57.9, MBPP 74.9, MATH 49.8, GSM8K 85.4 (vs Llama3-8B 66.6 / 57.7 / 33.5 / 53.9 / 20.5 / 55.3).

What stands out

  • Maths and code are the big moves. MATH 75.5 against 52.9 for the previous Qwen2 7B; HumanEval 84.8 against 79.9; MultiPL-E 70.4 against 59.1.
  • Instruction following is the one loss. Llama 3.1 8B Instruct scores 75.9 on IFEval strict-prompt to Qwen's 71.2.
  • GPQA is modest for all four. 36.4 is the best of the row, but all four are in the low-to-mid 30s.
  • No throughput figures here. Tokens per second depend on the GPU, quantization and context; this page does not quote speeds it has not measured. The GPU comparison guide covers measured runs on named cards.

Hardware Requirements

Short answer: an 8 GB GPU runs the default build with a 32K context (about 7 GB). A 12 GB card (RTX 3060 12GB, 4070) runs q8_0 at 32K. Only fp16 needs 24 GB. On CPU or Apple Silicon, 16 GB of RAM is the sensible floor for the default build; 8 GB machines should use q2_K or q3_K_M with a short context, or pick a 3B model instead.

How the numbers are worked out

These are arithmetic, not measurements. Weights are the Ollama file sizes above. The KV cache comes from config.json: 28 layers x 4 KV heads x 128 head dimension (3584 hidden / 28 heads) x 2 for keys and values x 2 bytes at fp16 = 57,344 bytes, or 56 KB per token. That is 0.45 GB at 8K context, 1.75 GB at 32K and 7 GB at the full 131K. Add about 0.5 GB for the runtime and compute buffers. Grouped-query attention with only 4 KV heads is why this 7B model's cache is smaller than a 3.8B model with 8 KV heads.

Memory by Quantization Level

QuantizationOllama fileMemory at 8K ctx*Memory at 32K ctx*Fits on
q2_K3.0 GB~4.0 GB~5.3 GB6 GB card (8K); 8 GB card (32K)
q3_K_M3.8 GB~4.8 GB~6.1 GB6 GB card (8K); 8 GB card (32K)
q4_04.4 GB~5.4 GB~6.7 GB8 GB card
q4_K_M (default: 7b, 7b-instruct)4.7 GB~5.7 GB~7.0 GB8 GB card; 16 GB RAM on CPU/Mac
q5_K_M5.4 GB~6.4 GB~7.7 GB8 GB card
q6_K6.3 GB~7.3 GB~8.6 GB8 GB card (8K); 12 GB card (32K)
q8_08.1 GB~9.1 GB~10.4 GB12 GB card
fp1615 GB~16 GB~17.3 GB24 GB card; 32 GB Mac

*File size + KV cache (56 KB/token) + ~0.5 GB overhead, rounded. Ollama's OLLAMA_KV_CACHE_TYPE=q8_0 halves the cache term. The full 131K context on the default build is about 12.2 GB by the same arithmetic. See quantization explained for what the tags mean, or adjust the context in the VRAM calculator.

By card

  • 8 GB (RTX 3050 8GB, 4060, 3070, 4060 Laptop): q4_K_M to 32K; q6_K to about 16K.
  • 12 GB (RTX 3060 12GB, 4070, 5070): q8_0 to 32K; q4_K_M to about 100K.
  • 16 GB (4060 Ti 16GB, 4080, 5080, 5070 Ti): q8_0 at 128K, or fp16 to about 8K.
  • 24 GB (3090, 4090, 5090 32GB): fp16 to 128K.

CPU and Apple Silicon

  • 8 GB RAM / 8 GB Mac: q2_K or q3_K_M at 8K, with 2-3 GB left for the OS. Tight.
  • 16 GB RAM / 16 GB Mac: q4_K_M to q6_K at 32K.
  • 32 GB: any build, any context.

Pick a card on the hardware page.

Installation & Setup

Installation & Setup Guide

System Requirements

  • ✓Ollama (any OS) for the one-command route, or Python 3.9+ with pip for Transformers
  • ✓8 GB VRAM for the default Q4_K_M build on GPU; 16 GB RAM for CPU or Apple Silicon
  • ✓5 GB of free disk for the Ollama build, 15 GB for the Hugging Face safetensors
  • ✓CUDA-capable GPU for Transformers acceleration (optional)
  • ✓git-lfs if you clone the Hugging Face repository instead of using huggingface-cli

Installation Methods

Ollama (recommended)
# Install Ollama (Linux/macOS); Windows has an installer at ollama.com
curl -fsSL https://ollama.com/install.sh | sh

# Pull and run the default 4.7 GB Q4_K_M build
ollama pull qwen2.5:7b
ollama run qwen2.5:7b

# Raise the context for long documents (KV cache grows 56 KB per token)
ollama run qwen2.5:7b --num_ctx 32768
Transformers (from the model card)
pip install torch transformers accelerate

from transformers import AutoModelForCausalLM, AutoTokenizer
model_name = "Qwen/Qwen2.5-7B-Instruct"
model = AutoModelForCausalLM.from_pretrained(
    model_name, torch_dtype="auto", device_map="auto"
)
tokenizer = AutoTokenizer.from_pretrained(model_name)

messages = [
    {"role": "system", "content": "You are Qwen, created by Alibaba Cloud. You are a helpful assistant."},
    {"role": "user", "content": "Give me a short introduction to large language models."},
]
text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer([text], return_tensors="pt").to(model.device)
out = model.generate(**inputs, max_new_tokens=512)
print(tokenizer.decode(out[0][len(inputs.input_ids[0]):], skip_special_tokens=True))

Terminal Commands

# Download only (4.7 GB)
ollama pull qwen2.5:7b

# Chat
ollama run qwen2.5:7b

# Show parameters, quantization, context and licence of the installed build
ollama show qwen2.5:7b

# A specific quantization
ollama pull qwen2.5:7b-instruct-q8_0

# One-shot prompt with a bigger context window
ollama run qwen2.5:7b --num_ctx 32768 "Summarise the following text: ..."

# Remove the model when done
ollama rm qwen2.5:7b

For the HTTP API, Modelfiles and tool calling, see the complete Ollama guide and the tool-calling guide; Windows installation is covered in Ollama on Windows.

Practical Applications

Practical Applications

Multilingual chat and translation

The model card lists support for over 29 languages including Chinese, English, French, Spanish, Portuguese, German, Italian, Russian, Japanese, Korean, Vietnamese, Thai and Arabic.

Notes:
  • • Same weights for every language
  • • Chinese and English are the training-data core
  • • AlignBench (Chinese) 7.33 on Qwen's table
Setup effort:
Low

Structured output and JSON

Qwen's release post singles out improved generation of structured outputs, especially JSON, and better handling of structured data such as tables.

Notes:
  • • JSON mode via the chat template
  • • Tool/function calling in Ollama
  • • IFEval strict-prompt 71.2
Setup effort:
Low to Medium

Coding help (or the Coder sibling)

HumanEval 84.8 and MBPP 79.2 on Qwen's table. For repository-scale work the separate qwen2.5-coder:7b (same 4.7 GB) is the better pick.

Notes:
  • • Multiple languages (MultiPL-E 70.4)
  • • Completion and explanation
  • • Coder variant for IDE use
Setup effort:
Medium

Long-document summarisation

131,072-token context on the card with 8,192-token generation. Past 32K the YaRN setting applies, and the KV cache grows at 56 KB per token.

Notes:
  • • Up to 128K input
  • • Up to 8K output
  • • Plan 7 GB of cache at full context
Setup effort:
Medium to High

Research & Documentation

Official Sources

Primary

Weights and code

Multilingual Capabilities and Chinese Language Support

Chinese and English first

Qwen is built by Alibaba Cloud's Qwen team and Chinese and English are the core of its training data. The model card does not publish a per-language breakdown, so the honest evidence is the Chinese-language benchmark on Qwen's table: AlignBench v1.1 7.33 for Qwen2.5-7B-Instruct against 7.05 for Gemma2-9b-IT and 4.75 for Llama3.1-8B-Instruct. Simplified and Traditional Chinese share one tokenizer and one set of weights.

What that means in practice

For Chinese-to-English and English-to-Chinese work on an 8 GB card, this is the model we would start with among the 2024 generation. We have not run our own translation evaluation and do not claim one.

29+ languages on the card

The model card's list: Chinese, English, French, Spanish, Portuguese, German, Italian, Russian, Japanese, Korean, Vietnamese, Thai, Arabic "and more", for a stated total of over 29. The multilingual coding benchmark on Qwen's table (MultiPL-E, 70.4) covers programming languages, not human ones.

Tokenizer

152,064-entry vocabulary (config.json). Non-Latin scripts cost more tokens per word than English, which matters for the 8,192-token generation cap and the KV-cache arithmetic.

Successor coverage

Qwen3 (April 2025) widened the pretraining corpus to 119 languages and dialects, per the Qwen3 release post. If a language outside the 29 matters to you, start there.

What superseded Qwen2.5 7B

Qwen3, released April 29, 2025 under Apache 2.0, is the direct successor. The release post lists dense models at 0.6B, 1.7B, 4B, 8B, 14B and 32B plus two MoE models, pretraining on about 36 trillion tokens across 119 languages (double Qwen2.5's 18 trillion), and claims that "even a tiny model like Qwen3-4B can rival the performance of Qwen2.5-72B-Instruct". The like-for-like replacement for this page's model is qwen3:8b, a 5.2 GB Ollama download (tag page); qwen3:4b is 2.5 GB.

Reasons to still run Qwen2.5 7B: it has no "thinking" mode to switch off, its 4.7 GB build is half a gigabyte smaller than qwen3:8b, and a large ecosystem of fine-tunes and GGUF conversions targets it. Reasons to move: Qwen3's language coverage and Qwen's own claims about reasoning quality at the same size.

Setup for the successor: How to run Qwen3 locally. For the 2026 generation at a larger size, see Qwen3.8-27B VRAM by quant.

Frequently Asked Questions

What are the minimum hardware requirements for Qwen2.5 7B?

The default Ollama build (qwen2.5:7b, Q4_K_M) is a 4.7 GB file. Add the KV cache (56 KB per token of context, so about 0.45 GB at 8K) and roughly 0.5 GB of runtime overhead and you need about 5.7 GB, which fits an 8 GB GPU. On CPU or Apple Silicon, 16 GB of RAM is the comfortable floor; 8 GB machines can load the q2_K or q3_K_M builds with a short context but leave little for the operating system.

How big is the Qwen2.5 7B download?

It depends on the tag. qwen2.5:7b and qwen2.5:7b-instruct are 4.7 GB (Q4_K_M). The other published builds are 3.0 GB (q2_K), 3.8 GB (q3_K_M), 4.4 GB (q4_0), 5.4 GB (q5_K_M), 6.3 GB (q6_K), 8.1 GB (q8_0) and 15 GB (fp16). The coding variant qwen2.5-coder:7b is also 4.7 GB at its default quantization. The unquantized safetensors on Hugging Face are about 15 GB.

How does Qwen2.5 7B compare to other 7B and 8B models?

On Qwen's own table the 7B instruct model leads Llama 3.1 8B Instruct and Gemma 2 9B IT on 13 of 14 rows; the exception is IFEval strict-prompt, where Llama 3.1 8B scores 75.9 to Qwen's 71.2. The largest gaps are MATH (75.5 vs 51.9 and 44.3) and MultiPL-E (70.4 vs 50.7 and 53.4). Those are Qwen's numbers; Meta and Google publish their own, which are not directly comparable.

Can Qwen2.5 7B run on consumer hardware?

Yes. An RTX 3060 12 GB runs the q8_0 build with a 32K context by the arithmetic on this page; an 8 GB card such as the RTX 4060 runs the default Q4_K_M build with room for a 32K context; a 16 GB Mac runs it on CPU/GPU shared memory. Only the fp16 build needs a 24 GB card.

Does Qwen2.5 7B really have a 128K context?

The model card states a 131,072-token context and 8,192-token generation. The config.json ships with max_position_embeddings set to 32,768; inputs beyond that use YaRN rope scaling, which the card explains how to enable (and warns that static YaRN in vLLM can affect short-text quality). The Ollama build loads with Ollama's default context, so set num_ctx explicitly when you need more. At 128K the KV cache alone is about 7 GB.

What is the difference between qwen2.5:7b and qwen2.5-coder:7b?

qwen2.5:7b is the general instruct model covered on this page. qwen2.5-coder:7b is a separate model from the Qwen2.5-Coder series, continued-pretrained on code; it is the same 4.7 GB at Q4_K_M and has its own page on this site. Use the coder build for completion and repository work, and the general build for chat, writing and multilingual tasks.

Reading now
Join the discussion
🎯
AI Learning Path

Go from reading about AI to building with AI

25 structured courses. Hands-on projects. Runs on your machine. Start free.

Or own it for life — Lifetime $149 $599, pay once

Ready to Go Beyond Tutorials?

25 structured courses with hands-on chapters - build RAG chatbots, AI agents, and ML pipelines on your own hardware.

Barsha Mohapatra

Written by

Barsha Mohapatra

Founder and Editor, Local AI Master · Founder, Little AI Master

Founder of Little AI Master and the editor behind every course and guide on Local AI Master. Years of tutoring students taught her where AI education actually fails.

📅 Published: September 26, 2025🔄 Last Updated: November 2, 2026✓ Manually Reviewed

Compare with Similar Models

Alternatives at nearby sizes

Qwen 2.5 14B

Same family, twice the parameters: ollama pull qwen2.5:14b is a 9.0 GB download at Q4_K_M, before context. The step up if you have a 12 GB or 16 GB card.

→ Qwen picks for 12 GB VRAM

Llama 3.1 8B

The comparison column on Qwen's table: ahead on IFEval (75.9 vs 71.2), behind on MATH, HumanEval and MultiPL-E. Meta community licence.

→ Llama 3.1 8B page

Mistral 7B

Apache 2.0 like Qwen2.5. On Qwen's base-model table Mistral-7B scores 64.2 MMLU and 29.3 HumanEval to Qwen2.5-7B's 74.2 and 57.9.

→ Mistral 7B page

Gemma 7B

Google's first-generation 7B. Its successor Gemma2-9b-IT is the column Qwen compares against (MMLU-Pro 52.1 vs 56.3).

→ Gemma 7B page

Phi-3 Mini

Microsoft's 3.8B option when 4.7 GB is too much; MIT licence. Its successor Phi-4 Mini has its own page.

→ Phi-3 Mini page

Qwen 2.5 3B

Same family at 3B for 6 GB cards and 8 GB laptops. Note the different (Qwen Research) licence.

→ Qwen 2.5 3B page

Which one: on an 8 GB card wanting one general model, Qwen2.5 7B; for code, Qwen 2.5 Coder 7B; for a 2025-or-later model at the same size, qwen3:8b. The model recommender takes your VRAM and returns a shortlist.

Related Guides

Continue your local AI journey with these comprehensive guides

More on Local AI Hardware
See the full AI Hardware Guide 2026 guide.
📚
Free · no account required

Grab the AI Starter Kit — career roadmap, cheat sheet, setup guide

No spam. Unsubscribe with one click.

🎯
AI Learning Path

Found your model? Now build something with it.

25 hands-on courses — RAG, agents, fine-tuning — all running locally. First chapter free, no card.

Or own it for life — Lifetime $149 $599, pay once
Free Tools & Calculators