Qwen 2.5 Math 7B: Benchmarks, VRAM and How to Run It
Alibaba’s Qwen team built Qwen2.5-Math-7B-Instruct for one job: solving English and Chinese math problems step by step. It is one of the LLMs you can run locally, but not from the official Ollama library.
Qwen 2.5 Math 7B at a glance
MATH (chain-of-thought): 83.6, Qwen technical report, Sep 2024
MATH (tool-integrated): 85.3, Qwen model card
GSM8K (chain-of-thought): 95.2, Qwen technical report
Size: ~7.6B parameters; 4.68 GB at Q4_K_M
Context: 4,096 tokens; licence Apache 2.0
Run it: ollama pull hf.co/bartowski/Qwen2.5-Math-7B-Instruct-GGUF:Q4_K_M
What Qwen 2.5 Math 7B is
Qwen2.5-Math was released in September 2024 as base and instruction-tuned models at 1.5B, 7B and 72B, plus a 72B math reward model. The 7B starts from the general Qwen2.5 7B base model and is trained further on what Qwen calls Qwen Math Corpus v2, “over 1T” tokens of math text, much of it synthesised by the earlier Qwen2-Math-72B-Instruct. The Instruct version adds supervised fine-tuning and reinforcement learning (GRPO) guided by that reward model.
It is a specialist. Qwen’s own model card says the series “mainly supports solving English and Chinese math problems through CoT and TIR” and that it does “not recommend using this series of models for other tasks.” Treat it as a math solver and tutor, not a general assistant.
| Detail | Value (from the Hugging Face config and card) |
|---|---|
| Parameters | ~7.6B (15.23 GB of BF16 weights) |
| Layers / hidden size | 28 / 3,584 (FFN 18,944) |
| Attention | 28 query heads, 4 key-value heads (grouped-query) |
| Context | 4,096 tokens |
| Vocabulary | 152,064 |
| Languages | English and Chinese math problems |
| Licence | Apache 2.0 |
Published benchmark scores
All of the scores below come from Table 3 of the Qwen2.5-Math technical report (arXiv:2409.12122, September 2024): zero-shot, greedy decoding, one answer per problem (MMLU STEM is few-shot). They are the authors’ own evaluations; this site has not re-run them.
MATH benchmark, chain-of-thought (Qwen2.5-Math report, Table 3)
| Benchmark | Chain-of-thought | Tool-integrated |
|---|---|---|
| GSM8K | 95.2 | 94.6 |
| MATH | 83.6 | 85.2 (85.3 on the model card) |
| Minerva Math | 37.1 | 39.0 |
| GaoKao 2023 En | 66.8 | 71.4 |
| OlympiadBench | 41.6 | 55.6 |
| College Math | 46.8 | 56.0 |
| MMLU STEM | 71.9 | 70.1 |
| Average | 62.9 | 67.4 |
Source: Qwen2.5-Math technical report, Table 3 (Qwen2.5-Math-7B-Instruct rows).
For context, the same table puts Llama-3.1-8B-Instruct at 76.6 on GSM8K and 47.2 on MATH, and Mathstral-7B-v0.1 at 84.9 and 56.6. On grade-school word problems the gap is modest; on competition-style MATH problems the specialist model is far ahead of general 7-8B models from the same period.
Competition problems: AIME and AMC
Table 5 of the report scores whole competitions. With greedy decoding (one attempt, the setting you get in a normal chat) Qwen2.5-Math-7B-Instruct solved:
- • AIME 2024: 5 of 30 with chain-of-thought, 6 of 30 with tool-integrated reasoning
- • AMC 2023: 25 of 40 with chain-of-thought, 27 of 40 with tool-integrated reasoning
Qwen’s blog adds that the 7B “could even solve up to 21” AIME 2024 problems, but that figure uses many sampled answers ranked by the 72B reward model, a pipeline a single local chat does not reproduce. Olympiad-proof writing is not what these scores measure: AIME and AMC take a single numeric answer.
Chain-of-thought vs tool-integrated reasoning
The model card gives two system prompts. Chain-of-thought (CoT): Please reason step by step, and put your final answer within \boxed{}. Tool-integrated reasoning (TIR): Please integrate natural language reasoning with programs to solve the problem above, and put your final answer within \boxed{}.
TIR has the model write Python for the exact arithmetic, such as roots of a quadratic or eigenvalues of a matrix, which is where CoT tends to slip. The catch for local use: the code has to be executed and its output fed back to the model. Ollama does not run code by itself; Qwen points to its Qwen-Agent framework for a TIR demo. In a plain Ollama chat, use the CoT prompt.
How to run it (not in the Ollama library)
ollama pull qwen2.5-math:7b fails: the official library has no qwen2.5-math model (checked against the Ollama registry on 2 November 2026). Ollama can, however, pull a GGUF straight from Hugging Face. bartowski’s Qwen2.5-Math-7B-Instruct-GGUF repository has every common quantisation.
Install Ollama
Linux one-liner shown; macOS and Windows installers are on ollama.com
Pull the GGUF from Hugging Face
qwen2.5-math is not in the official Ollama library; Ollama can pull bartowski's 4.68 GB Q4_K_M GGUF directly
Ask a question
Qwen's card recommends asking for step-by-step reasoning with the final answer in \boxed{}
With llama.cpp instead of Ollama, the same file loads with llama-cli -hf bartowski/Qwen2.5-Math-7B-Instruct-GGUF:Q4_K_M. In LM Studio, search for “Qwen2.5-Math-7B-Instruct”. To use it with Python and Transformers, the model card’s snippet loads Qwen/Qwen2.5-Math-7B-Instruct (transformers 4.37.0 or later) at BF16, which needs about 15 GB of GPU memory.
System Requirements
File sizes and VRAM
File sizes are from bartowski’s Hugging Face repository (2 November 2026). The KV cache is small because the model has only 4 key-value heads: 28 layers × 4 heads × 128 dimensions × 2 (keys and values) × 2 bytes = 56 KB per token, about 0.23 GB at the full 4,096-token context.
| Quant | File | Fits |
|---|---|---|
Q3_K_M | 3.81 GB | 6 GB card; noticeable quality loss |
Q4_K_M | 4.68 GB | 8 GB card — the usual choice |
Q5_K_M | 5.44 GB | 8 GB card |
Q6_K | 6.25 GB | 8 GB card, little room to spare |
Q8_0 | 8.10 GB | 12 GB card |
f16 | 15.24 GB | 24 GB card |
“Fits” = file + 0.23 GB KV cache + about 1 GB runtime allowance, rounded to the next common card size. Arithmetic, not a measurement. Pull a different quant by changing the suffix, e.g. hf.co/bartowski/Qwen2.5-Math-7B-Instruct-GGUF:Q8_0.
Speed: bandwidth ceilings
This site has not timed Qwen 2.5 Math 7B. Generation of a dense model is capped by memory bandwidth divided by the size of the file read for each token, so for the 4.68 GB Q4_K_M file the ceilings are: ~58 tok/s on an RTX 4060 (272 GB/s), ~77 tok/s on an RTX 3060 12GB (360 GB/s), ~215 tok/s on an RTX 4090 (1,008 GB/s) and ~32 tok/s on an Apple M3 Pro (150 GB/s). Real runs land below the ceiling.
The closest thing we have measured is qwen2.5-coder:7b, which shares the Qwen2.5 7B architecture and the same 4.68 GB Q4_K_M file size: it generated a median 29.0 tok/s on an Apple M3 Pro 18 GB with Ollama 0.40.13 on 2 November 2026. Different weights, same arithmetic, so expect a similar rate on that machine. Math answers are long, since the model writes out every step, so a 500-token solution at that rate takes roughly 17 seconds. On a rented NVIDIA L4 (24 GB, 300 GB/s) the same coder tag measured 52.0 tok/s (Ollama 0.40.13, 3 November 2026; every row is on /benchmarks).
Alternatives that run locally
If you would rather stay inside the official Ollama library: ollama pull mathstral:7b is Mistral’s math-tuned 7B (56.6 on MATH in Qwen’s table), and newer general models with a thinking mode, such as ollama pull qwen3:8b, are worth trying on the same problems; we have no like-for-like MATH score for them from the same report, so compare on your own problems.
| Model | Size | RAM Required | Cost/Month |
|---|---|---|---|
| Qwen 2.5 Math 7B Instruct (Q4_K_M GGUF) | 4.68 GB | ~6 GB | Free (Apache 2.0) |
| Mathstral 7B (mathstral:7b) | 4.1 GB | ~6 GB | Free (Apache 2.0) |
| Qwen 2.5 7B Instruct (qwen2.5:7b) | 4.7 GB | ~6 GB | Free (Apache 2.0) |
| Qwen3 8B (qwen3:8b) | 5.2 GB | ~7 GB | Free (Apache 2.0) |
Sizes from the Ollama registry and Hugging Face, 2 November 2026. “RAM” is file + KV cache + about 1 GB, arithmetic rather than measured. See also Mathstral 7B and Qwen 2.5 7B.
Frequently asked questions
Is Qwen 2.5 Math 7B on Ollama?
Not in the official library: ollama pull qwen2.5-math:7b fails because no such tag exists. Ollama can still run it from Hugging Face with ollama pull hf.co/bartowski/Qwen2.5-Math-7B-Instruct-GGUF:Q4_K_M, which downloads a 4.68 GB Q4_K_M GGUF.
How good is Qwen 2.5 Math 7B?
In Qwen's technical report (arXiv:2409.12122) the Instruct model scores 83.6 on MATH and 95.2 on GSM8K with chain-of-thought, and 85.3 on MATH with tool-integrated reasoning (the model card's figure). In the same table GPT-4o (2024-08-06) scores 81.1 on MATH. These are the authors' own evaluations.
Can it solve olympiad problems?
Some. With greedy decoding it solved 5 of 30 AIME 2024 problems with chain-of-thought and 6 with tool-integrated reasoning, per Table 5 of the report. Qwen reports up to 21 of 30 only when sampling many answers and picking one with its 72B reward model, which is not a setup you get from a single local chat.
How much VRAM does it need?
The Q4_K_M file is 4.68 GB and the KV cache for the full 4,096-token context is about 0.23 GB, so an 8 GB GPU holds it with room to spare. Q8_0 (8.10 GB) wants a 12 GB card; f16 (15.24 GB) a 24 GB card.
Can I use it as a general chatbot or for coding?
Qwen advises against it: the model card says the series mainly supports English and Chinese math problems and that it does not recommend the models for other tasks. For general use pick Qwen 2.5 7B or Qwen3 8B; for code, Qwen 2.5 Coder 7B.
What is the context length?
4,096 tokens. The configuration sets max_position_embeddings to 4,096, and Qwen's blog says the math corpus was trained with a 4K context. Long multi-step proofs can run out of room.
Sources
- Qwen2.5-Math-7B-Instruct model card and config — scope warning, TIR MATH score, prompts, architecture
- Qwen2.5-Math Technical Report (arXiv:2409.12122) — Tables 3 and 5
- Qwen2.5-Math release post (19 Sep 2024) — training corpus, reward-model results
- bartowski/Qwen2.5-Math-7B-Instruct-GGUF — quantised file sizes
Build Real AI on Your Machine
RAG, agents, NLP, vision, and MLOps - chapters across 25 courses that take you from reading about AI to building AI.
Related Guides
Continue your local AI journey with these comprehensive guides
Continue Learning
Go from reading about AI to building with AI
25 structured courses. Hands-on projects. Runs on your machine. Start free.

Written by
Barsha Mohapatra
Founder and Editor, Local AI Master · Founder, Little AI Master
Founder of Little AI Master and the editor behind every course and guide on Local AI Master. Years of tutoring students taught her where AI education actually fails.
- PILLARLocal AI Models Directory: Every Model Compared
- DeepSeek V4 Local Setup: 1M Context, MIT (2026)
- models/llama-3-1-8b
- models/deepseek-coder-v2-236b
- Phi-3 Mini 3.8B: 69% MMLU on 8GB RAM (Microsoft SLM)
- models/gemma-2-9b
- GPT-5.6 Explained: Luna, Terra, Sol Pricing & Local Options
- StarCoder 2 15B – Technical Analysis & Performance Guide
- models/gpt-oss
- models/devstral
Grab the AI Starter Kit — career roadmap, cheat sheet, setup guide
No spam. Unsubscribe with one click.
Found your model? Now build something with it.
25 hands-on courses — RAG, agents, fine-tuning — all running locally. First chapter free, no card.