Phi-4 Mini: Microsoft's 3.8B Small Language Model

What is Phi-4 Mini and how is it different from Phi-3.5 Mini?

Phi-4 Mini (Phi-4-mini-instruct) is Microsoft's 3.8-billion-parameter dense decoder model, released in February 2025 under the MIT licence with a 128K context. Microsoft's model card names three architectural changes versus Phi-3.5 Mini: a 200K-token vocabulary (200,064 entries), grouped-query attention, and shared input/output embeddings. It was trained on 5 trillion tokens of filtered public documents, synthetic "textbook-like" maths, coding and reasoning data, and chat-format supervised data, with a June 2024 data cutoff. On Microsoft's own table the gains over Phi-3.5 Mini are largest on maths and reasoning: GSM8K 88.6 vs 76.9, MATH 64.0 vs 49.8, BigBench Hard 70.4 vs 63.1, MMLU-Pro 52.8 vs 47.4. MMLU moves from 65.5 to 67.3.

Can Phi-4 Mini replace a 7B or 8B model?

On maths, yes, if you trust Microsoft's table: 64.0 on MATH beats the Llama-3.1-8B (47.6) and Qwen2.5-7B (60.4) columns of that table, and 88.6 on GSM8K is level with Qwen2.5-7B (88.7). On general knowledge, no: MMLU 67.3 sits below Llama-3.1-8B (68.1), Qwen2.5-7B (72.6) and Gemma2-9B (71.3), and GPQA 25.2 is the lowest in the table apart from Qwen2.5-3B. Treat it as a maths-and-reasoning-first small model, not a drop-in for an 8B generalist.

How much VRAM does Phi-4 Mini need?

The Ollama q4_K_M build is a 2.5 GB file; q8_0 is 4.1 GB; fp16 is 7.7 GB. The KV cache adds 128 KB per token of context (32 layers x 8 KV heads x 128 head dimension x 2 (K and V) x 2 bytes), so an 8K context costs about 1 GB and 32K about 4 GB. That arithmetic puts q4_K_M with an 8K context at roughly 4 GB, which fits a 6 GB card; q8_0 with 32K context at about 8.6 GB, which fits a 12 GB card; fp16 with 32K at about 12 GB, which fits 16 GB. On CPU or Apple Silicon, 8 GB of RAM covers the q4_K_M build with a short context.

Is Phi-4 Mini good for edge devices and mobile?

Microsoft lists "memory/compute constrained environments" and "latency bound scenarios" as the model's primary intended uses, and the 2.5 GB q4_K_M file and 128 KB-per-token KV cache are small by current standards. Microsoft also publishes ONNX Runtime builds for the Phi-4 family. We have not measured phone or Raspberry Pi throughput for this model, so this page gives no speed figures for those devices.

Phi-4 Mini vs Llama 3.2 3B vs Qwen2.5 3B: which small model is better?

On Microsoft's table Phi-4 Mini leads Llama-3.2-3B on every listed row (MMLU 67.3 vs 61.8, MATH 64.0 vs 46.7, GSM8K 88.6 vs 75.6, BBH 70.4 vs 55.4) and leads Qwen2.5-3B on all but ARC-C and GPQA, where the three models are within a point or two. On the multilingual rows the picture flips: Phi-3.5 Mini scores higher than Phi-4 Mini on Multilingual MMLU (51.8 vs 49.3), though Phi-4 Mini is far ahead on MGSM (63.9 vs 49.6). Licensing differs too: Phi-4 Mini is MIT, Llama 3.2 uses Meta's community licence, Qwen2.5-3B uses the Qwen Research licence.

How do I run Phi-4 Mini with Ollama?

Run "ollama run phi4-mini". That pulls the default tag (phi4-mini:latest, identical to phi4-mini:3.8b and phi4-mini:3.8b-q4_K_M, 2.5 GB) and opens a chat. The other published tags are phi4-mini:3.8b-q8_0 (4.1 GB) and phi4-mini:3.8b-fp16 (7.7 GB). There is no separate "instruct" tag on Ollama; every tag is the instruct model. The Ollama library lists the q8_0 and fp16 builds with a 4K default context, so set num_ctx yourself if you want more.

Custom Modelfile for Phi-4 Mini

A Modelfile pins a system prompt and sampling parameters so you do not retype them. The parameters below are standard Ollama Modelfile keys; the context value is a choice, not a requirement.

# Save as Modelfile.phi4-maths
FROM phi4-mini

SYSTEM """You are a careful maths tutor. Show each step,
check the arithmetic, and state the final answer on its own line."""

PARAMETER temperature 0.2
PARAMETER num_ctx 8192
PARAMETER top_p 0.9

# Create and run:
# ollama create phi4-maths -f Modelfile.phi4-maths
# ollama run phi4-maths

More Modelfile examples in the complete Ollama guide.

IDE integration (Continue.dev)

Point Continue.dev at the Ollama model for chat and tab completion on a machine that cannot hold a 7B model alongside the editor:

// .continue/config.json
{
  "models": [{
    "model": "phi4-mini",
    "provider": "ollama",
    "title": "Phi-4 Mini"
  }],
  "tabAutocompleteModel": {
    "model": "phi4-mini",
    "provider": "ollama"
  }
}

Other runtimes

• Transformers: load microsoft/Phi-4-mini-instruct directly; the model card's sample uses the chat template with bfloat16 weights (7.7 GB, the same as the Ollama fp16 build).
• ONNX Runtime: Microsoft publishes ONNX builds of the Phi-4 family on Hugging Face for CPU, GPU and mobile targets; the model card's intended-use text names on-device scenarios.
• llama.cpp / GGUF: the Ollama files are GGUF, so any llama.cpp front end loads the same quantizations.
• Phones and single-board computers: the file sizes make it feasible, but we have not measured throughput on those devices and do not quote any here.