Phi-4 14B Local Setup Guide: Microsoft's Reasoning Model on 12GB GPUs and Macs

Phi-4 is Microsoft's December 2024 release of their efficient-LLM family — a 14.7B-parameter model trained on heavily curated synthetic and academic data rather than raw web crawl. The result: on Microsoft's published numbers, a MATH score above Meta's published figure for Llama 3.1 70B, from a model whose default Ollama build is a 9.1 GB file that fits a 12 GB consumer GPU. MIT licensed for unrestricted commercial use. For users who want strong reasoning capability without the VRAM cost of 70B-class models, Phi-4 is one of the best 14B-class options in 2026.

This guide covers the full Phi-4 family (base 14B, mini 3.8B, multimodal 5.6B), setup across Ollama / vLLM / llama.cpp, prompting techniques for reasoning workloads, fine-tuning with QLoRA / Axolotl / Unsloth, and published benchmark scores vs Llama 3.1 8B / Qwen 2.5 14B / DeepSeek-R1, with every speed given as a labelled bandwidth ceiling.

Table of Contents

1. What Phi-4 Is
2. The Phi-4 Family: Base, Mini, Multimodal
3. What Makes Phi-4 Different (Synthetic Data)
4. Hardware Requirements & Quantization
5. Phi-4 vs Llama 3.1 8B vs Qwen 2.5 14B vs DeepSeek-R1
6. Ollama Setup
7. llama.cpp Setup with GGUF
8. vLLM Setup
9. LM Studio / oobabooga / KoboldCpp
10. Phi-4-mini for Edge Devices
11. Phi-4-multimodal for Vision + Audio
12. System Prompts & Sampling
13. Reasoning Tasks: Math, Code, Logic
14. Fine-Tuning with QLoRA
15. Function Calling and Structured Output
16. Tuning Recipes by GPU
17. Published Benchmarks and Speed Ceilings
18. Licensing
19. Troubleshooting
20. FAQ

---

What Phi-4 Is

Phi-4 (microsoft/phi-4 on HuggingFace) is the December 2024 release of Microsoft's Phi family. Architecture: dense decoder-only transformer with 14.7B parameters, 16K context, a ChatML-style chat template. Trained on 9.8T tokens; released December 12, 2024 (per the model card). License: MIT — most permissive of recent major releases.

The Phi family thesis: small, well-trained models on curated data beat larger models on reasoning. Phi-4 is the best demonstration to date.

---

The Phi-4 Family: Base, Mini, Multimodal

For most users the base 14B Phi-4 is the right starting point.

---

What Makes Phi-4 Different (Synthetic Data)

Most LLMs train on web crawl (Common Crawl, Reddit, books). Phi family trains primarily on:

• Synthetic textbooks generated by larger models with verification
• Academic papers with chain-of-thought reasoning
• Curated code from filtered GitHub
• Math problem sets with verified solutions
• Synthetic dialogues for instruction tuning

Smaller training corpus (9.8T tokens per the model card, vs 15T+ for Llama 3.1) but much higher data density. Result: better reasoning per parameter, weaker on raw knowledge breadth (less factual world coverage than Llama).

---

Hardware Requirements & Quantization

File sizes: Ollama's phi4 tag list for Q4_K_M, Q8_0 and FP16; bartowski/phi-4-GGUF on Hugging Face for the other quants. The speed column is a ceiling, not a measurement: NVIDIA lists the RTX 4090 at 1,008 GB/s of memory bandwidth, and a dense model reads its whole weight file once per generated token, so 1,008 ÷ file size is the most it can produce. Real generation lands below that and falls as the context grows. On an RTX 3060 12GB (360 GB/s) the same Q4_K_M file has a ~40 tok/s ceiling; on an RTX 4070 (504 GB/s), ~55.

For most 12-16 GB consumer GPUs: Q5_K_M is the sweet spot. For RTX 4090 24 GB: Q8_0 if you want highest fidelity, or use the freed VRAM for longer context.

---

Phi-4 vs Llama 3.1 8B vs Qwen 2.5 14B vs DeepSeek-R1

Where the numbers come from: Phi-4 and Qwen 2.5 14B Instruct are from Microsoft's Phi-4 model card and technical report (one harness for both); Llama 3.1 8B and 70B Instruct from Meta's Llama 3.1 model card; DeepSeek-R1 from DeepSeek's R1 paper. Different labs use different prompts and shot counts, so compare within a column family, not to the decimal across them. "Not published" means the source for that column gives no figure. The speed row is a bandwidth ceiling (1,008 GB/s ÷ file size), not a measurement.

For everyday math/code/reasoning on a 12-16 GB card: Phi-4 is the strongest scorer per GB on these tables. For competition math: DeepSeek-R1. For long-context: Llama / Qwen. For code specifically: Qwen 2.5 Coder 14B.

---

Ollama Setup

# Pull Phi-4 14B
ollama pull phi4

# Or a specific quant from the Ollama library (q4_K_M is the default)
ollama pull phi4:14b-q8_0

# Q5_K_M is not in the Ollama library; pull the GGUF straight from Hugging Face
ollama pull hf.co/bartowski/phi-4-GGUF:Q5_K_M

# Run
ollama run phi4 "Solve: integral of sin(x)*e^x dx, show work."

Ollama Modelfile customization:

FROM phi4
PARAMETER num_ctx 16384
PARAMETER temperature 0.4
PARAMETER min_p 0.05
PARAMETER repeat_penalty 1.05
SYSTEM """You are a precise reasoning assistant. Show your work step by step."""

---

llama.cpp Setup with GGUF

# Download Q5_K_M from bartowski's quants
huggingface-cli download bartowski/phi-4-GGUF \
    phi-4-Q5_K_M.gguf \
    --local-dir ./models

./llama-cli \
    -m models/phi-4-Q5_K_M.gguf \
    -ngl 999 \
    -c 16384 \
    -fa \
    --temp 0.4 \
    --min-p 0.05 \
    -p "Solve this step by step: ..."

For server mode:

./llama-server -m phi-4-Q5_K_M.gguf -ngl 999 -c 16384 --port 8080

---

vLLM Setup

# BF16 (needs 32+ GB VRAM)
vllm serve microsoft/phi-4 \
    --max-model-len 16384 \
    --gpu-memory-utilization 0.92

# AWQ-INT4 (12+ GB VRAM): pick any phi-4 AWQ build on the Hugging Face hub
vllm serve <hf-user>/phi-4-awq \
    --quantization awq \
    --max-model-len 16384

For high-throughput serving, see vLLM Complete Setup Guide.

---

LM Studio / oobabooga / KoboldCpp

All major desktop UIs support Phi-4:

• LM Studio: search "phi-4" in the model browser, download Q5_K_M, click Load
• oobabooga: download GGUF, place in models/, llama.cpp loader
• KoboldCpp (flags checked against v1.124.2, October 31, 2026): ./koboldcpp --model phi-4-Q5_K_M.gguf --usecuda --contextsize 16384 on NVIDIA, or --usevulkan on AMD/Intel; Metal is automatic on Apple Silicon. --usecuda is the current name (--usecublas still works as an alias), GPU layers default to autofit, and 16384 matches Phi-4's 16K context window.

See text-generation-webui guide and KoboldCpp guide.

---

Phi-4-mini for Edge Devices

Phi-4-mini (3.8B) is designed for tight resource budgets:

ollama run phi4-mini

The one speed we have measured ourselves: phi4-mini (Q4_K_M, 2.49 GB) generated a median 43.7 tok/s on an Apple M3 Pro with 18 GB, Ollama 0.40.13, 2 November 2026, against a 60 tok/s bandwidth ceiling (150 GB/s ÷ 2.49 GB). Other machines will differ.

Use cases:

• Mobile via MLC-LLM (Android / iOS)
• Browser via WebLLM
• Raspberry Pi 5 for offline assistants (CPU-only, so expect single-digit speeds)
• Embedded systems for on-device reasoning

Phi-4-mini is also the member of the family that supports function calling; Ollama lists phi4-mini with the tools capability.

128K context window is unusually long for a 3.8B model — useful for RAG over long documents on edge devices.

---

Phi-4-multimodal for Vision + Audio

from transformers import AutoModelForCausalLM, AutoProcessor

model = AutoModelForCausalLM.from_pretrained(
    "microsoft/Phi-4-multimodal-instruct",
    torch_dtype="auto",
    device_map="cuda",
    trust_remote_code=True,
)

processor = AutoProcessor.from_pretrained("microsoft/Phi-4-multimodal-instruct", trust_remote_code=True)

# Vision input
inputs = processor(
    images=[image],
    text="<|user|>\n<|image_1|>\nDescribe this image.<|end|>\n<|assistant|>",
    return_tensors="pt",
).to("cuda")

# Audio input similarly with audio_1 tag

For pure vision use cases, Llama 3.2 11B Vision and Qwen 2-VL 7B are stronger; Phi-4-multimodal's strength is doing all three modalities in one small model.

---

System Prompts & Sampling

ChatML-style format with Phi-4's separator token (from the model card):

<|im_start|>system<|im_sep|>You are a precise reasoning assistant.<|im_end|>
<|im_start|>user<|im_sep|>[user message]<|im_end|>
<|im_start|>assistant<|im_sep|>

Recommended sampling:

• Reasoning / math / code: temperature 0.0-0.3, min-p 0.05, no top-k
• Chat / general: temperature 0.5-0.7, min-p 0.05
• Creative writing: temperature 0.8-1.0, min-p 0.05

See LLM Sampling Parameters for full sampler reference.

---

Reasoning Tasks: Math, Code, Logic

For best reasoning output, use chain-of-thought prompting:

Solve this step by step. Show your work.

Problem: ...

Solution:

Or system-level CoT:

<|im_start|>system<|im_sep|>For every problem, work through the solution step by step before providing the final answer.<|im_end|>

Phi-4 was trained to produce CoT-style reasoning natively; it usually does this without explicit prompting.

For competition-grade math (AIME, Olympiad), use DeepSeek-R1-Distill variants — they outperform Phi-4 on hard problems with test-time compute.

---

Fine-Tuning with QLoRA

Using Unsloth (fastest):

from unsloth import FastLanguageModel
from datasets import load_dataset

model, tokenizer = FastLanguageModel.from_pretrained(
    model_name="unsloth/phi-4-bnb-4bit",
    max_seq_length=4096,
    load_in_4bit=True,
)

model = FastLanguageModel.get_peft_model(
    model,
    r=32, lora_alpha=32, lora_dropout=0,
    target_modules=["q_proj", "k_proj", "v_proj", "o_proj", "gate_proj", "up_proj", "down_proj"],
)

# Train on your dataset

The 4-bit base fits a single 24 GB card; training time depends on sequence length, epochs and dataset size, so time a short run on your own data first. MIT license means fine-tuned derivatives can be redistributed without restriction.

See LoRA Fine-Tuning Local Guide.

---

Function Calling and Structured Output

The 14B Phi-4 model card documents no native tool-calling format, and Ollama's phi4 build carries no tools capability. If you need function calling, use phi4-mini (Ollama lists it with tools) or serve Phi-4 behind vLLM with a custom tool-call parser and --enable-auto-tool-choice. The request shape is the OpenAI one:

{
  "model": "phi-4",
  "messages": [...],
  "tools": [{"type": "function", "function": {...}}],
  "tool_choice": "auto"
}

For JSON mode constrained output, use response_format:

{"response_format": {"type": "json_schema", "json_schema": {...}}}

vLLM uses xgrammar for guaranteed schema-valid output.

---

Tuning Recipes by GPU

RTX 3060 12GB / RTX 4070 (12 GB)

ollama run phi4:14b-q4_K_M
# Or via Modelfile: Q4_K_M, num_ctx 8192, temperature 0.4

RTX 4070 / 4080 (16 GB)

Q5_K_M, num_ctx 16384.

RTX 4090 / 5090 (24-32 GB)

Q8_0 or FP16, full 16K context, batch multiple users via vLLM.

Mac M2 / M3 / M4

Q4_K_M (phi4:14b, 9.1 GB) via Ollama Metal fits a 16 GB Mac, which lets the GPU address roughly 11 GB. Bandwidth ceilings for the 9.1 GB file, from Apple's published bandwidth: ~13 tok/s on a base M4 (120 GB/s), ~16 on an M3 Pro (150 GB/s), ~22 on an M1/M2 Pro (200 GB/s), ~44 on an M1/M2 Max (400 GB/s).

---

Published Benchmarks and Speed Ceilings

Scores as published by each model's maker (Phi-4 and Qwen 2.5 14B from Microsoft's Phi-4 report; Llama 3.1 8B from Meta's model card). Speeds are bandwidth ceilings for the Q5_K_M GGUF on an RTX 4090 (1,008 GB/s ÷ file size), not runs we did.

Phi-4 reads almost twice the weights per token that Llama 3.1 8B does, so its ceiling is roughly half; on these published scores it answers math and code correctly more often, which is the trade.

---

Licensing

MIT license — most permissive among recent major model releases. You can:

• Use commercially without restriction
• Modify and redistribute
• Bundle into proprietary products
• Sell as paid service
• Train derivative models without restriction

Compare to Llama 3.1 (Meta Llama Community License with monthly active user thresholds), Qwen 2.5 (Tongyi Qianwen License with EU restrictions). For strictest commercial cleanliness, Phi-4 is the safest choice.

---

Troubleshooting

---

FAQ

See answers to common Phi-4 questions below.

---

Sources: Phi-4 on Hugging Face | Phi-4 technical report | Microsoft Research | bartowski quants | Ollama phi4 tags | Llama 3.1 model card | DeepSeek-R1 paper. Speeds are bandwidth ceilings except the one measured phi4-mini row (Apple M3 Pro).

Related guides:

• Best Ollama Models
• Llama 4 Local Setup Guide
• Qwen 3 Local Setup Guide
• DeepSeek R1 Local Setup
• AI Models 16GB RAM
• Choose Right AI Model