Phi-3 Mini 3.8B
Microsoft Small AI

Phi-3 Mini hardware requirements

Phi-3 Mini at 4-bit is a 2.2 GB file and needs about 4.8 GB of memory with a 4K context, so it fits a 6 GB graphics card or a CPU-only machine with 8 GB of RAM. The 8-bit file (4.1 GB) wants an 8 GB card, and the unquantised fp16 file (7.6 GB) wants a 12 GB card. A GPU is optional: Microsoft's model card points CPU users at the GGUF files, which is the format Ollama and llama.cpp load.

Sizes are the download sizes on Ollama's phi3 tag list. The memory column is arithmetic, not a measurement: file size, plus the key-value cache, plus a 1 GB allowance for compute buffers. Phi-3 Mini has 32 layers, a hidden size of 3,072 and no grouped-query attention (its config.json lists 32 key-value heads for 32 attention heads), so the cache costs 2 × 32 × 3,072 × 2 bytes = 393 KB per token, or 1.6 GB for a 4,096-token context.

By graphics card

• 4 GB card: the 2.2 GB file loads, but file plus a 4K cache plus buffers is about 4.8 GB, so either shorten the context or let part of the model run on the CPU.
• 6 GB card: q4_0, Q4_K_M and Q5_K_M all fit with the full 4K context.
• 8 GB card: everything up to Q8_0 (about 6.7 GB needed).
• 12 GB card: fp16 (about 10.2 GB needed). At this size a larger model is usually a better use of the card than an unquantised 3.8B one.
• 16 GB and 24 GB cards: Phi-3 Mini leaves most of the memory idle. The 14B Phi-3 Medium is a 7.9 GB download on Ollama, and the VRAM requirements guide lists what else fits each tier.

The 128K variant costs far more memory than its file size suggests

The 128K and 4K files are the same size, but the cache scales with the context you actually allocate. At 393 KB per token, the full 131,072-token window is 51.5 GB of cache on top of the model. A 16K context is about 6.4 GB and a 32K context about 12.9 GB. Ollama's context-length documentation says it defaults to a 4K context on machines with under 24 GiB of VRAM, so a default phi3:mini session uses the small figure in the table above unless you raise it. The page for the Phi-3 Mini 128K variant covers the long-context build, and quantization explained covers what the q4, q5 and q8 names trade away.

Download Phi-3 Mini: Ollama tags and official files

There are three official places to get the weights: Ollama's library, Microsoft's Hugging Face repositories, and Microsoft's own GGUF repository. All tags below were checked against the Ollama tag list on 29 September 2026.

Ollama

# 2.2 GB, 4-bit. phi3, phi3:latest, phi3:mini and phi3:instruct are the same build
ollama pull phi3:mini
ollama run phi3:mini

# The 4K build, named explicitly (2.4 GB, Q4_K_M)
ollama pull phi3:mini-4k

# Higher precision for an 8 GB card (4.1 GB)
ollama pull phi3:3.8b-mini-4k-instruct-q8_0

One detail worth knowing: on the tag list, phi3:mini carries the same ID (4f2222927938) as phi3:mini-128k and phi3:3.8b-mini-128k-instruct-q4_0, so the short tag is the 128K build even though the library's readme lists it under 4K. If you specifically want Microsoft's 4K weights, pull phi3:mini-4k, which is the Q4_K_M file at 2.4 GB.

Hugging Face

The model card also links ONNX builds for ONNX Runtime, including DirectML on Windows, which is the route Microsoft documents for phones and machines without an NVIDIA card. If you have no GPU at all, the CPU-only guide and the list of models for 8 GB of RAM cover the alternatives at this size.

Phi-3 Mini 3.8B model card: the official facts

Everything in this table is taken from Microsoft's Phi-3-mini-4k-instruct model card, its config.json, or the Phi-3 technical report.

The prompt format is a simple tagged chat template. Ollama applies it for you; you only need it when calling the weights directly:

<|system|>
You are a helpful assistant.<|end|>
<|user|>
Question?<|end|>
<|assistant|>

Two limits are stated on the card itself. First, most of the code in the training data is Python using common packages, and Microsoft recommends manually checking any generated code that uses other packages or other languages. Second, the model is too small to hold much world knowledge, which shows up as a low TriviaQA score in the table below.

Phi-3 Mini MMLU score and other reported benchmarks

Microsoft reports three MMLU figures for Phi-3 Mini: 69% in the technical report's abstract, 68.8 for the original weights and 70.9 for the June 2024 update on the model card. The abstract also gives 8.38 on MT-bench. These are Microsoft's numbers from Microsoft's evaluation pipeline; this site has not run them.

The table is copied from the benchmarks section of the model card. Microsoft notes that all columns were produced with the same pipeline and “might differ from other published numbers”. The Mistral 7B column is v0.1.

Read across the rows rather than down them. On Microsoft's table Phi-3 Mini is ahead of the two 7B models on reasoning, maths and code, roughly level with Llama 3 8B, and behind GPT-3.5 Turbo on 14 of the 18 benchmarks listed. The one place it clearly loses to the 7B models is factual recall (TriviaQA 61.4 against 75.2 for Mistral 7B), which is the card's own caveat about model size. If you compare against Mistral 7B, note that it needs roughly twice the download for that recall advantage.

What replaced Phi-3 Mini

Phi-3 Mini is no longer Microsoft's current small model. Two same-size successors exist, both MIT-licensed, and the Phi-3 Mini model cards now open with banners linking to Phi-3.5 and Phi-4.

For a new install, pull Phi-4 Mini instead. It is 0.3 GB larger on disk, runs on the same hardware, and adds function calling and broader language coverage. Grouped-query attention also matters for memory: it shrinks the per-token cache that makes Phi-3 Mini's 128K window so expensive. On the Phi-4 Mini model card Microsoft reports MMLU 67.3 for Phi-4 Mini against 65.5 for Phi-3.5 Mini; that table comes from a different internal platform from the Phi-3 card's 70.9, so the two should not be compared directly. The Phi-4 local setup guide walks through the install.

Reasons to stay on Phi-3 Mini are narrow: you already have a fine-tune or a prompt set built on it, you need the exact weights for reproducibility, or a tool you depend on pins the phi3 tag. For everything else at this size, the small language models guide compares the current options.

Frequently asked questions

Sources