Phi-3.5 Mini (3.8B): Specs, Benchmarks, RAM Requirements and Ollama Setup

Technical Specifications

Everything in this table is from the official model card and its config.json. The architecture is the plain Phi-3 Mini one: no mixture of experts, no grouped-query attention (32 query heads and 32 key-value heads), a 32,064-token vocabulary shared with Phi-3 Mini, and LongRoPE scaling that extends the 4,096-token training length to 131,072.

Phi-3.5 Mini RAM Requirements on Ollama

Phi-3.5 Mini needs about 3.7 GiB of memory at Ollama's default 4K context (2.2GB of weights plus 1.5 GiB of KV cache), about 5.2 GiB at 8K, and about 14 GiB at 32K. The weights are small; the cache is not, because this model has no grouped-query attention. Each token keeps 2 × 32 layers × 32 KV heads × 96 dimensions × 2 bytes = 393,216 bytes, or 384 KiB, at f16. That is more cache per token than Llama 3.3 70B (320 KiB), and it is why the advertised 128K context would need 48 GiB on a model that downloads in 2.2GB.

Arithmetic, not measurement: KV = 384 KiB × tokens; totals add the Ollama download size and leave out the runtime's own few hundred MB. OLLAMA_KV_CACHE_TYPE=q8_0 roughly halves the cache column and q4_0 quarters it (Ollama FAQ).

What that means by machine:

• 4GB RAM (older laptop, Raspberry Pi 4): the 2.2GB q4_0 build with a 2K context is the only configuration that leaves headroom for the OS. Usable for short Q&A; our Raspberry Pi 5 guide covers the 8GB Pi.
• 8GB RAM: q4_0 at 8K–12K, or q8_0 at 4K–8K. This is the typical Phi-3.5 Mini machine.
• 16GB RAM or an 8GB GPU: q8_0 at 16K, or q4_0 at 32K with a q8_0 KV cache. An 8GB card holds the 2.2GB weights plus a 12 GiB cache only if the cache is quantised, so set both.
• 24GB+ GPU: 32K at f16 with room to spare; 64K with a q8_0 cache. The full 128K is a 48 GiB cache and belongs on a workstation, not a desktop.

The same arithmetic for every tag: file sizes are from the phi3.5 tag page.

The tag page lists the q4_0 build with a 128K context declared and the other quant tags with 4K; set num_ctx explicitly either way. See the quantisation guide for what each level costs in quality.

Performance Analysis: Speed You Can Predict

We have not benchmarked Phi-3.5 Mini on named hardware, so this page prints no measured tokens-per-second. What it can give you is the ceiling, because generating a token means reading every weight from memory once, which makes decoding memory-bandwidth bound: tokens/sec ≤ memory bandwidth ÷ bytes per token. With the 2.2GB q4_0 build that is:

• Dual-channel DDR4-3200 laptop or desktop (51.2 GB/s from the spec: 3,200 MT/s × 8 bytes × 2 channels): 51.2 ÷ 2.2 ≈ 23 tok/s ceiling on CPU alone.
• Dual-channel DDR5-5600 (89.6 GB/s): ≈ 40 tok/s ceiling.
• Raspberry Pi 5 (LPDDR4X-4267 on a 32-bit bus, ≈ 17 GB/s): ≈ 8 tok/s ceiling.
• Apple M3 Pro (Apple lists 150 GB/s): ≈ 68 tok/s ceiling.
• RTX 3060 12GB (360 GB/s): ≈ 160 tok/s ceiling; RTX 4090 (1,008 GB/s): ≈ 450 tok/s, at which point compute and overhead, not bandwidth, set the real pace.

Real throughput lands below every one of those numbers once attention over the cache, sampling and the runtime are counted, and CPU inference in particular rarely reaches half the bandwidth ceiling. Use the figures to rule configurations in or out, not as a promise: a 3.8B model on an ordinary laptop is comfortably interactive for chat, and the Pi 5 figure is the reason people run it there at all. What the model card does say about speed is architectural: a dense 3.8B model with no expert routing has a predictable cost per token, and the 128K extension does not change the per-token weight traffic, only the cache.

Installation Guide

Ollama is the quickest route. ollama run phi3.5, ollama run phi3.5:3.8b and ollama run phi3.5:latest all pull the same 2.2GB q4_0 build. The steps below are the whole process; the only decision is the context size, which the RAM table above settles.

For a specific quantisation, name the tag: ollama pull phi3.5:3.8b-mini-instruct-q8_0 (4.1GB) is the one to pick on a 16GB machine where quality matters more than the extra 2GB. Inside a chat, /set parameter num_ctx 8192 changes the context for that session; the API takes num_ctx in options. Outside Ollama, the card documents Hugging Face Transformers (with trust_remote_code on older versions), and Microsoft publishes ONNX builds of the Phi-3 family for ONNX Runtime on CPU, GPU and mobile.

Hardware Requirements

Minimum: a 64-bit machine with 4GB of free RAM for the q4_0 build at a 2K context. Comfortable: 8GB of RAM, any CPU from the last decade, no GPU. Fast: any GPU with 4GB or more of VRAM, which holds the 2.2GB weights and a 4K cache entirely. Nothing here is from the model card, which does not publish hardware requirements; it is the RAM arithmetic above with the operating system's own needs added.

Storage is the download size plus a little: 2.2GB for the default tag, 7.6GB for fp16. Network is needed once, for the pull; after that the model runs with no connection. For a machine-by-machine view see the hardware requirements tutorial and Ollama system requirements.

Benchmark Results (Model Card)

On Microsoft's own benchmark table, Phi-3.5 Mini averages 61.4 across its suite, level with Mistral Nemo 12B (61.3) and Llama 3.1 8B (61.0), behind Gemma 2 9B (63.3) and well behind GPT-4o mini (74.9). These are Microsoft's numbers, run with Microsoft's prompts; the card notes that other evaluations of the same models can differ (Meta's own card, for instance, gives Llama 3.1 8B a HumanEval of 72.6 where Microsoft's run gives 66.5). The rows below are the ones most people ask about.

Source: microsoft/Phi-3.5-mini-instruct model card, benchmark section. Evaluated by Microsoft with its own prompts and settings.

Multilingual

The card reports Multilingual MMLU of 55.4 (Mistral 7B 47.4, Llama 3.1 8B 56.2, GPT-4o mini 72.9) and MGSM of 47.9 (Mistral 7B 31.8, Llama 3.1 8B 56.7, GPT-4o mini 81.7). Multilingual support was the headline change from Phi-3 Mini, and these numbers put it roughly level with Llama 3.1 8B on knowledge and behind it on multilingual maths.

Long context (RULER)

The card also publishes RULER results, which test whether the advertised window is usable. At 128K Phi-3.5 Mini scores 63.6 against 77.0 for Llama 3.1 8B and 19.0 for Mistral Nemo 12B; averaged over all tested lengths the three score 84.1, 88.3 and 66.2. The practical reading: the 128K window works, with degraded recall at the far end, and the cost of using it is the 48 GiB cache in the RAM section, not the quality.

There is no SWE-bench or Aider figure on the card. Phi-3.5 Mini is not an agentic coding model; for that class see our best local AI coding models ranking.

Model Comparison: Phi-3.5 Mini vs Phi-3, Phi-2 and Phi-4 Mini

Within the family, Phi-3.5 Mini sits between Phi-3 Mini (same size, 4K/128K variants, April–June 2024) and Phi-4 Mini (same size, February 2025), and the newer model is better on every row Microsoft publishes for both. Each figure below is from the model's own card; the Ollama column is from the tag pages.

Phi-3.5 Mini vs Phi-3 Mini: the same 3.8B architecture and tokenizer; the 3.5 release added the 23-language training mix and a 128K window on the default build (Phi-3 Mini shipped separate 4K and 128K variants). On the card's HumanEval row the newer model gains about five points (62.8 vs 57.3), while MMLU is slightly lower (69.0 vs 70.9), which is the usual trade when a model is retrained for breadth.

Phi-3.5 Mini vs Phi-4 Mini: Phi-4 Mini's card re-ran Phi-3.5 Mini under its own harness and reports MMLU 67.3 vs 65.5, GSM8K 88.6 vs 76.9 and an overall 63.5 vs 60.5 in its favour. It also switched to grouped-query attention and a 200K-token vocabulary, which cuts the KV cache per token substantially and improves tokenisation for non-English text. For a new deployment there is no reason to pick 3.5 over 4 Mini unless you need the specific 3.5 weights for reproducibility; the Ollama download is 2.5GB vs 2.2GB.

Phi-3.5 Mini vs Llama 3.1 8B: Microsoft's table has them level on average (61.4 vs 61.0) with Llama ahead on HumanEval and multilingual maths and Phi ahead on GSM8K; Llama is twice the download (4.9GB) and has a third of the KV cost per token thanks to GQA, so on an 8GB machine it is the one that can afford a longer context. Our Llama 3.1 8B page has the details.

Phi-3.5 Alternatives: What Replaced It

The direct replacement is Phi-4 Mini (ollama run phi4-mini, 2.5GB, 128K, MIT), which keeps the size and licence and improves the scores and the memory profile. Beyond the Phi family, the 2026 small-model field for the same 2–5GB budget:

• Phi-4 Mini — the successor; same 3.8B, GQA, function calling supported on Ollama
• Llama 3.2 3B — Meta's small instruct model, 128K context, community licence; the usual comparison point for general chat at this size
• Qwen2.5-Coder 1.5B / 3B — if the job is code completion rather than chat, these are trained for it and Phi-3.5 Mini is not
• Gemma 3 270M — for the opposite extreme: a model that fits where even Phi-3.5 Mini does not
• Phi-3 Medium 14B — the bigger sibling (phi3:medium, 7.9GB) when 16GB of RAM is available and Phi-style reasoning is wanted

The wider list of small models, with sizes and what each is for, is on our small language models guide and the best local AI models for 8GB RAM page.

Recommended Use Cases

The model card lists the intended uses plainly: general-purpose AI systems in memory- or compute-constrained environments, latency-bound scenarios, and tasks needing strong reasoning, especially code, maths and logic, in the 23 supported languages. In practice that is:

Research & Documentation

The primary document is the Phi-3 Technical Report (arXiv 2404.14219, first posted April 2024; its August 2024 revision adds the Phi-3.5 models). The report describes phi-3-mini as "a 3.8 billion parameter language model trained on 3.3 trillion tokens" whose quality "rivals that of models such as Mixtral 8x7B and GPT-3.5" while being "small enough to be deployed on a phone", and introduces phi-3.5-mini, phi-3.5-MoE (16 × 3.8B experts, 6.6B active) and phi-3.5-vision. The training recipe, heavily filtered web data plus synthetic "textbook-quality" data, is the family's defining idea and is what the model card means by its data description.

Frequently Asked Questions

Resources & Further Reading