Mistral · Open-Weight · Modified MIT
Mistral Medium 3.5: 128B Dense, 4-GPU Open-Weight
Mistral Medium 3.5 (April 30, 2026) is the French AI lab Mistral's unified flagship — 128 billion dense parameters, 256K context, 77.6% SWE-Bench Verified, modified MIT licensed. The big design choice: a single model that handles general reasoning, vision, and coding well, and it took over as the default model in Mistral's Vibe CLI (replacing Devstral 2). Mistral still ships its separate Devstral 2 open-weight coding line alongside it. Runs on 4× H100 at full precision, or 1× H100 / 3× RTX 5090 at Q4 quantization. This is the realistic open-weight choice for prosumer hardware.
Key takeaways
- →128B dense — no MoE complexity, predictable VRAM, simpler deployment.
- →Unified generalist — one model for general reasoning, vision, and coding; now the Vibe CLI default (Devstral 2 still ships separately).
- →77.6% SWE-Bench Verified — competitive with DeepSeek V4-Flash (78.4%).
- →256K context — bigger than most prosumer-tier alternatives.
- →Runs on 1× H100 at Q4 — accessible without an 8× H100 cluster.
Quick verdict
Mistral Medium 3.5 is the right pick when you want a unified general/coding/vision model on prosumer infrastructure. Dense architecture means simpler deployment than DeepSeek V4-Flash's MoE. Single H100 at Q4 makes it viable without cluster-grade hardware. If you used an earlier Mistral Medium: the original Medium (December 2023) and Mistral Medium 3 (May 2025) were offered through Mistral's API and enterprise self-deployment rather than as a public download, so Medium 3.5 is the Medium you can pull and run yourself.
Where it loses: peak coding quality vs Qwen3-Coder-Next (smaller and slightly higher SWE-Bench), 1M context vs DeepSeek V4 (4× longer). For pure coding workloads on a single GPU, Qwen3-Coder-Next or Qwen3.6-27B may be better. For mixed coding + research + vision, Mistral Medium 3.5 is the cleanest single-model option.
Specs at a glance
| Vendor | Mistral AI |
| Architecture | Dense transformer (no MoE) |
| Parameters | 128 billion |
| Context window | 256,000 tokens |
| Modalities | Text · Code · Vision |
| License | Modified MIT |
| Storage (BF16) | ~256 GB |
| Storage (Q4_K_M) | ~80 GB |
| Hugging Face | mistralai/Mistral-Medium-3.5 |
Hardware & setup
| Hardware | Quant (Ollama size) | Context | Bandwidth ceiling |
|---|---|---|---|
| 4× H100 80GB (SXM, 3.35 TB/s each) | BF16 (255 GB) | 256K | ~52 tok/s ceiling (tensor parallel) |
| 1× H100 80GB (SXM, 3.35 TB/s) | Q4_K_M (80 GB) | Short only: the weights leave little room for KV cache | ~42 tok/s ceiling |
| 2× RTX 5090 (32GB each) | Q4_K_M (80 GB) | Does not fit: 80 GB of weights on 64 GB of VRAM | Not applicable; spills to system RAM |
| 3× RTX 5090 (96GB total, 1,792 GB/s each) | Q4_K_M (80 GB) | Reduced | ~22 tok/s ceiling (layer split) |
| 1× M3 Ultra (192GB, 819 GB/s) | Q4_K_M (80 GB) | 256K | ~10 tok/s ceiling |
These are upper bounds, not measurements: generating one token reads every weight once, so memory bandwidth divided by file size caps the speed. Real throughput lands below the ceiling. Bandwidth is from the vendor spec pages (NVIDIA H100 SXM, NVIDIA RTX 5090, Apple Mac Studio); file sizes are from Ollama's mistral-medium-3.5 tag list (checked November 2, 2026). With tensor parallelism each H100 reads a quarter of the weights; with Ollama's layer split the cards run one after another, so one card's bandwidth sets the pace.
Ollama (single-GPU prosumer)
# Q4_K_M, 80 GB download (the default tag)
ollama pull mistral-medium-3.5
ollama run mistral-medium-3.5
# Q8_0, 138 GB
ollama pull mistral-medium-3.5:128b-q8_0vLLM (production, 4× H100)
python -m vllm.entrypoints.openai.api_server \
--model mistralai/Mistral-Medium-3.5 \
--tensor-parallel-size 4 \
--max-model-len 262144 --port 8000Benchmarks
| Benchmark | Mistral Medium 3.5 | DeepSeek V4-Flash | Qwen3-Coder-Next | GLM-5 |
|---|---|---|---|---|
| SWE-Bench Verified | 77.6% | 78.4% | 70.6% | 77.8% |
| MMLU-Pro | 85.2% | 83.8% | 81.4% | 84.6% |
| GPQA Diamond | 76.4% | 76.9% | 71.2% | 79.4% |
| AIME 2025 | 81.6% | 82.4% | 76.8% | 85.2% |
| Vision-MME (image QA) | 73.4% | N/A | N/A | 68.7% |
When to pick Mistral Medium 3.5
- ✓You want one generalist model for general work + coding + vision (vs running Mistral's separate Devstral 2 coding line).
- ✓Dense architecture preference (simpler than MoE — predictable VRAM, no expert routing).
- ✓Single H100 / 3× RTX 5090 hardware (Q4 quantization).
- ✓EU sovereignty matters — Mistral is Paris-based, GDPR-aligned governance.
FAQ
What is Mistral Medium 3.5?
How much VRAM does Mistral Medium 3.5 need?
Mistral Medium 3.5 vs DeepSeek V4-Flash: which to pick?
How do I install Mistral Medium 3.5?
What does the unified design (general + vision + coding) mean?
Why does “modified MIT” license matter?
Related models
- → Mistral Large 123B — predecessor
- → DeepSeek V4 — frontier MoE alternative, 1M context
- → Qwen3-Coder-Next — smaller, coding-specialized
- → Qwen3.6-27B — single-GPU dense alternative
- → GLM-5.2 — 753B MIT-licensed MoE, the biggest open model you can self-host
- → AI model leaderboard — local and frontier models ranked side by side
Go from reading about AI to building with AI
25 structured courses. Hands-on projects. Runs on your machine. Start free.
Written by
Barsha Mohapatra
Founder and Editor, Local AI Master · Founder, Little AI Master
Founder of Little AI Master and the editor behind every course and guide on Local AI Master. Years of tutoring students taught her where AI education actually fails.