Yi-6B by 01.AI: Bilingual Chinese-English Model
Yi-6B is a 6-billion parameter language model from 01.AI (founded by Kai-Fu Lee), released in November 2023. It was one of the first Chinese-developed open-source models to achieve competitive English benchmarks while excelling at Chinese. With 63.2 on MMLU and 72.0 on C-Eval in 01.AI's technical report, it punched well above its weight class at launch. On Ollama, ollama run yi:6b is a 3.5 GB download that needs about 5 GB of memory.
Note (March 2026): Yi-6B was impressive at its November 2023 release, but newer models like Qwen 2.5 7B (74% MMLU) and Gemma 2 2B now offer better performance. Yi-6B remains a solid choice for Chinese-focused tasks on constrained hardware.
Benchmarks 01.AI Published
Yi-6B's scores below come from 01.AI's technical report (Tables 2 and 8). The other models' MMLU figures are each from their own developer's report. At launch, Yi-6B outperformed Llama 2 7B on these benchmarks despite having fewer parameters.
MMLU (5-shot), developer-reported
C-Eval, base models (Yi report, Table 2)
Performance Metrics
Benchmark Details
| Benchmark | Yi-6B | Source |
|---|---|---|
| MMLU (5-shot) | 63.2% | 01.AI report |
| C-Eval (5-shot) | 72.0% | 01.AI report |
| CMMLU (Chinese) | 75.5% | 01.AI report |
| HellaSwag | 74.4% | 01.AI report |
| ARC-Challenge | 50.3% | 01.AI report |
| TruthfulQA | Not published | Not in the 01.AI report |
| Winogrande | 71.3% | 01.AI report |
| GSM8K | 32.5% | 01.AI report |
Note: GSM8K at 32.5% shows the limit of a 2023 6B base model on multi-step maths. All figures are 01.AI's, from its own evaluation setup.
VRAM Requirements by Quantization
Yi-6B is a small model that runs comfortably on consumer hardware. The Q4_K_M quantization is the recommended balance of quality and size. File sizes are the Ollama registry's for the yi:6b-* tags; the VRAM column adds 0.27 GB of KV cache for the full 4K context (Yi-6B uses grouped-query attention with 4 key-value heads) and about 1 GB of buffers. It is arithmetic, not a measurement.
| Quantization | File Size | VRAM (GPU) | RAM (CPU) | Quality Loss | Recommendation |
|---|---|---|---|---|---|
| Q2_K | 2.6GB | ~3.9GB | ~6GB | High | Only for extreme constraints |
| Q4_K_M | 3.7GB | ~5.0GB | ~8GB | Minimal | Recommended |
| Q5_K_M | 4.3GB | ~5.6GB | ~8GB | Very Low | Good balance if you have 8GB VRAM |
| Q8_0 | 6.4GB | ~7.7GB | ~12GB | Negligible | Near-lossless, 8GB+ GPU |
| FP16 | 12.1GB | ~13.4GB | ~16GB | None | Full precision, 16GB+ GPU |
RAM (CPU) is the total system memory to plan for, including the operating system.
Architecture & the Yi Model Family
Technical Specifications
Yi Model Family
01.AI released Yi models in two sizes sharing the same architecture. The 6B variant uses the same Llama-style transformer as Yi-34B but with fewer layers (32 vs 60) and a smaller hidden dimension.
Yi-6B (this page)
6B params, 32 layers. Runs on 6GB RAM. Best for edge/constrained deployment and Chinese-focused tasks.
Yi-34B
34B params, 60 layers. 76.3 MMLU (01.AI report). Significantly stronger; yi:34b is a 19.5 GB download.
Yi-1.5 Series (2024)
Improved training data and alignment. Yi-1.5 6B and 9B variants with better overall performance.
Key Design Choices
- -- Large vocabulary (64K) for efficient Chinese tokenization
- -- Grouped Query Attention (GQA) for memory efficiency
- -- RoPE with an adjusted base frequency; the 200K variants were further pretrained on long-context data
- -- Trained on 3.1T tokens of curated English + Chinese data
Chinese Language Capabilities
Yi-6B's main differentiator at launch was its strong bilingual performance. With 72% on C-Eval (a comprehensive Chinese exam benchmark), it significantly outperformed Western models of similar size on Chinese tasks.
Chinese Strengths
- +C-Eval 72%: Strong Chinese exam performance, outperforming many 7B+ models at launch
- +Large CJK vocabulary: 64K vocab with good Chinese character coverage
- +Simplified + Traditional: Handles both character sets
- +Bilingual training: Balanced English-Chinese data mix
Limitations
- !Base model: Yi-6B (base) may not follow instructions well without fine-tuning. Use Yi-6B-Chat for conversational use.
- !Surpassed by Qwen 2.5: Qwen 2.5 7B now scores higher on both Chinese and English benchmarks
- !6B limitation: Complex reasoning and multi-step tasks limited compared to 13B+ models
- !GSM8K 32.5%: Weak at multi-step maths, by 01.AI's own numbers
Ollama Setup Guide
System Requirements
Install Ollama
Download and install the Ollama runtime
Pull Yi-6B
Download the default 4-bit build (3.5 GB)
Run Yi-6B
Start an interactive session
Test Chinese capabilities
Try a bilingual prompt to test Chinese understanding
Terminal Demo
Available Ollama Tags
| Command | Variant | Size |
|---|---|---|
| ollama run yi:6b | Default 4-bit chat build (same file as yi:6b-chat and yi:6b-v1.5) | 3.5GB |
| ollama run yi:6b-chat | Same file as yi:6b | 3.5GB |
| ollama run yi:6b-200k | 200K-context base model, 4-bit | 3.5GB |
| ollama run yi:34b | Yi-34B, 4-bit | 19.5GB |
Note: in the Ollama registry yi:6b, yi:6b-chat and yi:6b-v1.5 share one file, and Ollama's library page describes the family as Yi 1.5, so the default pull is the Yi 1.5 chat build rather than the November 2023 Yi-6B. Sizes read from the registry on 2 November 2026.
Local Model Comparison
All models shown below are free, open-weight, and runnable locally. MMLU is each developer's own 5-shot figure. Speeds are bandwidth ceilings on an RTX 4060 (272 GB/s ÷ file size), not measurements; real runs land below them.
| Model | Download | Memory | Speed | MMLU | Licence |
|---|---|---|---|---|---|
| Yi-6B (yi:6b, 4-bit) | 3.5 GB | ~5 GB | ~78 tok/s ceiling | 63.2 (01.AI) | Apache 2.0 |
| Qwen 2.5 7B (qwen2.5:7b, Q4_K_M) | 4.7 GB | ~6 GB | ~58 tok/s ceiling | 74.2 (Qwen, base) | Apache 2.0 |
| Gemma 2 2B (gemma2:2b-instruct-fp16) | 5.2 GB | ~6.5 GB | ~52 tok/s ceiling | 51.3 (Google) | Gemma Terms |
| Llama 2 7B (llama2:7b, 4-bit) | 3.8 GB | ~7 GB | ~71 tok/s ceiling | 45.3 (Meta) | Llama 2 Community |
| Mistral 7B (mistral:7b, Q4_K_M) | 4.4 GB | ~6 GB | ~62 tok/s ceiling | 60.1 (Mistral AI) | Apache 2.0 |
When to Choose Yi-6B
Choose Yi-6B when:
- -- Chinese language is your primary need
- -- You need Apache 2.0 licensing
- -- You have under 6GB VRAM
- -- Fine-tuning for Chinese-specific tasks
Choose Qwen 2.5 7B instead when:
- -- You want the best Chinese + English performance
- -- You have 7GB+ VRAM available
- -- You need coding capabilities
- -- You want the most current model
Choose Mistral 7B instead when:
- -- English-only tasks
- -- Maximum community/ecosystem support
- -- Sliding window attention needed
- -- Broad fine-tune availability
Honest Assessment & Alternatives
The Bottom Line
Yi-6B was a milestone model when it launched in November 2023 — it proved that Chinese AI labs could produce competitive open-source models with strong bilingual capabilities. Its 63.2 MMLU and 72.0 C-Eval were outstanding for a 6B model at the time.
However, the LLM landscape has moved quickly. As of March 2026, several newer models offer better performance in the same resource envelope:
| Model | MMLU | Chinese | VRAM (Q4) | License |
|---|---|---|---|---|
| Yi-6B | 63.2% | Strong | ~5GB | Apache 2.0 |
| Qwen 2.5 7B | 74% | Excellent | ~5GB | Apache 2.0 |
| Gemma 2 2B | 51% | Limited | ~2GB | Gemma Terms |
| Llama 3.2 3B | 63% | Basic | ~2.5GB | Meta License |
Our recommendation: For Chinese-focused tasks in 2026, Qwen 2.5 7B is the stronger choice. Yi-6B remains worth considering if you specifically need Apache 2.0 licensing, are fine-tuning on Chinese data, or are already invested in the Yi ecosystem.
Build Real AI on Your Machine
RAG, agents, NLP, vision, and MLOps - chapters across 25 courses that take you from reading about AI to building AI.
Yi-6B Architecture Overview
Yi-6B transformer architecture with 32 layers, GQA attention, RoPE positional encoding, and 64K vocabulary optimized for bilingual Chinese-English processing
Go from reading about AI to building with AI
25 structured courses. Hands-on projects. Runs on your machine. Start free.
Written by
Barsha Mohapatra
Founder and Editor, Local AI Master · Founder, Little AI Master
Founder of Little AI Master and the editor behind every course and guide on Local AI Master. Years of tutoring students taught her where AI education actually fails.
Related Guides
Continue your local AI journey with these comprehensive guides
Continue Learning
Explore related models in the small-to-medium bilingual category:
- PILLARLocal AI Models Directory: Every Model Compared
- DeepSeek V4 Local Setup: 1M Context, MIT (2026)
- models/llama-3-1-8b
- models/deepseek-coder-v2-236b
- Phi-3 Mini 3.8B: 69% MMLU on 8GB RAM (Microsoft SLM)
- models/gemma-2-9b
- GPT-5.6 Explained: Luna, Terra, Sol Pricing & Local Options
- StarCoder 2 15B – Technical Analysis & Performance Guide
- models/gpt-oss
- models/devstral
Grab the AI Starter Kit — career roadmap, cheat sheet, setup guide
No spam. Unsubscribe with one click.
Found your model? Now build something with it.
25 hands-on courses — RAG, agents, fine-tuning — all running locally. First chapter free, no card.