Google Gemma 7B: 64% MMLU from DeepMind
Built on the same research as Gemini. 7B parameters, 8K context, a 5.0GB Ollama download at Q4. Released February 2024 by Google DeepMind. Succeeded by Gemma 2.
Technical Specifications Overview
ollama run gemma:7bGoogle Gemma 7B Architecture
Decoder-only transformer with Multi-Query Attention, RoPE embeddings, and RMSNorm — built on Gemini research
Google DeepMind Architecture: Relationship to Gemini
Gemma 7B was released by Google DeepMind on February 21, 2024, built using the same research and technology as the Gemini family of models. The name "Gemma" comes from the Latin for "gem" — reflecting its position as a smaller, open version of Google's proprietary Gemini technology. It was trained on 6 trillion tokens from web documents, code, and mathematical text.
Key Architectural Features from Gemini
- Multi-Query Attention (MQA): Unlike standard multi-head attention where each head has its own key/value projections, Gemma uses multi-query attention where all heads share a single key-value pair. This reduces memory bandwidth during inference, enabling faster generation on consumer GPUs.
- RMSNorm Pre-Normalization: Gemma applies Root Mean Square Layer Normalization before each attention and feed-forward block (pre-norm style), rather than post-normalization. This improves training stability and was adopted from the Gemini architecture.
- RoPE Embeddings: Rotary Position Embeddings encode positional information directly into the attention mechanism, supporting the 8,192 token context window without fixed positional encodings.
- GeGLU Activation: Uses Gated Linear Unit with GELU activation in the feed-forward layers, which provides better gradient flow compared to standard ReLU or GELU alone.
- SentencePiece Tokenizer: 256K vocabulary size — significantly larger than Llama 2's 32K vocabulary. This reduces the number of tokens needed to represent text, improving effective context length and multilingual capability.
Research Papers and References
- Gemma: Open Models Based on Gemini Research and Technology — (Gemma Team, Google DeepMind, 2024)
- Gemma Official Documentation — Google AI Developer portal
- Gemma 7B on Hugging Face — Model weights, model card, and usage examples
- Google Blog: Gemma Announcement — Official launch post with background
Real Benchmark Results
Data source: All benchmark scores below are from Google's Gemma technical report (arXiv:2403.08295), Table 6, except TruthfulQA, which is from its HuggingFace H6 table (Table 7). MMLU is 5-shot, HellaSwag is 0-shot, GSM8K is maj@1. Google notes the Llama 2 figures are as reported by Meta. This site has not run these benchmarks.
MMLU Score: 7B Model Comparison
MMLU Accuracy (%)
MMLU (Massive Multitask Language Understanding) measures broad knowledge across 57 subjects. Higher is better.
Multi-Benchmark Capability Profile
Performance Metrics
Scores from Google's technical report (Tables 6 and 7). GSM8K (math) and TruthfulQA are notably weaker areas.
Detailed Benchmark Breakdown
| Benchmark | Gemma 7B | Mistral 7B | Llama 2 7B | Shots |
|---|---|---|---|---|
| MMLU | 64.3% | 62.5% | 45.3% | 5-shot |
| HellaSwag | 81.2% | 81.0% | 77.2% | 0-shot |
| ARC-Challenge | 53.2% | 54.9% | 45.9% | as reported |
| GSM8K | 46.4% | 35.4% | 14.6% | maj@1 |
| Winogrande | 72.3% | 74.2% | 69.2% | partial scoring |
| TruthfulQA | 44.8% | 42.2% | Not in the report | 0-shot (H6) |
VRAM & Hardware Requirements
Hardware Requirements by Quantization
System Requirements
Default Q4 (5.0GB file)
- Best for: Most local users
- Quality: Minor degradation vs FP16
- Speed: ~72 tok/s ceiling on an RTX 3060 12GB (360 GB/s)
- GPU: 8GB+ cards (RTX 3060 12GB, RTX 4060), M1 16GB
- Command:
ollama run gemma:7b
Q8_0 (9.1GB file)
- Best for: Quality-sensitive tasks
- Quality: Near-lossless
- Speed: ~55 tok/s ceiling on an RTX 4070 12GB (504 GB/s)
- GPU: 12GB+ cards (RTX 4070, RTX 3060 12GB), M1 Pro 16GB
- Command:
ollama run gemma:7b-instruct-q8_0
FP16 (17GB file)
- Best for: Research, fine-tuning
- Quality: Full precision
- Speed: ~59 tok/s ceiling on an RTX 4090 (1,008 GB/s)
- GPU: RTX 4090 24GB, M2 Ultra
- Note: Required for fine-tuning
Ollama Installation Guide
Install Ollama
Download and install Ollama from ollama.com
Pull Gemma 7B
Download the default Q4 build (5.0GB)
Run Gemma 7B
Start an interactive chat session
Test with a Prompt
Verify the model responds correctly
Check Model Info
Verify model details and VRAM usage
Terminal Example
Alternative: Hugging Face Transformers
Note: Hugging Face access requires accepting the Gemma Terms of Use at huggingface.co/google/gemma-7b and setting your HF_TOKEN.
VRAM by Quantization Level
VRAM Usage by Quantization (GB)
Gemma 7B memory needs vary significantly with quantization. Ollama's default 4-bit build is a 5.0 GB download and provides the best balance of quality and resource usage for most local deployments.
| Quantization | Ollama tag | Download |
|---|---|---|
| Default (gemma:7b) | gemma:7b | 5.0 GB |
| Q4_K_M | gemma:7b-instruct-q4_K_M | 5.5 GB |
| Q5_K_M | gemma:7b-instruct-q5_K_M | 6.3 GB |
| Q6_K | gemma:7b-instruct-q6_K | 7.2 GB |
| Q8_0 | gemma:7b-instruct-q8_0 | 9.1 GB |
| FP16 | gemma:7b-instruct-fp16 | 17 GB |
Sizes from Ollama's gemma tag list, checked November 2, 2026. Add about 1GB for the KV cache and runtime at the 8K context.
Apple Silicon Performance
- M1 8GB: default Q4 fits with little to spare, ~14 tok/s ceiling (68 GB/s)
- M1 Pro 16GB: Q8_0 comfortable, ~22 tok/s ceiling (200 GB/s)
- M2 Max 32GB: FP16 possible, ~23 tok/s ceiling (400 GB/s)
- M3 Pro 18GB: Q8_0 with room for context
- Unified memory: Shared CPU/GPU memory is ideal for LLMs
NVIDIA GPU Performance
- RTX 3060 12GB: default Q4, ~72 tok/s ceiling (360 GB/s)
- RTX 3070 8GB: Q4_K_M (5.5GB), ~81 tok/s ceiling (448 GB/s)
- RTX 3080 10GB: Q6_K (7.2GB), ~106 tok/s ceiling (760 GB/s)
- RTX 4090 24GB: FP16 with room, ~59 tok/s ceiling (1,008 GB/s)
- Key: ceilings are bandwidth divided by file size; real speed lands below them
7B Model Comparison (Published Benchmarks)
Local 7B Models: MMLU Comparison
Gemma 7B was strong for its February 2024 release date. However, Qwen 2.5 7B (released September 2024) significantly outperforms it on MMLU. MMLU scores come from the Gemma report (Gemma, Mistral, Llama 2), the Qwen2.5 report and Microsoft's Phi-2 announcement.
| Model | Size | Ollama download | Speed ceiling (RTX 3060 12GB) | MMLU (published) | Licence |
|---|---|---|---|---|---|
| Qwen 2.5 7B | 7B | 4.7GB | ~77 tok/s ceiling | 74.2% | Apache 2.0 |
| Gemma 7B | 7B | 5.0GB | ~72 tok/s ceiling | 64.3% | Gemma ToU |
| Mistral 7B | 7B | 4.4GB | ~82 tok/s ceiling | 62.5% | Apache 2.0 |
| Phi-2 (2.7B) | 2.7B | 1.6GB | ~225 tok/s ceiling | 56.7% | MIT |
| Llama 2 7B | 7B | 3.8GB | ~94 tok/s ceiling | 45.3% | Llama 2 License |
Speed is an upper bound, not a measurement: an RTX 3060 12GB's 360 GB/s of memory bandwidth divided by each model's default Ollama file. At the smallest sizes compute, not bandwidth, sets the real limit, so Phi-2 will land well below its ceiling.
Honest Assessment
Gemma 7B Strengths
- - Strong MMLU (64.3%) for a February 2024 7B model
- - Google DeepMind pedigree and Gemini-derived architecture
- - 256K vocabulary handles multilingual text efficiently
- - Permissive commercial license (Gemma Terms of Use)
- - Well-supported across frameworks (Ollama, HuggingFace, vLLM)
Gemma 7B Weaknesses
- - Only 8K context window (Mistral 7B has 32K)
- - Math is only middling: 46.4% GSM8K, well short of newer 7B models
- - Superseded by Gemma 2 9B which is better in every metric
- - TruthfulQA score (44.8%) indicates hallucination risk
- - Qwen 2.5 7B now significantly outperforms it at 74.2% MMLU
Gemma 2 Upgrade Path
Why Upgrade to Gemma 2?
Google released Gemma 2 in July 2024, with models at 2B, 9B, and 27B parameters. For users currently running Gemma 7B, the Gemma 2 9B Instruct model is the natural upgrade path — it offers substantially better performance at similar VRAM cost.
| Metric | Gemma 7B | Gemma 2 9B IT | Improvement |
|---|---|---|---|
| MMLU | 64.3% | 71.3% | +7 points |
| Parameters | 7B | 9B | +2B |
| Context Window | 8,192 | 8,192 | Same |
| VRAM (Q4) | 5.0GB | 5.4GB | +0.4GB |
| Instruction Tuned | Separate IT variant | Built-in IT | Better chat |
Migration Command
ollama run gemma2:9bDrop-in replacement in Ollama. If you have Gemma 7B fine-tunes, you will need to re-fine-tune on the Gemma 2 architecture, as the model structures are not compatible.
Local AI Alternatives
If you are evaluating Gemma 7B for a new project in 2026, consider these alternatives that may better suit your needs.
| Model | MMLU | VRAM (Q4) | Context | License | Best For |
|---|---|---|---|---|---|
| Gemma 2 9B IT | 71.3% | 5.4GB | 8K | Gemma ToU | Direct Gemma 7B upgrade |
| Qwen 2.5 7B | 74.2% | 4.7GB | 128K | Apache 2.0 | Best 7B overall, long context |
| Mistral 7B v0.3 | 62.5% | 4.4GB | 32K | Apache 2.0 | Apache license, 32K context |
| Llama 3.1 8B | 69.4% | 4.9GB | 128K | Llama 3.1 License | Strong all-rounder, huge context |
| Phi-3 Mini 3.8B | 70.9% | 2.2GB | 128K | MIT | Tiny but strong, MIT license |
Recommendation for 2026: If you need a Google model, upgrade to Gemma 2 9B IT. For the strongest 7B-class model with Apache 2.0 license, choose Qwen 2.5 7B. For the best balance of size and capability, Llama 3.1 8B is excellent.
Resources & Further Reading
Official Google Resources
- Google AI Gemma Documentation
Official API docs, model cards, and deployment guides
- Gemma Technical Report (arXiv)
Full benchmark data and architecture details
- Hugging Face: google/gemma-7b
Model weights, model card, and community discussion
- Google Blog: Gemma Launch
Official announcement with background context
Local Deployment Tools
- Ollama: Gemma Library
One-command install with automatic quantization
- llama.cpp (supports Gemma)
C++ inference engine with GGUF format support
- LM Studio
GUI-based local inference with Gemma support
- vLLM
High-throughput serving engine for production deployment
Gemma 2 Resources
- Gemma 2 9B IT on Hugging Face
Recommended successor with instruction tuning
- Ollama: Gemma 2 Library
ollama run gemma2:9b — direct upgrade from Gemma 7B
- Kaggle: Gemma 2 Models
Kaggle notebooks and fine-tuning examples
- Gemma PyTorch GitHub
Open-source reference implementation
Troubleshooting & Common Issues
Out of Memory (OOM) Errors
Gemma 7B at FP16 is a 17GB file. If you see OOM errors, you are likely trying to load a quantization too large for your GPU.
Solutions:
- - Use the default 4-bit build (5.0GB): fits an 8GB card with room for context
- - Close other GPU-intensive applications (browsers with hardware acceleration, games)
- - On Ollama, the default gemma:7b is already 4-bit — if OOM, your GPU likely has less than 6GB VRAM
- - For GPUs with 4GB or less VRAM, consider Gemma 2B or Phi-3 Mini instead
- - On Apple Silicon, check Activity Monitor to ensure enough unified memory is free
Slow Generation Speed
If you are getting less than 10 tokens/second, the model is likely running on CPU instead of GPU.
Solutions:
- - Verify GPU detection:
ollama psshows which device is in use - - Install NVIDIA CUDA toolkit if on Linux with NVIDIA GPU
- - On macOS, Apple Silicon Metal acceleration is automatic in Ollama
- - Reduce context length if generation slows over long conversations
- - Bandwidth ceilings for the 5.0GB default: ~72 tok/s on an RTX 3060 12GB, ~14 tok/s on a base M1, ~18 tok/s CPU-only on dual-channel DDR5-5600; real speed lands below these
Hugging Face Access Denied
Gemma models on Hugging Face require accepting Google's Terms of Use before downloading.
Steps:
- 1. Visit huggingface.co/google/gemma-7b and accept the license agreement
- 2. Create a Hugging Face access token at huggingface.co/settings/tokens
- 3. Set the token:
export HF_TOKEN=your_token_here - 4. Alternatively, use Ollama which does not require Hugging Face authentication
Frequently Asked Questions
What is Gemma 7B and how is it related to Google Gemini?
Gemma 7B is a 7-billion parameter open model released by Google DeepMind in February 2024. It uses the same research and technology that powers the Gemini models, including Multi-Query Attention and RMSNorm pre-normalization. It scores 64.3% on MMLU and 81.2% on HellaSwag. Unlike Gemini, Gemma weights are freely downloadable for local inference under the Gemma Terms of Use, which permits commercial applications.
How much VRAM does Gemma 7B need to run locally?
It depends on quantization: Ollama's default gemma:7b is a 5.0GB download, Q8_0 is 9.1GB and FP16 (full precision) is 17GB, plus about 1GB for the KV cache and runtime. For most users the default build on an 8GB+ GPU (RTX 3060 12GB, RTX 4060) is the right balance. CPU-only inference works but is limited by system memory bandwidth: dual-channel DDR5-5600 (89.6 GB/s) caps the 5GB file at roughly 18 tokens/second, against a ceiling near 72 tok/s on an RTX 3060 12GB.
How does Gemma 7B compare to Mistral 7B and Llama 2 7B?
In Google's Gemma report, Gemma 7B scores 64.3% on MMLU versus 62.5% for Mistral 7B and 45.3% for Llama 2 7B. It edges Mistral on HellaSwag (81.2% vs 81.0% vs 77.2%) and leads clearly on math (GSM8K 46.4% vs 35.4% vs 14.6%), while Mistral is ahead on ARC-Challenge (54.9% vs 53.2%) and Winogrande. Mistral 7B v0.2 and later have a 32K context window versus Gemma's 8K. Both have been succeeded by stronger models: Gemma 2 and Mistral Nemo.
Should I use Gemma 7B or upgrade to Gemma 2?
For new projects in 2026, Google Gemma 2 9B IT is the recommended choice. It scores significantly higher on benchmarks (71.3% MMLU in the Gemma 2 report) while requiring similar VRAM to Gemma 7B at equivalent quantization levels. Gemma 7B remains useful if you have existing fine-tunes or need the specific 7B architecture. Run Gemma 2 via Ollama with: ollama run gemma2:9b.
Can I use Gemma 7B commercially?
Yes. Gemma 7B is released under the Gemma Terms of Use, which is a permissive license allowing commercial use, fine-tuning, and redistribution. You must include the license notice and cannot use the Gemma name to endorse your products. This is more permissive than Llama 2's license for large-scale commercial deployment. Note: this is NOT Apache 2.0 — it is Google's own Gemma Terms of Use.
Build Real AI on Your Machine
RAG, agents, NLP, vision, and MLOps - chapters across 25 courses that take you from reading about AI to building AI.
Related Guides
Continue your local AI journey with these comprehensive guides
Go from reading about AI to building with AI
25 structured courses. Hands-on projects. Runs on your machine. Start free.
Written by
Barsha Mohapatra
Founder and Editor, Local AI Master · Founder, Little AI Master
Founder of Little AI Master and the editor behind every course and guide on Local AI Master. Years of tutoring students taught her where AI education actually fails.