Llama 2 13B and 7B: Specs, VRAM and Ollama
Updated November 2, 2026
Note: Llama 2 13B (July 2023) has been superseded by Llama 3.1 8B (66.7 MMLU against 54.8, 128K context, a smaller file). This page covers Llama 2 13B for existing deployments and fine-tuning.
Llama 2 13B has 40 layers, a 5,120 hidden size, 40 attention heads and a 4,096-token context, and scores 54.8% on MMLU (arXiv:2307.09288). ollama run llama2:13b downloads a 7.4 GB 4-bit file that needs about 11.7 GB at 4K context: a 12 GB card such as the RTX 3060 12GB, or a 16 GB Mac. The smaller Llama 2 7B is covered below. It remains a common base for fine-tuning with QLoRA.
🔬 Model Architecture & Specifications
Model Parameters
Training Details
📊 Performance Benchmarks & Analysis
🎯 Standardized Benchmark Results
Academic Benchmarks
Grouped Benchmarks (model card)
Every score above is Meta's, from the Llama 2 paper (arXiv:2307.09288, Tables 19-21) and the Llama 2 13B model card. This site has not re-run them.
Llama 2 13B VRAM requirements
The default llama2:13b download needs about 11.7 GB at a 4K context, so plan on a 12 GB graphics card; it does not fit an 8 GB card. Llama 2 13B has no grouped-query attention, so its key-value cache is large: 2 × 40 layers × 5,120 × 2 bytes = 819 KB per token, or 3.4 GB at the full 4,096 tokens. Sizes are from the Ollama registry; memory is file + cache + about 1 GB of buffers, worked out, not measured.
| Ollama tag | Download | Memory at 4K | Fits |
|---|---|---|---|
llama2:13b-chat-q2_K | 5.43 GB | ~9.8 GB | 12 GB card |
llama2:13b (q4_0, the default) | 7.37 GB | ~11.7 GB | 12 GB card, with little to spare |
llama2:13b-chat-q4_K_M | 7.87 GB | ~12.2 GB | 16 GB card |
llama2:13b-chat-q5_K_M | 9.23 GB | ~13.6 GB | 16 GB card |
llama2:13b-chat-q8_0 | 13.83 GB | ~18.2 GB | 24 GB card |
llama2:13b-chat-fp16 | 26.03 GB | ~30.4 GB | 32 GB card or 48 GB Mac |
llama2:13b is the chat model; llama2:13b-text (also 7.37 GB) is the base model for completion and fine-tuning experiments.
Llama 2 7B
Llama 2 7B has 32 layers, a 4,096 hidden size and 32 attention heads, the same 4K context, and scores 45.3% on MMLU. ollama run llama2:7b is a 3.8 GB download that needs about 7 GB at 4K, so it fits an 8 GB card. The unquantised fp16 weights are 13.5 GB (llama2:7b-chat-fp16, 13.48 GB). Against the 13B, the paper gives it 12.8% against 18.3% on HumanEval, 14.6% against 28.7% on GSM8K, and 77.2 against 80.7 on HellaSwag.
| Ollama tag | Download | Memory at 4K | Fits |
|---|---|---|---|
llama2:7b (q4_0, the default) | 3.83 GB | ~7.0 GB | 8 GB card |
llama2:7b-chat-q4_K_M | 4.08 GB | ~7.2 GB | 8 GB card |
llama2:7b-chat-q8_0 | 7.16 GB | ~10.3 GB | 12 GB card |
llama2:7b-chat-fp16 | 13.48 GB | ~16.6 GB | 24 GB card |
The largest size, Llama 2 70B, is a 38.9 GB download (llama2:70b) and the only Llama 2 model with grouped-query attention.
System Requirements
Installation & Setup Guide
Verify System Requirements
Check hardware compatibility
Install Ollama Runtime
Download and install the AI model platform
Download Llama 2 13B
Pull the 7.4 GB model file (Q4_0 chat model)
Test Model Functionality
Verify installation with sample interaction
Command Line Interface Examples
Llama 2 Family Comparison
Llama 2 13B against current models of similar size. MMLU is each vendor's own published figure; the speed column is a bandwidth ceiling on an RTX 3060 12GB (360 GB/s divided by the file size), an upper bound rather than a measurement.
| Model | Ollama tag | Download | Speed ceiling, RTX 3060 | MMLU (source) | Licence |
|---|---|---|---|---|---|
| Llama 2 13B | llama2:13b | 7.37 GB | ~49 tok/s ceiling | 54.8 (Meta) | Llama 2 Community |
| Llama 3.1 8B | llama3.1:8b | 4.92 GB | ~73 tok/s ceiling | 66.7 (Meta) | Llama 3.1 Community |
| Qwen 2.5 14B | qwen2.5:14b | 8.99 GB | ~40 tok/s ceiling | 79.7 (Qwen) | Apache 2.0 |
| Mistral 7B | mistral | 4.37 GB | ~82 tok/s ceiling | 60.1 (Mistral) | Apache 2.0 |
Meta's own releases are only half the story — the 13B slot is where the community fine-tune ecosystem grew up. The most widely used derivative is Pygmalion 13B, a Llama 2 13B tune aimed at uncensored roleplay. The merge-everything culture that produced it started a generation earlier with LLaMA-1 models like Manticore 13B.
Implementation & Deployment Strategies
⚙️ Deployment Options
- ✓ Local inference via Ollama
- ✓ Docker containerization
- ✓ API server deployment
- ✓ Cloud platform integration
- ✓ Batch processing pipelines
🎯 Application Areas
- ✓ Content generation and writing
- ✓ Code review and documentation
- ✓ Conversational AI interfaces
- ✓ Data analysis and summarization
- ✓ Educational tutoring systems
Performance Optimization Strategies
🚀 GPU Acceleration Configuration
Optimize inference speed with GPU offloading:
💾 Memory Management
Efficient memory usage for 16GB systems:
⚡ CPU Optimization
Maximize CPU inference performance:
API Integration Examples
🔧 Python Integration
import ollama
from typing import Dict, List
class Llama2Client:
def __init__(self, model: str = "llama2:13b"):
self.client = ollama.Client()
self.model = model
def generate_response(self, prompt: str, **kwargs) -> str:
"""Generate text response"""
response = self.client.generate(
model=self.model,
prompt=prompt,
options=kwargs
)
return response['response']
def chat_completion(self, messages: List[Dict], **kwargs) -> str:
"""Chat completion with conversation history"""
response = self.client.chat(
model=self.model,
messages=messages,
options=kwargs
)
return response['message']['content']
def stream_response(self, prompt: str, **kwargs):
"""Stream response token by token"""
for chunk in self.client.generate(
model=self.model,
prompt=prompt,
stream=True,
options=kwargs
):
yield chunk['response']
# Usage examples
client = Llama2Client()
# Simple generation
text = client.generate_response(
"Explain machine learning in simple terms",
temperature=0.7,
top_p=0.9
)
# Chat with context
messages = [
{"role": "system", "content": "You are a helpful AI assistant."},
{"role": "user", "content": "What is Python?"}
]
response = client.chat_completion(messages, temperature=0.3)
# Streaming output
for token in client.stream_response("Write a story", temperature=0.8):
print(token, end="", flush=True)🌐 Node.js API Server
const express = require('express');
const { Ollama } = require('ollama-node');
const app = express();
app.use(express.json());
const ollama = new Ollama();
// Health check endpoint
app.get('/health', (req, res) => {
res.json({ status: 'healthy', model: 'llama2:13b' });
});
// Text generation endpoint
app.post('/api/generate', async (req, res) => {
try {
const { prompt, options = {} } = req.body;
const response = await ollama.generate({
model: 'llama2:13b',
prompt: prompt,
options: {
temperature: 0.7,
top_p: 0.9,
...options
}
});
res.json({
response: response.response,
model: 'llama2:13b',
done: response.done,
context: response.context
});
} catch (error) {
res.status(500).json({ error: error.message });
}
});
// Chat completion endpoint
app.post('/api/chat', async (req, res) => {
try {
const { messages, options = {} } = req.body;
const response = await ollama.chat({
model: 'llama2:13b',
messages: messages,
options: {
temperature: 0.7,
top_p: 0.9,
...options
}
});
res.json({
response: response.message.content,
model: 'llama2:13b',
done: response.done
});
} catch (error) {
res.status(500).json({ error: error.message });
}
});
// Streaming endpoint
app.post('/api/stream', async (req, res) => {
const { prompt, options = {} } = req.body;
res.setHeader('Content-Type', 'text/event-stream');
res.setHeader('Cache-Control', 'no-cache');
res.setHeader('Connection', 'keep-alive');
try {
const stream = await ollama.generate({
model: 'llama2:13b',
prompt: prompt,
stream: true,
options: options
});
for await (const chunk of stream) {
res.write(`data: ${JSON.stringify(chunk)}
`);
}
res.end();
} catch (error) {
res.write(`data: ${JSON.stringify({ error: error.message })}
`);
res.end();
}
});
const PORT = process.env.PORT || 3000;
app.listen(PORT, () => {
console.log(`Llama 2 13B API server running on port ${PORT}`);
});Practical Use Cases & Applications
💼 Business Applications
Content Generation
Marketing copy, product descriptions, blog posts, and documentation with consistent brand voice.
Customer Support
Automated responses, ticket triage, FAQ generation, and support documentation.
Data Analysis
Report summarization, data insights, and natural language querying of structured data.
👨💻 Development Applications
Code Generation
Boilerplate code, unit tests, documentation, and refactoring suggestions.
Code Review
Code quality analysis, security vulnerability detection, and optimization suggestions.
Technical Documentation
API documentation, README files, and technical specification generation.
Technical Limitations & Considerations
⚠️ Model Limitations
Knowledge Constraints
- • Training cutoff in early 2023
- • Limited recent event knowledge
- • May generate outdated information
- • No real-time data access
- • Fixed knowledge base
Performance Constraints
- • 12 GB GPU (or 16 GB RAM on CPU) for the 4-bit file
- • Slower inference than cloud APIs
- • 4,096-token context window
- • May struggle with complex reasoning
- • Computational resource intensive
🤔 Frequently Asked Questions
What are the advantages of Llama 2 13B over cloud-based alternatives?
Llama 2 13B offers data privacy, offline operation, no API costs, and customization capabilities. While cloud models may have more recent knowledge, local deployment provides complete control over data and infrastructure, making it ideal for sensitive applications and cost-conscious deployments.
How does Llama 2 13B handle different types of tasks?
It is a capable general text model for its size and generation: conversation, summarisation and drafting are fine. It is weak at maths (28.7% GSM8K) and code (18.3% HumanEval), and its 4,096-token context limits long documents. For those jobs a newer model of the same size is much better.
What is the commercial license for Llama 2 13B?
Llama 2 is released under the Llama 2 Community License, which permits commercial use. Organizations with over 700 million monthly active users must request a special license from Meta. For most businesses and developers, the model can be used in commercial applications without additional licensing fees.
What is the context length and hidden size of Llama 2 13B?
The context length is 4,096 tokens for every Llama 2 size, per the model card and the max_position_embeddings value in its config. The 13B's hidden size is 5,120, with 40 layers, 40 attention heads and an FFN size of 13,824. The 7B has a 4,096 hidden size and 32 layers.
How much VRAM does Llama 2 7B need, and how big is it in fp16?
The default llama2:7b download is a 3.8 GB 4-bit file that needs about 7 GB at 4K context, so an 8 GB card is enough. The fp16 weights are 13.5 GB on their own and need a 16 GB card at minimum, more comfortably 24 GB.
Can Llama 2 13B be fine-tuned for specific applications?
Yes, Llama 2 13B supports fine-tuning using techniques like LoRA (Low-Rank Adaptation). This allows customization for specific domains, company terminology, or particular use cases. Fine-tuning typically requires GPU resources and training datasets but can significantly improve performance on specialized tasks.
Learn AI in the Right Order
Structured courses with hands-on projects and local-first workflows that reduce API dependency where they fit.
Better Local Alternatives to Llama 2 13B (2026)
Newer models score far higher in the same memory range. MMLU figures are each vendor's own; sizes are Ollama downloads.
| Model | MMLU | Download | Context | Ollama Command |
|---|---|---|---|---|
| Qwen 2.5 14B | 79.7% | 9.0 GB | 128K | ollama run qwen2.5:14b |
| Llama 3.1 8B | 66.7% | 4.9 GB | 128K | ollama run llama3.1:8b |
| Gemma 2 9B | 71.3% | 5.4 GB | 8K | ollama run gemma2:9b |
| Mistral 7B | 60.1% | 4.4 GB | 32K | ollama run mistral |
| Llama 2 13B | 54.8% | 7.4 GB | 4K | ollama run llama2:13b |
Recommendation: Qwen 2.5 14B (see the Qwen 2.5 guide) scores 79.7 on MMLU against 54.8, for a 9.0 GB download against 7.4 GB, with 32x the context window.
Related Foundation Models
Go from reading about AI to building with AI
25 structured courses. Hands-on projects. Runs on your machine. Start free.

Written by
Barsha Mohapatra
Founder and Editor, Local AI Master · Founder, Little AI Master
Founder of Little AI Master and the editor behind every course and guide on Local AI Master. Years of tutoring students taught her where AI education actually fails.
Related Guides
Continue your local AI journey with these comprehensive guides
Continue Learning
Explore these essential AI topics to expand your knowledge:
- PILLARLocal AI Hardware Requirements (2026): Complete Guide
- Best Mac for Local AI: Every Apple Silicon Chip Ranked M1–M6
- Ollama System Requirements: Minimum RAM, GPU, Disk Size
- Run an LLM on Your Phone: Offline AI on Android & iPhone
- Llama 3.1 Nemotron 70B Instruct: Ollama, VRAM, Chat Template
- Gemma 4 Hardware Requirements: VRAM/RAM per Model
- Qwen3-Coder 30B & 480B: VRAM, Benchmarks, Ollama Setup
- WizardLM 13B: Evol-Instruct, MMLU 54.6%, 8GB VRAM (Q4)
- Mistral Nemo 12B: 128K Context, VRAM, Ollama Setup
- "No Kernel Image Is Available" on RTX 50: sm_120 Fix
Grab the AI Starter Kit — career roadmap, cheat sheet, setup guide
No spam. Unsubscribe with one click.
Found your model? Now build something with it.
25 hands-on courses — RAG, agents, fine-tuning — all running locally. First chapter free, no card.