★ Reading this for free? Get 25 structured AI courses + per-chapter AI tutor — the first chapter of every course free, no card.Start free in 30 secondsOr own every course: $149 once
META AI FOUNDATION MODEL

Llama 2 13B and 7B: Specs, VRAM and Ollama

Updated November 2, 2026

Note: Llama 2 13B (July 2023) has been superseded by Llama 3.1 8B (66.7 MMLU against 54.8, 128K context, a smaller file). This page covers Llama 2 13B for existing deployments and fine-tuning.

Llama 2 13B has 40 layers, a 5,120 hidden size, 40 attention heads and a 4,096-token context, and scores 54.8% on MMLU (arXiv:2307.09288). ollama run llama2:13b downloads a 7.4 GB 4-bit file that needs about 11.7 GB at 4K context: a 12 GB card such as the RTX 3060 12GB, or a 16 GB Mac. The smaller Llama 2 7B is covered below. It remains a common base for fine-tuning with QLoRA.

🔬 Open Source🚀 Commercial Use Allowed⚡ Efficient Inference

🔬 Model Architecture & Specifications

Model Parameters

Parameters13 Billion
ArchitectureTransformer
Context Length4096 tokens
Hidden Size5120
Attention Heads40
Key-Value Heads40 (no GQA)
FFN Size / Vocabulary13,824 / 32,000
Layers40

Training Details

Training Data2 Trillion tokens
Trained / ReleasedJan-Jul 2023 / 18 Jul 2023
Optimizer / Peak LRAdamW / 3.0e-4
Chat Fine-tuningSFT + RLHF (Llama 2-Chat)
Quantization4-bit (GGUF) available
LicenseLlama 2 Community

📊 Performance Benchmarks & Analysis

🎯 Standardized Benchmark Results

Academic Benchmarks

MMLU (Knowledge)
54.8%
HumanEval (Coding)
18.3%
GSM8K (Math)
28.7%
HellaSwag (Reasoning)
80.7%

Grouped Benchmarks (model card)

Commonsense Reasoning66.9
World Knowledge55.4
Reading Comprehension65.8
BIG-Bench Hard39.4

Every score above is Meta's, from the Llama 2 paper (arXiv:2307.09288, Tables 19-21) and the Llama 2 13B model card. This site has not re-run them.

Llama 2 13B VRAM requirements

The default llama2:13b download needs about 11.7 GB at a 4K context, so plan on a 12 GB graphics card; it does not fit an 8 GB card. Llama 2 13B has no grouped-query attention, so its key-value cache is large: 2 × 40 layers × 5,120 × 2 bytes = 819 KB per token, or 3.4 GB at the full 4,096 tokens. Sizes are from the Ollama registry; memory is file + cache + about 1 GB of buffers, worked out, not measured.

Ollama tagDownloadMemory at 4KFits
llama2:13b-chat-q2_K5.43 GB~9.8 GB12 GB card
llama2:13b (q4_0, the default)7.37 GB~11.7 GB12 GB card, with little to spare
llama2:13b-chat-q4_K_M7.87 GB~12.2 GB16 GB card
llama2:13b-chat-q5_K_M9.23 GB~13.6 GB16 GB card
llama2:13b-chat-q8_013.83 GB~18.2 GB24 GB card
llama2:13b-chat-fp1626.03 GB~30.4 GB32 GB card or 48 GB Mac

llama2:13b is the chat model; llama2:13b-text (also 7.37 GB) is the base model for completion and fine-tuning experiments.

Llama 2 7B

Llama 2 7B has 32 layers, a 4,096 hidden size and 32 attention heads, the same 4K context, and scores 45.3% on MMLU. ollama run llama2:7b is a 3.8 GB download that needs about 7 GB at 4K, so it fits an 8 GB card. The unquantised fp16 weights are 13.5 GB (llama2:7b-chat-fp16, 13.48 GB). Against the 13B, the paper gives it 12.8% against 18.3% on HumanEval, 14.6% against 28.7% on GSM8K, and 77.2 against 80.7 on HellaSwag.

Ollama tagDownloadMemory at 4KFits
llama2:7b (q4_0, the default)3.83 GB~7.0 GB8 GB card
llama2:7b-chat-q4_K_M4.08 GB~7.2 GB8 GB card
llama2:7b-chat-q8_07.16 GB~10.3 GB12 GB card
llama2:7b-chat-fp1613.48 GB~16.6 GB24 GB card

The largest size, Llama 2 70B, is a 38.9 GB download (llama2:70b) and the only Llama 2 model with grouped-query attention.

System Requirements

▸
Operating System
Windows 10+, macOS 11+, Ubuntu 20.04+
▸
RAM
16GB for CPU-only use of the 4-bit 13B
▸
Storage
7.4GB for llama2:13b (3.8GB for llama2:7b)
▸
GPU
12 GB VRAM for llama2:13b at 4K context (8 GB is enough for the 7B)
▸
CPU
6+ cores (8+ recommended)

Installation & Setup Guide

1

Verify System Requirements

Check hardware compatibility

$ free -h # Linux/Mac wmic OS get TotalVisibleMemorySize /value # Windows
2

Install Ollama Runtime

Download and install the AI model platform

$ curl -fsSL https://ollama.com/install.sh | sh
3

Download Llama 2 13B

Pull the 7.4 GB model file (Q4_0 chat model)

$ ollama pull llama2:13b
4

Test Model Functionality

Verify installation with sample interaction

$ ollama run llama2:13b "Hello, please introduce yourself and explain your capabilities."

Command Line Interface Examples

Terminal
$ollama run llama2:13b
pulling manifest pulling d80e890702c7... 100% ▕████████████████▏ 7.4 GB pulling 8c17c2ebb0ea... 100% ▕████████████████▏ 7.0 KB verifying sha256 digest writing manifest success >>> Send a message (/? for help)
$>>> What are your strengths and limitations?
As Llama 2 13B, here are my honest capabilities: **Strengths:** - General text generation and conversation - Summarization and paraphrasing - Basic coding assistance (Python, JavaScript) - Creative writing and brainstorming - Runs on consumer hardware (a 12 GB GPU for the 4-bit file) **Limitations:** - Knowledge cutoff: early 2023 - MMLU: 54.8% (vs 66.6% for newer Llama 3.1 8B) - Math: GSM8K 28.7% — struggles with multi-step problems - Coding: HumanEval 18.3% — basic code only - Context: 4,096 tokens (vs 128K for Llama 3.1) For new projects, consider Llama 3.1 8B: a smaller file, higher scores and a 128K context.
$_

Llama 2 Family Comparison

Llama 2 13B against current models of similar size. MMLU is each vendor's own published figure; the speed column is a bandwidth ceiling on an RTX 3060 12GB (360 GB/s divided by the file size), an upper bound rather than a measurement.

ModelOllama tagDownloadSpeed ceiling, RTX 3060MMLU (source)Licence
Llama 2 13Bllama2:13b7.37 GB~49 tok/s ceiling54.8 (Meta)Llama 2 Community
Llama 3.1 8Bllama3.1:8b4.92 GB~73 tok/s ceiling66.7 (Meta)Llama 3.1 Community
Qwen 2.5 14Bqwen2.5:14b8.99 GB~40 tok/s ceiling79.7 (Qwen)Apache 2.0
Mistral 7Bmistral4.37 GB~82 tok/s ceiling60.1 (Mistral)Apache 2.0

Meta's own releases are only half the story — the 13B slot is where the community fine-tune ecosystem grew up. The most widely used derivative is Pygmalion 13B, a Llama 2 13B tune aimed at uncensored roleplay. The merge-everything culture that produced it started a generation earlier with LLaMA-1 models like Manticore 13B.

Implementation & Deployment Strategies

⚙️ Deployment Options

  • ✓ Local inference via Ollama
  • ✓ Docker containerization
  • ✓ API server deployment
  • ✓ Cloud platform integration
  • ✓ Batch processing pipelines

🎯 Application Areas

  • ✓ Content generation and writing
  • ✓ Code review and documentation
  • ✓ Conversational AI interfaces
  • ✓ Data analysis and summarization
  • ✓ Educational tutoring systems

Performance Optimization Strategies

🚀 GPU Acceleration Configuration

Optimize inference speed with GPU offloading:

# Ollama auto-detects GPU (NVIDIA/AMD/Apple Silicon)
# No manual GPU config needed
ollama run llama2:13b
# Check GPU usage
nvidia-smi # NVIDIA
# Or check in Activity Monitor on macOS
# llama2:13b is Q4_0, a 7.4 GB file: ~11.7 GB at 4K context
# Fits on: RTX 3060 12GB, 16 GB+ cards, 16 GB+ Macs (not 8 GB cards)

💾 Memory Management

Efficient memory usage for 16GB systems:

# Ollama serves Q4_0 by default (7.4 GB file)
ollama run llama2:13b
# Custom Modelfile for context tuning
FROM llama2:13b
PARAMETER num_ctx 2048 # halves the 3.4 GB cache
# Limit concurrent requests to save memory
export OLLAMA_MAX_LOADED_MODELS=1

⚡ CPU Optimization

Maximize CPU inference performance:

# Serve to multiple concurrent users
export OLLAMA_NUM_PARALLEL=2
ollama serve
# Use the REST API
curl http://localhost:11434/api/generate \
-d '{"model":"llama2:13b","prompt":"Hello"}'

API Integration Examples

🔧 Python Integration

import ollama
from typing import Dict, List

class Llama2Client:
    def __init__(self, model: str = "llama2:13b"):
        self.client = ollama.Client()
        self.model = model

    def generate_response(self, prompt: str, **kwargs) -> str:
        """Generate text response"""
        response = self.client.generate(
            model=self.model,
            prompt=prompt,
            options=kwargs
        )
        return response['response']

    def chat_completion(self, messages: List[Dict], **kwargs) -> str:
        """Chat completion with conversation history"""
        response = self.client.chat(
            model=self.model,
            messages=messages,
            options=kwargs
        )
        return response['message']['content']

    def stream_response(self, prompt: str, **kwargs):
        """Stream response token by token"""
        for chunk in self.client.generate(
            model=self.model,
            prompt=prompt,
            stream=True,
            options=kwargs
        ):
            yield chunk['response']

# Usage examples
client = Llama2Client()

# Simple generation
text = client.generate_response(
    "Explain machine learning in simple terms",
    temperature=0.7,
    top_p=0.9
)

# Chat with context
messages = [
    {"role": "system", "content": "You are a helpful AI assistant."},
    {"role": "user", "content": "What is Python?"}
]
response = client.chat_completion(messages, temperature=0.3)

# Streaming output
for token in client.stream_response("Write a story", temperature=0.8):
    print(token, end="", flush=True)

🌐 Node.js API Server

const express = require('express');
const { Ollama } = require('ollama-node');

const app = express();
app.use(express.json());

const ollama = new Ollama();

// Health check endpoint
app.get('/health', (req, res) => {
  res.json({ status: 'healthy', model: 'llama2:13b' });
});

// Text generation endpoint
app.post('/api/generate', async (req, res) => {
  try {
    const { prompt, options = {} } = req.body;

    const response = await ollama.generate({
      model: 'llama2:13b',
      prompt: prompt,
      options: {
        temperature: 0.7,
        top_p: 0.9,
        ...options
      }
    });

    res.json({
      response: response.response,
      model: 'llama2:13b',
      done: response.done,
      context: response.context
    });
  } catch (error) {
    res.status(500).json({ error: error.message });
  }
});

// Chat completion endpoint
app.post('/api/chat', async (req, res) => {
  try {
    const { messages, options = {} } = req.body;

    const response = await ollama.chat({
      model: 'llama2:13b',
      messages: messages,
      options: {
        temperature: 0.7,
        top_p: 0.9,
        ...options
      }
    });

    res.json({
      response: response.message.content,
      model: 'llama2:13b',
      done: response.done
    });
  } catch (error) {
    res.status(500).json({ error: error.message });
  }
});

// Streaming endpoint
app.post('/api/stream', async (req, res) => {
  const { prompt, options = {} } = req.body;

  res.setHeader('Content-Type', 'text/event-stream');
  res.setHeader('Cache-Control', 'no-cache');
  res.setHeader('Connection', 'keep-alive');

  try {
    const stream = await ollama.generate({
      model: 'llama2:13b',
      prompt: prompt,
      stream: true,
      options: options
    });

    for await (const chunk of stream) {
      res.write(`data: ${JSON.stringify(chunk)}

`);
    }
    res.end();
  } catch (error) {
    res.write(`data: ${JSON.stringify({ error: error.message })}

`);
    res.end();
  }
});

const PORT = process.env.PORT || 3000;
app.listen(PORT, () => {
  console.log(`Llama 2 13B API server running on port ${PORT}`);
});

Practical Use Cases & Applications

💼 Business Applications

Content Generation

Marketing copy, product descriptions, blog posts, and documentation with consistent brand voice.

Customer Support

Automated responses, ticket triage, FAQ generation, and support documentation.

Data Analysis

Report summarization, data insights, and natural language querying of structured data.

👨‍💻 Development Applications

Code Generation

Boilerplate code, unit tests, documentation, and refactoring suggestions.

Code Review

Code quality analysis, security vulnerability detection, and optimization suggestions.

Technical Documentation

API documentation, README files, and technical specification generation.

Technical Limitations & Considerations

⚠️ Model Limitations

Knowledge Constraints

  • • Training cutoff in early 2023
  • • Limited recent event knowledge
  • • May generate outdated information
  • • No real-time data access
  • • Fixed knowledge base

Performance Constraints

  • • 12 GB GPU (or 16 GB RAM on CPU) for the 4-bit file
  • • Slower inference than cloud APIs
  • • 4,096-token context window
  • • May struggle with complex reasoning
  • • Computational resource intensive

🤔 Frequently Asked Questions

What are the advantages of Llama 2 13B over cloud-based alternatives?

Llama 2 13B offers data privacy, offline operation, no API costs, and customization capabilities. While cloud models may have more recent knowledge, local deployment provides complete control over data and infrastructure, making it ideal for sensitive applications and cost-conscious deployments.

How does Llama 2 13B handle different types of tasks?

It is a capable general text model for its size and generation: conversation, summarisation and drafting are fine. It is weak at maths (28.7% GSM8K) and code (18.3% HumanEval), and its 4,096-token context limits long documents. For those jobs a newer model of the same size is much better.

What is the commercial license for Llama 2 13B?

Llama 2 is released under the Llama 2 Community License, which permits commercial use. Organizations with over 700 million monthly active users must request a special license from Meta. For most businesses and developers, the model can be used in commercial applications without additional licensing fees.

What is the context length and hidden size of Llama 2 13B?

The context length is 4,096 tokens for every Llama 2 size, per the model card and the max_position_embeddings value in its config. The 13B's hidden size is 5,120, with 40 layers, 40 attention heads and an FFN size of 13,824. The 7B has a 4,096 hidden size and 32 layers.

How much VRAM does Llama 2 7B need, and how big is it in fp16?

The default llama2:7b download is a 3.8 GB 4-bit file that needs about 7 GB at 4K context, so an 8 GB card is enough. The fp16 weights are 13.5 GB on their own and need a 16 GB card at minimum, more comfortably 24 GB.

Can Llama 2 13B be fine-tuned for specific applications?

Yes, Llama 2 13B supports fine-tuning using techniques like LoRA (Low-Rank Adaptation). This allows customization for specific domains, company terminology, or particular use cases. Fine-tuning typically requires GPU resources and training datasets but can significantly improve performance on specialized tasks.

Learn AI in the Right Order

Structured courses with hands-on projects and local-first workflows that reduce API dependency where they fit.

Better Local Alternatives to Llama 2 13B (2026)

Newer models score far higher in the same memory range. MMLU figures are each vendor's own; sizes are Ollama downloads.

ModelMMLUDownloadContextOllama Command
Qwen 2.5 14B79.7%9.0 GB128Kollama run qwen2.5:14b
Llama 3.1 8B66.7%4.9 GB128Kollama run llama3.1:8b
Gemma 2 9B71.3%5.4 GB8Kollama run gemma2:9b
Mistral 7B60.1%4.4 GB32Kollama run mistral
Llama 2 13B54.8%7.4 GB4Kollama run llama2:13b

Recommendation: Qwen 2.5 14B (see the Qwen 2.5 guide) scores 79.7 on MMLU against 54.8, for a 9.0 GB download against 7.4 GB, with 32x the context window.

Related Foundation Models

📚 Authoritative Sources & Research

🎯
AI Learning Path

Go from reading about AI to building with AI

25 structured courses. Hands-on projects. Runs on your machine. Start free.

Or own it for life — Lifetime $149 $599, pay once
Barsha Mohapatra

Written by

Barsha Mohapatra

Founder and Editor, Local AI Master · Founder, Little AI Master

Founder of Little AI Master and the editor behind every course and guide on Local AI Master. Years of tutoring students taught her where AI education actually fails.

📅 Published: 2025-09-26🔄 Last Updated: November 2, 2026✓ Manually Reviewed
Reading now
Join the discussion
More on Local AI Hardware
See the full AI Hardware Guide 2026 guide.
📚
Free · no account required

Grab the AI Starter Kit — career roadmap, cheat sheet, setup guide

No spam. Unsubscribe with one click.

🎯
AI Learning Path

Found your model? Now build something with it.

25 hands-on courses — RAG, agents, fine-tuning — all running locally. First chapter free, no card.

Or own it for life — Lifetime $149 $599, pay once
Free Tools & Calculators