CodeLlama-13B: Technical Analysis

CodeLlama 13B hardware requirements

The default codellama:13b download is a 7.4 GB 4-bit file that needs about 11.8 GB of memory with a 4K context, so it fits a 12 GB graphics card with almost nothing to spare and does not fit an 8 GB card. Using the full 16K context raises that to about 21.8 GB, which is a 24 GB card. On CPU, 16 GB of system RAM runs the 4-bit file at 4K context.

Sizes are the download sizes on Ollama's codellama tag list. The memory columns are arithmetic, not measurements: file size, plus the key-value cache, plus a 1 GB allowance for compute buffers. Code Llama 13B inherits Llama 2's layout of 40 layers and a hidden size of 5,120 with no grouped-query attention, so the cache costs 2 × 40 × 5,120 × 2 bytes = 819 KB per token. That is 3.4 GB at 4,096 tokens and 13.4 GB at the full 16,384, which is why context length matters more here than the quantisation level.

By graphics card

• 8 GB card: no 13B tag fits with a 4K context. Even the 5.4 GB q2_K file needs about 9.8 GB. Ollama will split the model between GPU and CPU, which works but is slow; CodeLlama 7B (3.8 GB) or a newer 7B code model is the better fit.
• 12 GB card: q4_0 at 4K context. Close other GPU applications first, or drop to q3_K_M for headroom.
• 16 GB card: Q4_K_M or Q5_K_M at 4K, or q4_0 with roughly 8K of context.
• 24 GB card: q4_0 through Q5_K_M with the full 16K context, or Q8_0 at 4K.

Ollama's context-length documentation gives a 4K default below 24 GiB of VRAM and 32K from 24 GiB, so on a 24 GB card it is worth setting the context to 16K explicitly, since that is this model's limit. The VRAM calculator runs the same sum for other models, and quantization explained covers what each level trades away.

The paper: Code Llama: Open Foundation Models for Code (arXiv 2308.12950)

The Code Llama paper is arXiv:2308.12950, by Baptiste Rozière, Jonas Gehring, Fabian Gloeckle and colleagues at Meta AI. It was submitted on 24 August 2023 and last revised on 31 January 2024; the current version also covers the 70B models. What it says, in the order most readers need it:

The official code is at github.com/meta-llama/codellama, which lists the unquantised 13B weights at 24 GB, and the Transformers-format weights are at codellama/CodeLlama-13b-hf.

Reported benchmarks

The paper reports HumanEval pass@1 of 36.0% for Code Llama 13B, 42.7% for the Instruct variant and 43.3% for the Python variant, with MBPP pass@1 of 47.0%, 49.4% and 49.0%. These are Meta's figures; this site has not run them.

Two things stand out. The step from 7B to 13B is small on HumanEval pass@1 (33.5% to 36.0%) and larger on MBPP (41.4% to 47.0%), while the step from 13B to 34B is much bigger on both. And within the 13B size, the Instruct and Python variants are about seven points ahead of the base model on HumanEval, so the base model is the one to pick only when you want raw completion or infilling.

The paper's multilingual table (MultiPL-E, pass@1) gives Code Llama 13B 39.1% on C++, 38.0% on Java, 34.2% on PHP, 29.6% on TypeScript, 27.3% on C# and 15.2% on Bash. Shell scripting is its weakest language by a wide margin.

Ollama setup and tags

Every tag below was checked against the Ollama tag list on 29 September 2026. One thing the short name hides: codellama:13b has the same ID as codellama:13b-instruct, so the default 13B download is the Instruct model, not the base model.

# Install Ollama (Linux; macOS and Windows use the installer from ollama.com)
curl -fsSL https://ollama.com/install.sh | sh

# Instruct, 4-bit, 7.4 GB — chat-style coding questions
ollama run codellama:13b

# Base model, 7.4 GB — completion and fill-in-the-middle
ollama pull codellama:13b-code

# Python specialist, 7.4 GB
ollama pull codellama:13b-python

# Smaller file for headroom on a 12 GB card, 6.3 GB
ollama pull codellama:13b-instruct-q3_K_M

Fill-in-the-middle

Infilling is the feature that still gives Code Llama a job in editor autocomplete. The base model takes a prefix and a suffix and writes what goes between them. Ollama's library page documents the prompt format with the 7B tag; the same format applies to 13b-code:

ollama run codellama:13b-code '<PRE> def compute_gcd(x, y): <SUF>return result <MID>'

For wiring a local model into an editor, see the Continue.dev with Ollama guide, and for current autocomplete picks the local autocomplete models comparison.

Base, Instruct or Python: which 13B to pull

The official repository is explicit that the base and Python models “are not fine-tuned to follow instructions” and should be prompted so that the answer is the natural continuation of the prompt. If a Code Llama model answers a question with more code comments instead of an answer, that is usually the reason.

What replaced CodeLlama 13B

For new work, use a newer code model. Qwen2.5-Coder 7B is a smaller download than CodeLlama 13B (4.7 GB against 7.4 GB), is Apache 2.0 licensed, has a 131,072-token context, and scores more than twice as high on HumanEval in its own technical report.

The scores below are from the instruct-model table of the Qwen2.5-Coder technical report (arXiv:2409.12186), published by Alibaba's Qwen team, who evaluated CodeLlama alongside their own models. Their CodeLlama-13B-Instruct HumanEval figure (40.2) differs slightly from the 42.7% in Meta's paper because the evaluation setups differ. Download sizes are from Ollama.

The site has pages for Qwen 2.5 Coder 7B, Qwen 2.5 Coder 14B, Qwen3-Coder-Next and Devstral; the current shortlist is on best local coding models, and the Ollama coding-model guide sorts them by VRAM tier.

Reasons to keep CodeLlama 13B are narrow. You may have a fine-tune or an evaluation built on it and need the same weights; you may be reproducing the paper; or a tool you use is configured for Code Llama's infill tokens. Within the family, the 34B model is the better use of a 24 GB card, though it does not support infilling.

Frequently asked questions

Sources