Llama 3 8B:
Technical Analysis & Setup

Llama 3 8B architecture: layers, hidden size, FFN and heads

Llama 3 8B has 32 transformer layers, a hidden size of 4,096, a feed-forward intermediate size of 14,336, 32 attention heads and 8 key-value heads (head dimension 128), a 128,256-token vocabulary and 8,192 maximum positions. Every figure is from the model’s config.json. It is a dense decoder-only transformer with grouped-query attention, SwiGLU feed-forward blocks (SiLU activation) and RMSNorm; the model card adds that it was “trained on sequences of 8,192 tokens” and uses a 128K-token tokenizer, up from 32K in Llama 2.

From config.json (ungated mirror of Meta’s repository) and the Hugging Face model page. The 8 key-value heads are why the context cache is small: 2 × 32 layers × 8 heads × 128 × 2 bytes is 128 KB per token, about 1.07 GB for the full 8,192-token window.

Llama architecture table: dimensions, layers and heads from 7B to 70B

The same four knobs (hidden size, feed-forward size, layer count, head count) define every Llama generation. Llama 3 8B keeps the 7B shape of Llama 1 and 2 but widens the feed-forward block from 11,008 to 14,336, switches to grouped-query attention, and quadruples the vocabulary, which is where the extra 1.3 billion parameters come from. The figures are from each model’s config.json and the Hugging Face parameter counts.

Llama 1 configs from the huggyllama mirrors (7B, 13B, 30B, 65B) and the LLaMA paper; Llama 2 from the NousResearch -hf mirrors; Llama 3 70B from its config. “—” means multi-head attention, where every query head has its own KV head. The parameter counts are the exact Hugging Face figures rounded: Llama 1 7B is 6,738,417,664, Llama 2 70B is 68,976,653,312, Llama 3 70B is 70,553,706,496. Pages for Llama 2 7B, Llama 2 13B, Llama 2 70B and Llama 3 70B.

Llama 3 8B hardware requirements

Llama 3 8B needs about 7 GB of memory at Q4_K_M with its full 8K context, so an 8 GB GPU is the standard setup; the default 4.7 GB q4_0 tag needs about 6.8 GB. A 12 GB card covers Q8_0, a 24 GB card the unquantised fp16 build, and 16 GB of system RAM is the equivalent floor without a GPU. The arithmetic is parameters × bits per weight ÷ 8: 8.03B × 4.90 bits ÷ 8 = 4.92 GB at Q4_K_M.

File sizes are from bartowski/Meta-Llama-3-8B-Instruct-GGUF and Ollama’s tag list. The context cache is 1.07 GB for 8,192 tokens (arithmetic from the configuration above). “Memory needed” is file + that cache + a 1 GB allowance for buffers.

• System RAM, no GPU: the file loads into RAM instead, so the same totals apply; 16 GB of RAM covers every build up to Q8_0 with room for the operating system. See running an LLM on CPU only.
• Mac: unified memory follows the same arithmetic; a 16 GB Mac is comfortable, an 8 GB Mac is a squeeze that only the Q2 and Q3 builds fit. See running Llama 3 on a Mac.
• 6 GB card: Q2_K or Q3_K_S fit entirely; anything larger spills part of the model to system RAM, which works but is slower.
• Disk: the download size in the table, once per build you pull.

This site has not measured generation speed for this model, so none is quoted. For what else fits each tier see best LLM for 8 GB of VRAM, best LLM for 12 GB of VRAM and the Ollama RAM and VRAM table; the build names are explained in quantization explained.

Llama 3 8B size in GB: every Ollama tag

The default llama3:8b is 4.7 GB, and it is the q4_0 format, not Q4_K_M. Ollama carries fifteen instruct builds of the 8B from 3.2 GB to 16 GB, all with the 8K context; the sizes below are copied from the tag list. Note that llama3 with no tag is this 8B model, not the 70B.

Sizes from ollama.com/library/llama3/tags. The original BF16 safetensors on Hugging Face are 8.03B × 2 bytes, about 16 GB. For the whole Llama family in one place: 70B models are 40–43 GB at 4-bit, and llama3.1:8b is 4.9 GB because its default is Q4_K_M.

Benchmarks published by Meta: the Llama 3 8B MMLU score

Llama 3 8B Instruct scores 68.4 on MMLU (5-shot) on Meta’s model card; the base model scores 66.6. These are Meta’s own results from its internal evaluation library. The card compares Llama 3 only with Llama 2; it contains no figures for any other company’s models, and nothing on it supports the 69.9 MMLU or 61.2 HumanEval that earlier versions of this page quoted.

The reading of these tables that matters for a buyer: the 8B beats the Llama 2 13B on every row of both tables except TriviaQA and QuAC, so “8B vs 13B” is settled by the card itself. Against the Llama 3 70B it is a different class of model, 14 points behind on MMLU and 20 on MATH. For the 3.1 refresh’s numbers on the same rows, see the Llama 3.1 8B page; what the benchmark names mean is covered in the guide to AI benchmarks.

Llama 3 8B context length: 8,192 tokens

The context length is 8,192 tokens. Meta’s card says the models were trained on sequences of 8,192 tokens, the configuration sets 8,192 maximum positions, and every Ollama tag is listed with an 8K window. Llama 3 8B has no RoPE scaling, so asking it for more than 8K does not work the way it does on Llama 3.1; that longer window is the main reason to pull the newer model instead.

Ollama’s documentation says it opens a 4,096-token window by default; to use the full 8K, run /set parameter num_ctx 8192 inside ollama run, start the server with OLLAMA_CONTEXT_LENGTH=8192 ollama serve, or pass num_ctx in the API request. The cache figures are 2 × 32 layers × 8 KV heads × 128 × 2 bytes = 131,072 bytes per token; totals add a 1 GB buffer allowance.

Installation

With Ollama installed, pull the tag that matches your memory. Every tag below is on ollama.com/library/llama3/tags. Ollama’s CLI takes no --num-ctx or --num-gpu flags; those are parameters set inside a session, in a Modelfile, or through the API.

# Default 8B build: q4_0, 4.7 GB — for an 8 GB card
ollama pull llama3:8b
ollama run llama3:8b

# Q4_K_M, 4.9 GB — the newer 4-bit format
ollama pull llama3:8b-instruct-q4_K_M

# Q8_0, 8.5 GB — for a 12 GB card
ollama pull llama3:8b-instruct-q8_0

# Use the full 8K context inside a session (default is 4,096)
/set parameter num_ctx 8192

For a quantisation Ollama does not carry, such as the 2.4 GB IQ2_XXS file, download the GGUF from the bartowski repository linked above and load it in llama.cpp or LM Studio. The original weights are on Hugging Face; the repository is gated, so you accept Meta’s licence before downloading. Serving it behind an API or a chat interface is covered in the Open WebUI setup guide.

Unless you need this exact model, pull llama3.1:8b instead. It is 4.9 GB, runs on the same hardware and has a 128K context.

Model specifications

From Meta’s Llama 3 model card; the layer and head counts are from the model’s configuration file.

The Llama 3 technical report

The Llama 3 paper is “The Llama 3 Herd of Models”, arXiv 2407.21783, published in July 2024 with the Llama 3.1 release. It covers the 8B, 70B and 405B models, the data mix, the scaling decisions and the evaluation method; it is the document to cite for architecture questions that the model card does not answer. Note that arXiv 2402.08945, which several pages (including an earlier version of this one) cited as the Llama 3 technical report, is an unrelated mathematics paper.

For the April 2024 release itself, Meta published the model card and an evaluation-details file in the same repository rather than a paper; the benchmark tables above are from that card. The original 2023 LLaMA paper, arXiv 2302.13971, is the source for the 7B/13B/33B/65B dimensions in the architecture table.

Licence

Llama 3 is released under the Meta Llama 3 Community License (version release date April 18, 2024). Meta describes it as a custom commercial licence. It is not an open-source licence such as MIT or Apache 2.0. The terms that matter in practice:

• Commercial use is permitted, along with modification and redistribution.
• If you distribute the model or a product that uses it, you must include the licence and “prominently display ‘Built with Meta Llama 3’” on a related website, interface, blog post, about page or documentation.
• A model you train or fine-tune from it and distribute must “include ‘Llama 3’ at the beginning of any such AI model name”.
• If, on the release date, your products or services had more than 700 million monthly active users in the preceding month, you must request a separate licence from Meta.
• Use is also subject to Meta’s Acceptable Use Policy; the card lists languages other than English as out of scope unless you fine-tune for them.

This is a summary, not legal advice; read the licence before relying on it. Other model licences are compared in LLM licences for commercial use.

What has replaced it

Llama 3 8B was superseded three months after release, and the 8K context is the main reason to move on:

• Llama 3.1 8B — released 23 July 2024 with a 128K context, eight supported languages and tool calling, on the same 8,030,261,248 parameters. Meta’s 3.1 card puts its MMLU at 69.4 (instruct) against 68.5 for this model on the same evaluation. It is the one to pull today.
• There is no Llama 3.3 8B. The Llama 3.3 model card lists a single 70B size, so searches for “Llama 3.3 8B” land on either this model or the 3.1 8B. See Llama 3.3 70B.
• Llama 3.2 3B and Llama 3.2 1B — smaller text models for machines where even 4.7 GB is too much.
• Llama 4 Scout — Meta’s next generation accepts images as well as text, but its Ollama tag is 67 GB; it is not a replacement for anything that runs on an 8 GB card.

If you have 8 or 12 GB and want the strongest model that fits rather than this model specifically, the best Ollama models list is kept current. Llama 3 8B is still worth keeping if you already have it and your prompts fit in 8K tokens.

Frequently asked questions

Sources

• Meta Llama 3 model card — release date, context, training data, knowledge cutoff, benchmark tables
• Meta Llama 3 Community License — the licence text
• Meta’s Llama 3 announcement
• The Llama 3 Herd of Models (arXiv 2407.21783) and LLaMA: Open and Efficient Foundation Language Models (arXiv 2302.13971)
• Meta-Llama-3-8B-Instruct on Hugging Face — original weights, parameter count
• config.json (ungated mirror) — layers, hidden size, FFN, heads, KV heads, vocabulary, positions
• Ollama tag list — every tag with its download size
• bartowski GGUF files — the file sizes in the hardware table
• Ollama FAQ — the 4,096-token default context window
• Llama 3.1 model card and Llama 3.3 model card — successors