Llama 3 70B:
Technical Analysis & Setup

Llama 3 70B hardware requirements

Llama 3 70B needs about 46 GB of memory at Q4_K_M with its full 8K context, so the standard setup is 48 GB of VRAM: two 24 GB cards or one 48 GB card. The 26 GB Q2_K build fits a single 32 GB card. A single 24 GB card holds only the 19 GB IQ2_XXS build, or runs a larger build with part of the model in system RAM. The arithmetic is parameters × bits per weight ÷ 8: 70.55B × 4.82 bits ÷ 8 = 42.5 GB at Q4_K_M.

File sizes are from bartowski/Meta-Llama-3-70B-Instruct-GGUF and Ollama's tag list. The context cache is arithmetic from the model's config.json: 80 layers × 8 key-value heads × 128 dimensions, stored twice at 2 bytes, is 328 KB per token, or 2.7 GB for 8,192 tokens. “Memory needed” is file + that cache + a 1 GB allowance for buffers.

• One 24 GB card: the 40 GB default does not fit. Ollama will split the model between the card and system RAM, which works but is slower than running fully on the GPU; you need about 48 GB of combined memory.
• Mac: the same totals apply to unified memory, so the 40–43 GB builds need a 64 GB machine. See running Llama 3 on a Mac.
• No GPU: the file loads into system RAM, so 64 GB of RAM covers Q4_K_M.
• Disk: the download size in the table, once per build you pull.

This site has not measured generation speed for this model, so none is quoted. For what else fits each tier see best LLM for 24 GB of VRAM, best LLM for 32 GB of VRAM and the Ollama RAM and VRAM table; splitting a model across cards is covered in the Ollama multi-GPU guide, and the build names in quantization explained.

Llama 3 vs 3.1 vs 3.3 70B sizes on Ollama

All three 70B generations have the same 70,553,706,496 parameters, so they take the same memory at the same quantisation. The reason llama3:70b is 40 GB and the other two are 43 GB is that the Llama 3 default tag is the older q4_0 format while the 3.1 and 3.3 defaults are Q4_K_M.

Sizes are from the Ollama tag lists for llama3, llama3.1 and llama3.3. Note that llama3 with no tag is the 8B model (4.7 GB), not the 70B.

Model specifications

From Meta's Llama 3 model card; the layer and head counts are from the model's configuration file.

Benchmarks published by Meta

Llama 3 70B Instruct scores 82.0 on MMLU (5-shot) on Meta's model card; the base model scores 79.5. These are Meta's own results from its internal evaluation library. The card compares Llama 3 only with Llama 2; it contains no figures for any other company's models.

For the base model the card reports MMLU 79.5, ARC-Challenge 93.0, Winogrande 83.1, CommonSenseQA 83.8, BIG-Bench Hard 81.3, TriviaQA-Wiki 89.7 and DROP 79.7. Earlier versions of this page quoted an MMLU of 79.2 and a HumanEval of 67.0; neither figure is on Meta's card and both have been removed. What the benchmark names mean is covered in the guide to AI benchmarks.

Installation

With Ollama installed, pull the tag that matches your memory. Every tag below is on ollama.com/library/llama3/tags.

# Default 70B build: q4_0, 40 GB — for 48 GB of VRAM
ollama pull llama3:70b
ollama run llama3:70b

# Q4_K_M, 43 GB
ollama pull llama3:70b-instruct-q4_K_M

# Q2_K, 26 GB — for one 32 GB card
ollama pull llama3:70b-instruct-q2_K

For a quantisation Ollama does not carry, such as the 19 GB IQ2_XXS file, download the GGUF from the bartowski repository linked above and load it in llama.cpp. The original weights are on Hugging Face; the repository is gated, so you accept Meta's licence before downloading.

Unless you need this exact model, pull llama3.3:70b instead. It is 43 GB, runs on the same hardware and has a 128K context.

Licence

Llama 3 is released under the Meta Llama 3 Community License. Meta describes it as a custom commercial licence. It is not an open-source licence such as MIT or Apache 2.0. The terms that matter in practice:

• Commercial use is permitted, along with modification and redistribution.
• If you distribute the model or a product built on it, you must include the licence and display “Built with Meta Llama 3”.
• A model you train or fine-tune from it and distribute must have a name beginning with “Llama 3”.
• A company with more than 700 million monthly active users must request a separate licence from Meta.
• Use is also subject to Meta's Acceptable Use Policy.

This is a summary, not legal advice; read the licence before relying on it.

What has replaced it

Llama 3 70B has been superseded twice at the same size, and the 8K context is the main reason to move on:

• Llama 3.1 70B — released 23 July 2024 with a 128K context. Meta's card for it lists the longer context window, multilingual input and output, and tool integration as the new capabilities.
• Llama 3.3 70B — the most recent 70B text model in the Llama 3 line, also 128K context. It is the one to pull today.
• Llama 4 Scout — Meta's next generation accepts images as well as text. The Ollama llama4:scout tag is 67 GB, so it needs more memory than any 70B build at 4-bit.

If you have 48 GB of VRAM and want the strongest model that fits, rather than this model specifically, the best Ollama models list is kept current. Llama 3 70B is still worth keeping if you already have it and your prompts fit in 8K tokens.

Frequently asked questions

Sources

• Meta Llama 3 model card — release date, context, training data, benchmark tables
• Meta Llama 3 Community License — the licence text
• Meta's Llama 3 announcement
• Meta-Llama-3-70B-Instruct on Hugging Face — original weights
• Ollama tag list — every tag with its download size
• bartowski GGUF files — the file sizes in the hardware table
• Llama 3.1 model card and Llama 3.3 model card