Llama 3.1 405B: benchmark scores and what it takes to run

Llama 3.1 405B MMLU, HumanEval and GPQA scores

Every number in this table is copied from Table 2 of Meta's paper, The Llama 3 Herd of Models (arXiv:2407.21783). Meta ran the evaluations, including the ones on the closed models, so treat the whole table as vendor-reported. This site has not run any of them. The paper calls the model “Llama 3 405B”; it is the same model that shipped as Llama 3.1 405B.

Three details that get lost when these scores are quoted. The 87.3 is the instruction-tuned model; the model card gives the pretrained model 85.2 on the same 5-shot MMLU. GPQA is 51.1 in the paper and 50.7 in the model card, and Meta does not explain the difference. And the often-repeated claim that the 405B beats GPT-4 is true of GPT-4 (0125) in this table, not of GPT-4o, which scores 89.1 on MMLU and 90.2 on HumanEval.

The model card also reports DROP F1 84.8 for the pretrained model and 84.95 to 85.08 on MMLU translated into Portuguese, Spanish and Italian for the instruct model. To see how these scores sit against newer local and hosted models, see the AI model leaderboard.

Llama 3.1 405B hardware requirements

The default 4-bit file is 243 GB and needs about 248 GB of memory with an 8K context: four 80 GB GPUs, three 96 GB GPUs, or a CPU or Apple silicon machine with 384 GB to 512 GB of memory. The smallest file, Q2_K, is 149 GB. The arithmetic is parameters × bits per weight: 405.85 billion × 16 bits ÷ 8 is 812 GB at fp16, and the 243 GB Q4_K_M file works out to 4.8 bits per weight.

File sizes are from Ollama's llama3.1 tag list. The memory column adds 4.2 GB of KV cache for an 8K context and a 1 GB allowance for compute buffers. The KV cache is 126 layers × 8 KV heads × 128 dimensions × 2 × 2 bytes, which is 516 KB per token: 16.9 GB at 32K and 67.6 GB if you use the full 128K window. The GPU and RAM columns are what that total fits into. They are arithmetic, not measurements.

In card terms, 80 GB means an NVIDIA A100 or H100 and 96 GB means an RTX PRO 6000 Blackwell. Eight 24 GB cards add up to 192 GB, which holds the Q2_K file on paper, but that is a rig with eight risers and two or three power supplies. A 24 GB or 32 GB gaming card on its own holds a tenth of the model.

Running from system RAM on a server board works with llama.cpp or Ollama and is slow, because every token reads through the whole file. No measured speed figure exists for that setup, so there is not one here. For builds that are realistic at home, see the local AI server guide, the RTX 5090 model picks and the VRAM calculator. Quantization explained covers what a 2-bit file costs in quality.

Architecture and training

Specification fields are from the model card; the layer and width figures are from the config.json of an ungated mirror, because Meta's repository sits behind a licence form.

16,000 H100s: what Meta has said about training

Meta's announcement says the 405B was trained on “over 15 trillion tokens” and that training was pushed “to over 16 thousand H100 GPUs”. The model card puts the 405B at 30.84 million GPU hours on H100-80GB hardware. Dividing one by the other gives 1,928 hours, or about 80 days, if all 16,000 GPUs ran for the whole job. That is my arithmetic from two Meta figures, not a duration Meta has stated. I could not find the “89 days” figure that circulates in any Meta source.

Inference speed: what is actually published

Meta has not published tokens-per-second figures for the 405B on H100s, and this site has not run the model, so there is no H100 speed table on this page.

The one widely quoted number is from Cerebras, which wrote on 18 November 2024 that it ran a customer workload on Llama 3.1 405B at 969 tokens per second. That is a hosted service on Cerebras's own wafer-scale hardware and the company's own claim. It says nothing about what a GPU server you own will do.

Ollama tags

llama3.1:405b is the instruct model at Q4_K_M, 243 GB. The 405b-text tags are the pretrained completion model at the same sizes. Check free disk space before you pull: the download alone is larger than most desktop SSDs.

# Instruct, Q4_K_M, 243 GB
ollama pull llama3.1:405b

# Smallest published file, Q2_K, 149 GB
ollama pull llama3.1:405b-instruct-q2_K

# What most people should pull instead: 43 GB
ollama pull llama3.3:70b

For llama.cpp, GGUF files including the IQ3 sizes Ollama does not carry are in mradermacher/Meta-Llama-3.1-405B-Instruct-GGUF. The original weights are at meta-llama/Llama-3.1-405B-Instruct.

What to run instead

Llama 3.3 70B, released on 6 December 2024, is the practical replacement. In the Llama 3.3 model card Meta lists it beside the 405B: MMLU with chain of thought 86.0 against 88.6, HumanEval 88.4 against 89.0, and IFEval 92.1 against 88.6 in the smaller model's favour. It is under a fifth of the size. The Llama 4 family followed in 2025.

Llama 4 Maverick is not a way down in size: its Ollama tag is 245 GB, the same class as the 405B. For picks by card, see the best LLM for 24 GB of VRAM and the best local models for a 24 GB GPU. The Llama 4 local setup guide covers the newer family.

Frequently asked questions

Sources

• Llama 3.1 model card — Meta: specifications, benchmark tables, training GPU hours, licence
• The Llama 3 Herd of Models — the paper, arXiv:2407.21783; Table 2 is the comparison table above
• Meta's Llama 3.1 announcement — training tokens and GPU count
• Ollama llama3.1 tags — every 405B tag with its download size
• Cerebras: Llama 3.1 405B at 969 tokens/s — the vendor's own speed claim
• Ollama llama3.3 tags and llama4 tags — sizes of the models to run instead