gpt-oss 20B and 120B: VRAM, Ollama tags and hardware

Overview

gpt-oss is a pair of open-weight reasoning models from OpenAI, published with a model card on 5 August 2025 under the Apache 2.0 licence. Anyone can download, modify and deploy the weights, including commercially; the training data and code are not released.

The family is gpt-oss-120b (116.8B total parameters, 5.1B active per token, 36 layers) and gpt-oss-20b (20.9B total, 3.6B active, 24 layers). Both use a Mixture of Experts design, a 128K context and the same 201,088-token tokenizer, and both let you set the reasoning effort to low, medium or high in the system prompt.

MoE Architecture and MXFP4

gpt-oss uses a Mixture of Experts (MoE) Transformer. These are the details from OpenAI's model card and the released config.json files:

The MoE design means the model activates only a fraction of its parameters per token: gpt-oss-120b uses 5.1B of its 116.8B parameters, about 4.4%. Every expert still has to sit in memory, which is why the 120b needs 80 GB even though only a few billion parameters are active at a time. Because generation speed for MoE models depends on how experts are routed and placed, this page gives no tokens-per-second figure.

The other unusual choice is the format. OpenAI post-trained the expert weights, over 90% of the parameters, directly in MXFP4, a 4-bit floating-point format with shared scales. The released checkpoints are 60.8 GiB (120b) and 12.8 GiB (20b), already the size of a typical 4-bit quantisation, and OpenAI ran all its evaluations on these weights. In practice that means there is no larger “full-quality” build to download and little to gain from quantising further.

The alternating dense and banded attention follows GPT-3's approach: half the layers only look back 128 tokens, which keeps the KV cache small. Even counting every layer at full length, the cache is at most 49 KB per token for the 20b and 74 KB for the 120b, so the full 128K context adds at most 6.4 GB and 9.7 GB.

Benchmarks (gpt-oss-120b)

Quick Start with Ollama

Ollama publishes two local tags: gpt-oss:20b (13.8 GB; gpt-oss and gpt-oss:latest point to the same file) and gpt-oss:120b (65.4 GB). Both are the MXFP4 weights; there are no Q4_K_M, Q8_0 or fp16 tags. The -cloud variants run on Ollama's servers, not your machine.

from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:11434/v1",
    api_key="ollama"  # required but unused
)

response = client.chat.completions.create(
    model="gpt-oss:20b",
    messages=[
        {"role": "system", "content": "Reasoning: medium"},
        {"role": "user", "content": "Explain MoE architectures"}
    ]
)
print(response.choices[0].message.content)

# Start Ollama with gpt-oss
ollama serve &
ollama pull gpt-oss:20b

# Launch Open WebUI
docker run -d -p 3000:8080 \
  --add-host=host.docker.internal:host-gateway \
  -e OLLAMA_BASE_URL=http://host.docker.internal:11434 \
  ghcr.io/open-webui/open-webui:main

Hardware Requirements

gpt-oss-20b needs about 15 GB with an 8K context (13.8 GB file + up to 0.4 GB of KV cache + ~1 GB of buffers); gpt-oss-120b needs about 67 GB (65.4 GB + up to 0.6 GB + ~1 GB). The table shows what that means for common GPUs and unified-memory machines. It is arithmetic from the file sizes, not a measurement.

vs Other Models

All of these are open-weight and locally runnable on the right hardware. Scores are each developer's own and come from different setups, so treat them as rough positioning.

Best Use Cases

Advanced Setup

# gpt-oss Modelfile for Ollama
FROM gpt-oss:20b

# 32K context adds at most ~1.6 GB of KV cache on the 20b
PARAMETER num_ctx 32768

# Reasoning effort is set in the system prompt
SYSTEM """Reasoning: high
You are an expert software engineer. Write clean,
well-documented code. When you're unsure, say so rather than guessing."""

# Build and run the custom model
ollama create my-gpt-oss -f Modelfile
ollama run my-gpt-oss

ollama run gpt-oss:120b
# in another terminal: PROCESSOR should read 100% GPU
ollama ps

from agents import Agent, Runner, OpenAIChatCompletionsModel
from openai import AsyncOpenAI

client = AsyncOpenAI(
    base_url="http://localhost:11434/v1",
    api_key="ollama"
)

agent = Agent(
    name="local-agent",
    model=OpenAIChatCompletionsModel(model="gpt-oss:20b", openai_client=client),
    instructions="You are a helpful research assistant."
)

result = Runner.run_sync(agent, "Summarize the key points of MoE models")
print(result.final_output)

Sources

• gpt-oss-120b & gpt-oss-20b Model Card (arXiv 2508.10925) — Architecture, parameter counts, MXFP4, benchmark tables
• OpenAI: Introducing gpt-oss — Official announcement
• Hugging Face: openai/gpt-oss-120b and openai/gpt-oss-20b — Weights, config and hardware notes
• GitHub: openai/gpt-oss — Official repository
• Ollama: gpt-oss tags — Download sizes (read from the registry on 2 November 2026)

Frequently Asked Questions