GPT-OSS: OpenAI's First Open-Source Model

Overview

GPT-OSS marks a watershed moment: OpenAI's first open-source model family, released under the permissive Apache 2.0 license. After years of keeping their models closed, OpenAI released two state-of-the-art models that anyone can download, modify, and deploy without restrictions.

The family includes GPT-OSS 120B (117B total parameters, 5.1B active per token) for production-grade inference and GPT-OSS 20B (21B total parameters, 3.6B active per token) for local and edge deployment. Both use Mixture of Experts (MoE) architecture for efficient inference.

MoE Architecture Deep Dive

GPT-OSS uses a Mixture of Experts (MoE) Transformer architecture with several key innovations:

The MoE design means the model activates only a fraction of its parameters per token. GPT-OSS 120B activates 5.1B of its 117B parameters — just 4.4% — giving it the inference speed of a ~5B model with the quality of a much larger one. However, all 117B parameters must fit in memory, which is why VRAM requirements remain high.

The alternating dense and sparse attention (similar to GPT-3's approach) balances quality with efficiency. Grouped multi-query attention with a group size of 8 further reduces memory usage during inference, enabling longer context windows on the same hardware.

Benchmarks (GPT-OSS 120B)

Quick Start with Ollama

from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:11434/v1",
    api_key="ollama"  # required but unused
)

response = client.chat.completions.create(
    model="gpt-oss:20b",
    messages=[
        {"role": "user", "content": "Explain MoE architectures"}
    ]
)
print(response.choices[0].message.content)

# Start Ollama with GPT-OSS
ollama serve &
ollama pull gpt-oss:20b

# Launch Open WebUI
docker run -d -p 3000:8080 \
  --add-host=host.docker.internal:host-gateway \
  -e OLLAMA_BASE_URL=http://host.docker.internal:11434 \
  ghcr.io/open-webui/open-webui:main

Hardware Requirements

vs Other Models

MMLU-Pro scores shown. All models compared are open-weight and locally runnable.

Best Use Cases

Advanced Setup

# GPT-OSS Modelfile for Ollama
FROM gpt-oss:20b

# Optimize for reasoning tasks
PARAMETER temperature 0.3
PARAMETER top_p 0.9
PARAMETER num_ctx 32768

# System prompt for coding assistant
SYSTEM """You are an expert software engineer. Write clean,
well-documented code. Explain your reasoning step by step.
When you're unsure, say so rather than guessing."""

# Build and run the custom model
ollama create my-gpt-oss -f Modelfile
ollama run my-gpt-oss

# Ensure both GPUs are visible
CUDA_VISIBLE_DEVICES=0,1 ollama serve

# Pull and run — Ollama splits layers automatically
ollama run gpt-oss:120b

from agents import Agent, Runner
from openai import AsyncOpenAI

client = AsyncOpenAI(
    base_url="http://localhost:11434/v1",
    api_key="ollama"
)

agent = Agent(
    name="local-agent",
    model="gpt-oss:20b",
    instructions="You are a helpful research assistant."
)

result = Runner.run_sync(agent, "Summarize the latest AI news")

Sources

• OpenAI: Introducing GPT-OSS — Official announcement
• GPT-OSS Model Card (arXiv 2508.10925) — Architecture and benchmark details
• Hugging Face: openai/gpt-oss-120b — Model weights and documentation
• GitHub: openai/gpt-oss — Official repository
• Ollama: gpt-oss — Ollama model library