AI Context Windows Explained: 4K vs 128K vs 1M Tokens

Published November 2, 2025 • Updated November 2, 2026 • 15 min read

A context window is the maximum number of tokens a model can hold at once: your prompt, the files you pasted, the conversation so far, and everything the model has generated in reply. When an assistant "forgets" code you discussed ten minutes ago, or refuses a large paste, you have hit it. "128K" means 131,072 tokens on almost every model that advertises it, which is roughly 96,000 English words, about half a megabyte of plain text, or on the order of 6,500 lines of code. This guide gives you the exact numbers for every common size, where the GPT-4 8K/32K/128K figures came from, what the current frontier models offer, what the open-weight models you can run with Ollama offer, and, because this is a local-AI site, how much memory each extra thousand tokens actually costs you.

Quick Summary: Context Window Sizes at a Glance

Words use Anthropic's published rule of thumb (200K tokens ≈ 150K words; 1M ≈ 555K words on its current tokenizer, 750K on earlier ones). Plain-text size assumes ~4 characters per token. Lines of code use Google's figure that 1M tokens ≈ 50,000 lines at 80 characters, i.e. about 20 tokens per line. All three are estimates that move with the language and the tokenizer; the token counts are exact.

Before going deeper, the open-weight side of this table is covered model by model on our best local AI coding models ranking and the Qwen3 Coder guide.

What Is a Context Window?

Everything a language model knows about your problem arrives through the context window. The model has no other memory of the session: no file system, no scrollback, nothing it read earlier unless it is still inside the window. The window is measured in tokens, not characters or words, and it is shared between everything you send and everything the model writes back.

For a developer that means the window decides:

• how many files the model can see at once,
• whether it can follow a call from one file into another,
• how long a conversation can run before the beginning falls out,
• how much of an agent's tool output (test logs, diffs, directory listings) survives to the next step,
• and, locally, how much memory the model needs on top of its weights.

Tokens, not words

A token is a chunk of text from the model's vocabulary: a common word, part of a rare word, a punctuation mark, a run of spaces. English prose averages around four characters per token, which is why 1,000 tokens is often quoted as about 750 words; Anthropic's model documentation puts it at "200k tokens is roughly 150k words", the same ratio. Source code tokenizes worse than prose because of symbols, indentation and identifiers like calculateUserBalance, which may split into several tokens. The practical consequence is that a 300-line file is usually 4,000–8,000 tokens, not 2,000.

Context window vs memory

The window is often called the model's "working memory", and the distinction from real memory matters. Nothing the model reads is learned: the weights do not change during a conversation, so once text leaves the window it is gone. When a chat product appears to remember you across sessions, it is storing notes outside the model and pasting the relevant ones back into the window at the start of the next conversation, which is the same mechanism as retrieval (RAG). For coding this means a tool that "knows your repository" is either indexing it and retrieving files into the window, or you are paying for those files in tokens on every request.

Both directions count. If you paste 100K tokens of code into a 128K window, the model has 31K tokens left for its answer and for the rest of the conversation, and most APIs also cap the reply separately (the current Claude models cap output at 128K tokens; the GPT-6 family at 128K; Gemini 2.5 and 3 at 64K).

How Much Is 128K Tokens?

128K tokens is 131,072 tokens (2¹⁷), which is about 98,000 words of English, roughly 512 KB of plain text, or in the region of 6,500 lines of code. Vendors write "128K" because the window is a power of two, the same way "4K" means 4,096. The exceptions are the vendors that quote decimal numbers: Anthropic's "200K" is 200,000, and OpenAI's current models are listed at "1.05M", which is 1,048,576 rounded.

A few conversions people search for, all arithmetic from the figures above:

• 128K context in bytes: 131,072 tokens × ~4 characters ≈ 524,000 characters, about 0.5 MB if the text is ASCII. Code with heavy indentation and symbols tokenizes closer to 3 characters per token, so the same window may hold only 0.4 MB of source.
• 128K in pages: at roughly 500 words per page, about 190 pages of prose.
• 128K in files: at 20 tokens per line, 6,500 lines, so about twenty 300-line source files, or one large generated file. That is enough for a feature and its tests, not for a repository.
• 128K vs 32K: four times the text. 1M vs 128K: eight times.

The honest caveat, which the rest of this page comes back to: a model that accepts 128K tokens does not necessarily use all 128K well.

Context Window Sizes Compared: 4K, 32K, 64K, 128K and 1M

4K–8K tokens holds one file and a short conversation. It is what Ollama gives you by default (the FAQ says 4,096), and it is why a freshly installed local model seems to forget the start of a long paste: the start was silently dropped. Fine for a single-function question, a regex, a stack trace. Not fine for "refactor this module".

16K–32K tokens holds a handful of related files. 32,768 is the value Qwen2.5 ships in its config before YaRN scaling, and it is Ollama's automatic default when the machine has 24–48 GiB of VRAM. A feature spread over five or six files, with room to discuss it, fits here.

64K tokens is the number Ollama's own context-length documentation recommends as a minimum for "web search or coding tools", because agent tool output (test runs, grep results, file reads) is bulky. It is also the maximum output of Gemini 2.5 and 3 models.

128K tokens became the mainstream ceiling with GPT-4 Turbo in late 2023 and is now the floor for serious open-weight models: Llama 3.1 and 3.3, gpt-oss, the DeepSeek-R1 distills, Qwen2.5-Coder (with YaRN) and even Phi-3.5 Mini all advertise it. Twenty files, a long debugging session, a module and its tests.

200K–256K tokens: Claude 3 through 4.x offered 200,000; Qwen3-Coder, Qwen3-Coder-Next and Devstral Small 2 ship 262,144 natively. This is where "give the agent the whole service" starts to be realistic.

1M tokens is the current frontier across Anthropic, OpenAI and Google, and Qwen3-Coder reaches it with YaRN scaling. Google's documentation describes 1M tokens as about 50,000 lines of code or eight average English novels. Whole repositories of moderate size fit, which is the point, but see the section on whether the model actually uses it.

10M tokens exists on one open-weight model, Llama 4 Scout, whose config lists a 10,485,760-token maximum. Nobody runs it at that length locally: the KV cache alone would be hundreds of gigabytes (arithmetic below).

GPT-4 Context Window History: 8K, 32K and 128K

GPT-4 launched in March 2023 with an 8,192-token window (gpt-4-0314, later gpt-4-0613), a 32,768-token variant (gpt-4-32k), and then in November 2023 GPT-4 Turbo (gpt-4-1106-preview, later gpt-4-turbo) with 128,000 tokens. Those three numbers, 8K, 32K and 128K, are the ones people still search for, and OpenAI's deprecations page is the primary record: it lists the 8K context for the original gpt-4 snapshots, the gpt-4-32k models as shut down on 6 June 2025, and the 128K context for gpt-4-1106-preview and gpt-4-turbo, which are scheduled for removal on 23 October 2026.

Why the jump mattered: 8K was too small to hold a real file plus a conversation, 32K cost more per token and was rationed, and 128K was the first window where "paste the whole module" worked for most codebases. Every open-weight model that followed treated 128K as the target to match, which is why the local models in the table below cluster there.

Current Frontier Context Windows (Vendor Documentation, October 2026)

The three big API vendors now all document a 1-million-token window on their current models. Read from their model pages on 15 October 2026:

Two things changed since this guide was first written. Pricing is quoted per million tokens, and on Anthropic's documentation prompt-cache reads cost 10% of the base input price (5% on Opus 5.5 and Sonnet 5.5, 2.5% on Fable 5.1), which is what makes repeatedly sending the same 300K tokens of repository affordable. And the older ceilings (GPT-4 Turbo 128K, Claude 4 200K, Gemini 2.5 Pro 1M) are now history rather than the comparison. For the generation just before the current one, see our best Claude model for coding guide (which keeps a table of older Claude models, including Opus 4.8) and the GPT-5.5 and Gemini 3.1 Pro pages.

Local Model Context Windows (From the Model Cards)

Open-weight models you can run with Ollama advertise anything from 2K to 10M tokens, but the number that matters is the one you can afford in memory. Figures below are from each model's card or config.json; the Ollama column is what the tag page prints.

The pattern: everything from 2024 onward advertises 128K or more, and the 2025–2026 coding models advertise 256K natively. The two caveats are that an advertised length is often an extended length (RoPE scaling, YaRN, LongRoPE) rather than the length the model was mostly trained at, and that nothing stops you from asking Ollama for a window the model cannot fit in memory, in which case it spills to system RAM and slows to a crawl. Our RoPE, YaRN and NTK guide explains the extension mechanisms.

Ollama's Context Window: Default and How to Change It

Ollama does not give you the model's advertised window by default. The Ollama FAQ states, "By default, Ollama uses a context window size of 4096 tokens." The newer context-length page describes a default that scales with VRAM instead: 4K below 24 GiB, 32K between 24 and 48 GiB, and 256K at 48 GiB or more, with cloud models set to their maximum. The Modelfile reference still prints "Default: 2048" for the num_ctx parameter. Which one you get depends on your Ollama version and hardware; what does not change is that you should set it yourself for coding, and the docs recommend at least 64,000 tokens for coding tools.

Three documented ways to set it:

• Server-wide: OLLAMA_CONTEXT_LENGTH=65536 ollama serve
• Per session inside ollama run: /set parameter num_ctx 65536
• Per request through the API: pass num_ctx in the options object of /api/generate or /api/chat

Then run ollama ps. The PROCESSOR column should read 100% GPU; any CPU share means the larger context pushed part of the model or its cache out of VRAM.

Two more documented switches trade memory for context. Flash attention is on automatically where supported (OLLAMA_FLASH_ATTENTION=1 forces it), and with it enabled OLLAMA_KV_CACHE_TYPE can be set to q8_0 or q4_0 instead of the default f16, which cuts the cache in the next section to roughly half or a quarter.

The KV Cache: Why a Bigger Context Costs Memory

Every token in the window keeps a key and a value vector in every attention layer, and that cache lives in memory alongside the weights. Its size is a fixed amount per token, so it grows linearly with the context you ask for:

KV bytes per token = 2 (key + value) × layers × KV heads × head dimension × 2 bytes (f16)

The three architecture numbers come from each model's config.json. Worked out:

f16 cache, before Ollama's q8_0 (≈½) or q4_0 (≈¼) options. gpt-oss alternates full and 128-token sliding-window layers, so its real figure is lower than the bound shown.

You may read elsewhere that attention memory grows with the square of the context, so that doubling the window quadruples the memory. That describes the full attention-score matrix in a naive implementation. FlashAttention (Dao et al., 2022) computes attention in tiles without ever storing that matrix, and llama.cpp, Ollama and vLLM all use flash-attention-style kernels where the hardware supports them. With a modern engine the memory that grows is the KV cache above, and it grows linearly: twice the context, twice the cache. The compute for reading a long prompt still grows faster than linearly, which is why prefill on a huge paste is slow (next sections).

Three things fall out of the table. First, the cache is about architecture, not parameter count: Phi-3.5 Mini, a 3.8B model with no grouped-query attention (32 KV heads), needs more cache per token than Llama 3.3 70B (8 KV heads). Its advertised 128K context would need 48 GiB of cache on top of a 2.2 GB download. Second, a 24 GB card running Qwen2.5-Coder 32B (20 GB of weights) has roughly 4 GB left, which is a 16K context at f16 or 32K at q8_0, whatever the model card advertises. Third, the newest coding models were designed around this: Qwen3-Coder-Next's hybrid attention keeps the whole 256K window under 6 GiB, which is why "256K context on a Mac" is a real sentence for it and not for a 2024 model. The mechanics, including paged attention, are in our KV cache guide.

Does the Model Actually Use the Whole Window?

No, not reliably, and this has been measured rather than guessed. Liu et al.'s "Lost in the Middle" (July 2023) found that "performance is often highest when relevant information occurs at the beginning or end of the input context, and significantly degrades when models must access relevant information in the middle of long contexts." The RULER benchmark (Hsieh et al., April 2024) evaluated 17 long-context models on 13 tasks and reported that "only half of them can maintain satisfactory performance at the length of 32K" despite advertising 32K or more.

Model cards now publish RULER rows, which is the number to read before trusting an advertised window. Microsoft's Phi-3.5 Mini card, for example, reports a RULER score of 63.6 at 128K against 77.0 for Llama 3.1 8B and 19.0 for Mistral Nemo 12B, with averages across lengths of 84.1, 88.3 and 66.2. All three models advertise 128K; one of them is barely usable there.

For coding the practical rules are:

1. Put the files that matter first and last; the middle of a long paste is where recall is weakest.
2. Treat the last third of any advertised window as best-effort.
3. Prefer a model whose card publishes RULER or a similar long-context score, and read the number at the length you intend to use.
4. Retrieve, do not dump: the sections below are about keeping the window small on purpose.

How Context Length Affects Speed

Longer prompts cost time twice: once to read them, and a little more on every token generated afterwards. Reading the prompt ("prefill") is compute-bound and scales with the number of tokens, so a 100K-token paste takes noticeably longer to start answering than a 4K one, on any hardware. Generating each new token then has to attend over everything in the cache, so per-token speed drifts down as the conversation grows. On a local machine there is a third cost: if the cache no longer fits in VRAM, Ollama offloads to system RAM and generation speed falls off a cliff. That is the spill ollama ps shows.

Hosted APIs hide most of the prefill cost with prompt caching, which is why Anthropic prices cache reads at a tenth of the input rate. Locally, Ollama keeps the model loaded for five minutes by default (OLLAMA_KEEP_ALIVE changes it) and reuses the cache for a continuing conversation, so the expensive moment is the first big paste, not every turn. This page prints no seconds-per-request figures because they depend entirely on hardware and prompt; the bandwidth arithmetic for per-token speed is on our coding models page.

How to Use Context Wisely for Coding

The window is a budget. Spending it well matters more than having a bigger one.

Select, do not dump. Give the model the file you are changing, its direct imports, the relevant config and the tests. Cursor's @-mentions, Continue.dev's context providers and Cline's file reads all exist so you do not paste the repository.

Retrieve. For questions across a large codebase, index it once and retrieve the handful of files that match the question. A retrieval step turns a 1M-token repository into a 10K–30K-token prompt that a 16GB local machine can handle, and it sidesteps the lost-in-the-middle problem by never having a middle.

Scope, then expand. Ask "which files handle authentication?" with a small context, then load only those files for the real question. Two cheap calls beat one enormous one.

Start fresh for unrelated tasks. A long session carries every earlier exchange in the window. When you switch from one feature to another, open a new chat; locally this also frees the cache.

Keep useful comments, strip noise. Stripping comments and whitespace saves tokens but removes exactly the hints a model uses to understand intent. Remove generated files, vendored code and lockfiles from what you paste; leave the docstrings.

Set the window on purpose. Locally, set num_ctx to what your memory allows using the table above, not to the advertised maximum. An agent with a 64K window that stays on the GPU will finish before an agent with a 256K window that spilled to RAM.

Conclusion: Choosing the Right Context Size

Use 4K–16K for single-file questions, quick fixes and anything a small local model handles on a laptop. Use 32K–64K for a feature across several files and for any coding agent, which is also the range a 16–24 GB GPU can afford at f16 for a 7B–14B model. Use 128K–256K for a module and its tests, or a long agent run, on a 24 GB+ card with a quantised cache or on a 32 GB+ Mac. Use 1M on a hosted model when you genuinely need a whole repository in view and have read its long-context score first.

Golden rule: start with the smallest window that might work and grow it when you hit the wall, because every extra token costs memory locally, money remotely, and attention everywhere.

For the models themselves, see our best local AI coding models ranking, the Qwen3 Coder guide for the open-weight 1M route, and the AI coding tools guide for which editors expose context controls.

Frequently Asked Questions

What is a context window in AI? The maximum number of tokens a model can process at once: the prompt, any files included, the conversation so far and the model's own reply all share it. Anything outside it does not exist for the model.

How many words is 128K tokens? About 98,000 English words, using the 0.75-words-per-token ratio implied by Anthropic's documentation (200K tokens ≈ 150K words). Code is denser in tokens, so 128K tokens of source is fewer "words" and roughly 6,500 lines.

Is 128K tokens 128,000 or 131,072? 131,072 on models that define it as 2¹⁷, which includes GPT-4 Turbo, Llama 3, gpt-oss and the DeepSeek-R1 distills. Vendors round to "128K". Anthropic's 200K is a decimal 200,000.

What is a 64K context window good for? It is the minimum Ollama's documentation recommends for coding tools, and the output limit on Gemini 2.5 and 3. For a local agent it is the point where tool output stops being truncated mid-task.

What was GPT-4's context window? 8,192 tokens at launch in March 2023, with a 32,768-token gpt-4-32k variant; GPT-4 Turbo in November 2023 raised it to 128,000. OpenAI's deprecations page records all three.

How much memory does a 128K context need locally? It depends on the architecture, not the size: 16 GiB of KV cache for Llama 3.1 8B, 32 GiB for Qwen2.5-Coder 32B, 48 GiB for Phi-3.5 Mini, 3 GiB for Qwen3-Coder-Next, all at f16 and on top of the weights. Ollama's q8_0 cache option halves those numbers.

Does a bigger context window make the model smarter? No. It lets the model see more, but published long-context tests (Lost in the Middle, RULER) show recall degrading well before the advertised limit on most models. Smaller, targeted context usually produces better answers than a full dump.

Which local model has the largest usable context? Qwen3-Coder-Next and Qwen3-Coder 30B-A3B advertise 262,144 tokens natively and have the smallest KV cache per token of the models on this page, so they are the ones that can actually be run at a long context on consumer hardware. Llama 4 Scout advertises 10M, but its cache at that length is far beyond any desktop.