StarCoder 2 3B: Technical Analysis & Performance Guide
StarCoder2 3B is the smallest of BigCode's open code-completion models from February 2024. On Ollama, starcoder2:3b is a 1.7 GB download and needs about 3.2 GB of VRAM with its full 16K context, so it runs on a 4 GB graphics card and, with no GPU at all, on a laptop with 8 GB of RAM. It was trained on 17 programming languages. It completes code; it is not a chat model. This page has not benchmarked it, and every figure is taken from BigCode, Hugging Face or Ollama.
Updated: October 15, 2026
What hardware does StarCoder2 3B run on?
A 4 GB card runs every tag up to q5_K_M at the full 16K context. A 6 GB card runs q6_K and q8_0. An 8 GB card runs the fp16 file. CPU-only works: the default tag plus its cache is under 3 GB, so any machine with 8 GB of RAM can hold it.
“VRAM needed” is arithmetic, not a measurement: the file size, plus the key-value cache for the whole 16,384-token context, plus 1 GB for compute buffers. The cache comes from config.json: 30 layers × 2 key-value heads × 128 dimensions × 2 (keys and values) × 2 bytes = 30,720 bytes per token, which is 0.50 GB at full context. Grouped-query attention with only two key-value heads is why that number is so small.
| Ollama tag | Ollama download | GGUF file | VRAM needed (16K context) | Fits |
|---|---|---|---|---|
3b-q2_K | 1.1 GB | 1.15 GB | 2.7 GB | 4 GB card |
3b-q3_K_M | 1.5 GB | 1.51 GB | 3.0 GB | 4 GB card |
3b (default, q4_0) | 1.7 GB | 1.71 GB | 3.2 GB | 4 GB card |
3b-q4_K_M | 1.8 GB | 1.85 GB | 3.4 GB | 4 GB card |
3b-q5_K_M | 2.2 GB | 2.16 GB | 3.7 GB | 4 GB card, with little headroom |
3b-q6_K | 2.5 GB | 2.49 GB | 4.0 GB | 6 GB card |
3b-q8_0 | 3.2 GB | 3.22 GB | 4.7 GB | 6 GB card |
3b-fp16 | 6.1 GB | not in the GGUF repository | 7.6 GB | 8 GB card |
BigCode's model card prints its own memory footprints for the Transformers library, which land in the same range: 12,625 MB at float32, 6,312 MB at bfloat16, 3,434 MB in 8-bit and 1,995 MB in 4-bit.
With no GPU, Ollama loads the model into system RAM. The 1.7 GB default tag plus a 0.5 GB cache is small enough that an 8 GB laptop, a Raspberry Pi 5 with 8 GB, or an Intel Mac can all hold it. Completion speed on a processor depends on the machine and has not been measured here; for as-you-type autocomplete, which is this model's job, the relevant number is the latency of a short completion, not sustained tokens per second. See running LLMs on CPU only and the Raspberry Pi 5 LLM guide for what to expect on those machines.
Because the whole thing fits in a few gigabytes, there is little reason to drop below the default q4_0 tag. The one case for q8_0 is a card with 6 GB or more to spare while a larger chat model sits in the rest of it. The general rules are in Ollama system requirements.
What StarCoder2 3B is
StarCoder2 is a family of code models from the BigCode project, a collaboration led by Hugging Face and ServiceNow, described in the paper StarCoder 2 and The Stack v2: The Next Generation, submitted on February 29, 2024. The 3B model is the smallest of the three sizes.
| Spec | StarCoder2 3B |
|---|---|
| Parameters | 3B (3.03B in the weights file) |
| Architecture | Transformer decoder, 30 layers, 24 attention heads, 2 key-value heads (grouped-query attention) |
| Context window | 16,384 tokens, with sliding-window attention of 4,096 tokens |
| Vocabulary | 49,152 tokens |
| Training data | The Stack v2, 17 programming languages, 3+ trillion tokens |
| Training objective | Fill-in-the-Middle |
| Training hardware | 160 × A100 |
| Published | February 2024 (Hugging Face repository created November 29, 2023) |
| Licence | BigCode OpenRAIL-M v1 |
The point that matters most in use is on the model card: StarCoder2 is “not an instruction model and commands like ‘Write a function that computes the square root.’ do not work well.” You give it the start of some code, or the code around a gap, and it continues. That is the job an editor's autocomplete does, and the Fill-in-the-Middle training objective is exactly the skill of completing a gap given the text on both sides of it.
An earlier revision of this page said the model was trained “with emphasis on embedded systems and IoT” and supported “80+ programming languages”. Neither is true. The 3B and 7B were trained on 17 languages drawn from The Stack v2; only the 15B saw the 600-plus language set. BigCode does not publish a language-by-language breakdown for the 3B, so this page does not print one.
Is StarCoder a code-generating AI that runs on most computers?
Yes, in the 3B size. StarCoder2 3B is a code-generating model whose default download is 1.7 GB and whose working memory with the full context is about 3.2 GB, which is less than the free memory on most laptops sold in the last several years. Ollama runs it on Windows, macOS and Linux, with or without a graphics card. The 7B (4.0 GB) still fits an 8 GB card, and the 15B (9.1 GB) needs a 12 GB card, so the honest version of the sentence is: the 3B runs on most computers, the 15B runs on most gaming PCs.
What it will not do on any computer is chat. Ask it a question in English and it will try to continue your question as if it were a comment in a source file. If you want a small coding model you can talk to, the alternatives in the last section are the ones to look at.
StarCoder2 model sizes: 3B, 7B and 15B
| Model | Languages | Training tokens | Ollama tag | Download |
|---|---|---|---|---|
| StarCoder2 3B | 17 | 3+ trillion | starcoder2:3b | 1.7 GB |
| StarCoder2 7B | 17 | 3.5+ trillion | starcoder2:7b | 4.0 GB |
| StarCoder2 15B | 600+ | 4+ trillion | starcoder2:15b | 9.1 GB |
All three share the 16,384-token context and the Fill-in-the-Middle objective. The language count is the practical difference: if you work in a language outside the 17 the smaller models were trained on, the 15B is the one that has seen it. Figures are from the three model cards on Hugging Face.
Ollama tags and install
ollama pull starcoder2 with no tag downloads this 3B model, at q4_0, 1.7 GB. The larger sizes need their tag spelled out.
# StarCoder2 3B, q4_0, 1.7 GB (the default; a completion model)
ollama pull starcoder2:3b
# Higher-quality quantisation, 3.2 GB
ollama pull starcoder2:3b-q8_0
# Completion test: give it the start of a function, not a question
ollama run starcoder2:3b "def fibonacci(n):"- Every 3B tag lists a 16K context on Ollama, matching the model card. Ollama's own default context is 4,096 tokens; raise it with
OLLAMA_CONTEXT_LENGTHif you want the model to see a whole file. - Tags:
3b-followed by q2_K, q3_K_S, q3_K_M, q3_K_L, q4_0, q4_1, q4_K_S, q4_K_M, q5_0, q5_1, q5_K_S, q5_K_M, q6_K, q8_0 or fp16. There is no 3B instruct tag; the only Instruct build BigCode published is for the 15B. - No tool calling. The Ollama library page has no tools badge.
To use it for completion inside an editor, the Continue + Ollama setup guide walks through pointing an autocomplete provider at a local model, and best local autocomplete models covers the Fill-in-the-Middle prompt format these models expect. Full tag list: ollama.com/library/starcoder2/tags.
StarCoder2 benchmarks, as reported by BigCode
These scores are from the metadata of the three model cards. They are the publisher's figures and this site has not reproduced them.
| Benchmark | StarCoder2 3B | StarCoder2 7B | StarCoder2 15B |
|---|---|---|---|
| HumanEval (pass@1) | 31.7 | 35.4 | 46.3 |
| HumanEval+ (pass@1) | 27.4 | 29.9 | 37.8 |
| DS-1000 (pass@1) | 25.0 | 27.8 | 33.8 |
| CruxEval-I (pass@1) | 32.7 | 34.6 | 48.1 |
| GSM8K, PAL (accuracy) | 27.7 | 40.4 | 65.1 |
| RepoBench v1.1 (edit similarity) | 71.19 | 72.07 | 74.08 |
The HumanEval scores look low beside chat models because HumanEval asks for a function from a description, which is the instruction-style task these base models were not built for. RepoBench, which measures completing code inside a repository, is closer to what they do, and there the 3B sits only three points behind the 15B. That is the strongest argument for the 3B: for in-editor completion, most of the capability of the family arrives in the smallest file. Sources: 3B card, 7B card, 15B card.
Use cases and applications
StarCoder2 3B does one thing: it continues code, including filling a gap in the middle of a file. The uses that follow from that are narrower than an earlier revision of this page claimed, and more useful.
- Editor autocomplete. The primary use. Continue, and other editor extensions that speak the Ollama API, can send a Fill-in-the-Middle prompt and show the completion inline. A 1.7 GB model that fits beside your IDE is the point of the 3B.
- Autocomplete on a machine with no GPU. Because the file plus cache is under 3 GB, this is one of the few code models that is practical on an 8 GB laptop or a Raspberry Pi 5 for short completions.
- A second model beside a chat model. On a 12 GB or 16 GB card you can keep a chat model loaded for questions and this one for completions; Ollama's
OLLAMA_MAX_LOADED_MODELScontrols how many stay resident. - Fine-tuning experiments. BigCode's GitHub repository ships a fine-tuning script, and a 3B base model with a permissive-data training set is a cheap starting point for a domain-specific completer.
What it is not for: answering questions, explaining code, writing a function from a sentence, or agent work that needs tool calling. For those you want an instruction-tuned coding model; the Ollama coding-model guide lists them by VRAM tier.
StarCoder2 licence
All three sizes are under the BigCode OpenRAIL-M v1 licence agreement. An earlier revision of this page summarised it as “commercial use permitted”, which is incomplete: it is a responsible-use licence that permits commercial use but attaches use restrictions to the model and to anything derived from it. It is not Apache 2.0 or MIT. Read the agreement before building a product on the weights.
The model card adds a separate caution about output. The training data was filtered to permissively licensed and unlicensed code, but the model “can generate source code verbatim from the dataset”, and that code's own licence may require attribution. BigCode provides a search index for checking where a generated snippet came from. How this compares with other model licences is in which local models you can legally use at work.
Performance settings that exist, and ones that do not
An earlier revision of this page told Raspberry Pi users to export OLLAMA_GPU_MEMORY_FRACTION, OLLAMA_CPU_THREADS and OLLAMA_CONTEXT_SIZE. None of these is an Ollama setting; they are absent from the Ollama FAQ and Ollama ignores them. The settings that do matter for a completion model are these:
# Let the model see a whole file (Ollama's default is 4,096 tokens)
OLLAMA_CONTEXT_LENGTH=16384 ollama serve
# Keep it resident so the first completion after a pause is not slow
# (the default unloads a model after 5 minutes idle)
OLLAMA_KEEP_ALIVE=1h ollama serve
# Keep a chat model and this one loaded at the same time
OLLAMA_MAX_LOADED_MODELS=2 ollama serve- Context. Each extra 1,000 tokens of context costs about 31 MB of cache on this model, so running the full 16K is cheap. Do it if your editor sends large prefixes.
- Keep-alive. For autocomplete, the model should stay in memory between keystrokes. A generous
OLLAMA_KEEP_ALIVEmatters more than any quantisation choice. - Quantisation. With this little memory at stake, q4_0 against q8_0 is a quality decision, not a fit decision. On a card with room, take q8_0.
- Threads. On CPU, Ollama picks a thread count itself; there is no documented variable to force it.
Is it still worth running?
This section is opinion. StarCoder2 dates from February 2024, and newer small coding models cover both completion and chat in one download, with Fill-in-the-Middle support included. Qwen 2.5 Coder 1.5B is the closest newer model at this size, and the best local autocomplete models shortlist ranks the current options from 0.5B to 30B.
Two things still set StarCoder2 apart: a training set built with opt-out requests excluded and a public index for tracing output, and a licence written around responsible use. If provenance of the training data matters to you, it remains a reasonable completion model for a 4 GB card or a machine with no GPU. Stable Code 3B is the other completion model of the same vintage and size.
Frequently asked questions
How big is starcoder2:3b on Ollama?
1.7 GB for the default q4_0 tag. Other 3B tags run from 1.1 GB (q2_K) to 6.1 GB (fp16).
What devices can run StarCoder2 3B?
Any computer with about 3 GB of free memory: a 4 GB graphics card, an 8 GB laptop with no GPU, or a Raspberry Pi 5 with 8 GB of RAM. Speed on those devices has not been measured on this site.
When was StarCoder2 released?
The paper was submitted on February 29, 2024. The 3B repository on Hugging Face was created on November 29, 2023 and last modified on March 4, 2024.
Can I chat with StarCoder2 3B?
No. It only continues code. BigCode published an Instruct fine-tune for the 15B only; for a small model you can talk to, use an instruction-tuned coder instead.
What is the context window?
16,384 tokens, with a 4,096-token sliding window inside it. An earlier revision of this page said 8,192; the model card and every Ollama tag say 16K.
Where is the StarCoder2 source code?
At github.com/bigcode-project/starcoder2, which holds the quick-start and fine-tuning scripts. The weights are on Hugging Face.
Sources
- StarCoder2-3B model card: architecture, training data, memory footprints, scores, licence
- StarCoder2-7B model card and StarCoder2-15B model card: the sibling columns
- StarCoder 2 and The Stack v2 (arXiv:2402.19173)
- bigcode-project/starcoder2 on GitHub
- Ollama tag list for starcoder2
- second-state GGUF files: the file sizes in the hardware table
- Ollama FAQ: default context, environment variables
Build Real AI on Your Machine
RAG, agents, NLP, vision, and MLOps - chapters across 25 courses that take you from reading about AI to building AI.
Related Guides
Continue your local AI journey with these comprehensive guides
Go from reading about AI to building with AI
25 structured courses. Hands-on projects. Runs on your machine. Start free.

Written by
Barsha Mohapatra
Founder and Editor, Local AI Master · Founder, Little AI Master
Founder of Little AI Master and the editor behind every course and guide on Local AI Master. Years of tutoring students taught her where AI education actually fails.
- PILLARLocal AI Models Directory: Every Model Compared
- DeepSeek V4 Local Setup: 1M Context, MIT (2026)
- models/llama-3-1-8b
- models/deepseek-coder-v2-236b
- models/codestral-22b
- Gemini 3.1 Pro Review: 1M Context, MoE Thinking (2026)
- Qwen2-Audio 7B: ASR + Audio Understanding Setup (2026)
- models/qwen-2-5-3b
- models/llama-3-1-405b
- Bark by Suno AI: Open-Source Text-to-Audio Generation Guide
Grab the AI Starter Kit — career roadmap, cheat sheet, setup guide
No spam. Unsubscribe with one click.
Found your model? Now build something with it.
25 hands-on courses — RAG, agents, fine-tuning — all running locally. First chapter free, no card.