StarCoder 2 15B: Technical Analysis & Performance Guide

What hardware does StarCoder2 15B run on?

A 12 GB card runs the default tag. A 16 GB card runs q4_K_M through q6_K. A 24 GB card runs q8_0. No 15B file fits an 8 GB card with the full context, so on 8 GB use the 7B or 3B model.

“VRAM needed” is the file size, plus the key-value cache for the whole 16,384-token context, plus 1 GB for compute buffers. The cache comes from config.json: 40 layers × 4 key-value heads × 128 dimensions × 2 (keys and values) × 2 bytes = 81,920 bytes per token, which is 1.34 GB at full context. Grouped-query attention is why that number is small for a 15B model.

BigCode's model card prints its own memory footprints for the Transformers library, which land in the same range: 32,251 MB at bfloat16, 16,900 MB in 8-bit and 9,225 MB in 4-bit.

With no GPU, Ollama loads the model into system RAM, so a machine with 16 GB of RAM can hold the 9.1 GB tag. Completion speed on a processor depends on the machine and has not been measured here. For as-you-type completion on modest hardware, the smaller models below are the usual choice; see best local autocomplete models.

What StarCoder2 is

StarCoder2 is a family of code models from the BigCode project, described in the paper StarCoder 2 and The Stack v2: The Next Generation, submitted on February 29, 2024. The 15B model is the largest of the three.

The point that matters most in use is on the model card: StarCoder2 is “not an instruction model and commands like ‘Write a function that computes the square root.’ do not work well.” You give it the start of some code, or the code around a gap, and it continues. That is the job an editor's autocomplete does.

For instructions, BigCode published a separate fine-tune, StarCoder2-15B-Instruct-v0.1, in April 2024. Its card says it is designed for “coding-related instructions in a single turn” and was tuned mainly on Python tasks. On Ollama it is starcoder2:instruct.

StarCoder2 model sizes: 3B, 7B and 15B

All three share the 16,384-token context. The language count is the practical difference: if you work in a language outside the 17 the smaller models were trained on, the 15B is the one that has seen it. Figures are from the three model cards on Hugging Face.

Ollama tags and install

ollama pull starcoder2 with no tag downloads the 3B model, not the 15B. Name the size you want.

# StarCoder2 15B, q4_0, 9.1 GB (completion model)
ollama pull starcoder2:15b

# StarCoder2 15B Instruct v0.1, q4_0, 9.1 GB (takes instructions)
ollama pull starcoder2:instruct

# Higher-quality quantisation for a 24 GB card, 17 GB
ollama pull starcoder2:15b-q8_0

• Every 15B tag lists a 16K context on Ollama, matching the model card.
• Instruct tags are named 15b-instruct-v0.1- followed by the quantisation, at the same sizes as the base tags.
• No tool calling. The Ollama library page has no tools badge, so agent frameworks that send tool definitions will be refused.

To use it for completion inside an editor, the Continue + Ollama setup guide walks through pointing an autocomplete provider at a local model. Full tag list: ollama.com/library/starcoder2/tags.

StarCoder2 benchmarks, as reported by BigCode

These scores are from the metadata of the two model cards. They are the publisher's figures and this site has not reproduced them. A dash means the card gives no figure.

The base model's HumanEval score looks low beside chat models because HumanEval asks for a function from a description, which is the instruction-style task the base model was not built for. RepoBench, which measures completing code inside a repository, is closer to what it does. Sources: base model card, Instruct model card.

StarCoder2 licence

All three sizes and the Instruct fine-tune are under the BigCode OpenRAIL-M v1 licence agreement. It is not Apache 2.0 or MIT: it is a responsible-use licence that attaches use restrictions to the model. Read the agreement before building a product on the weights.

The model card adds a separate caution about output. The training data was filtered to permissively licensed and unlicensed code, but the model “can generate source code verbatim from the dataset”, and that code's own licence may require attribution. BigCode provides a search index for checking where a generated snippet came from.

Is it still worth running?

This section is opinion. StarCoder2 dates from February 2024, and newer open coding models cover both completion and chat in one download. For a current shortlist see best local autocomplete models and the Ollama coding-model guide; Qwen 2.5 Coder 14B is the closest newer model in this size class.

Two things still set the 15B apart: the breadth of its training languages, and a training set built with opt-out requests excluded and a public index for tracing output. If either matters to you, it remains a reasonable completion model for a 12 GB or 16 GB card. People working almost entirely in Python sometimes compare it with WizardCoder Python 34B, and the DeepSeek Coder V2 16B is a chat-capable model at nearly the same download size.

Frequently asked questions

Sources

• StarCoder2-15B model card: architecture, training data, memory footprints, licence
• StarCoder 2 and The Stack v2 (arXiv:2402.19173)
• bigcode-project/starcoder2 on GitHub
• Ollama tag list for starcoder2
• second-state GGUF files: the file sizes in the hardware table