DeepSeek Coder V2 236B
Hardware Requirements, Cost and Local Setup

DeepSeek Coder V2 hardware requirements

DeepSeek-Coder-V2-Lite (16B total, 2.4B active) runs on a single 24 GB card at Q8_0 with its full 128K context, and on a 12 GB card at Q4_K_M with a short context. The full DeepSeek-Coder-V2 (236B total, 21B active) does not fit any single consumer GPU: the smallest published GGUF is 52.7 GB, the standard Q4_K_M is 142.5 GB, and DeepSeek's own model card says BF16 inference needs “80GB*8 GPUs”. Both models are Mixture-of-Experts, so the active-parameter count sets speed, not memory — every expert has to be resident in VRAM or system RAM before the first token. The MoE VRAM calculator does the same arithmetic for any expert count.

File sizes are the shard sums reported by the Hugging Face API for bartowski/DeepSeek-Coder-V2-Lite-Instruct-GGUF and bartowski/DeepSeek-Coder-V2-Instruct-GGUF; the fp16 and larger 236B rows come from the sizes on Ollama's tag list. KV cache is arithmetic from each config.json: Coder V2 uses multi-head latent attention, so llama.cpp caches (512 + 64) values per token per layer at 2 bytes — 27 layers for Lite = 31 KB per token (0.5 GB at 16K context, 4.1 GB at 128K), 60 layers for the 236B = 69 KB per token (1.1 GB at 16K, 9.1 GB at 128K). “Memory needed” below is file + 16K KV cache + a 1 GB compute-buffer allowance.

Lite 16B: what fits a single card

236B: 48–96 GB of VRAM, or system RAM offload

The RAM column is file size + 1 GB buffer + the 1.1 GB 16K KV cache, rounded up to the next common DDR5 kit with room for the OS; offloading keeps the attention layers and cache on the card and streams the routed experts from RAM over PCIe, so it works but is bandwidth-bound. Ollama's 236b tag is q4_0 (133 GB), not Q4_K_M. For a coding model that stays on one 24 GB card at full precision, compare Qwen3-Coder-Next and the best models for a 24 GB GPU; for the newer DeepSeek generation see DeepSeek V4 and its hardware requirements by memory tier. How the K-quant names trade size for quality is covered in quantization explained.

Model specifications

Figures below are from the DeepSeek-Coder-V2-Instruct model card and the paper DeepSeek-Coder-V2: Breaking the Barrier of Closed-Source Models in Code Intelligence (DeepSeek-AI, submitted 17 June 2024). The model is a Mixture-of-Experts continuation of a DeepSeek-V2 checkpoint, further trained on 6 trillion tokens; the paper reports the language count growing from 86 to 338 and the context from 16K to 128K.

Reported benchmarks

These are DeepSeek's own numbers, copied from the table in the DeepSeek-Coder-V2 README for the two Instruct models. This site has not run these benchmarks; treat them as vendor-reported. The SWE-Bench column is the one to look at if you want an agentic coder rather than a completion model — the Lite model scores 0.0 there.

Installation

The quickest route is Ollama. deepseek-coder-v2 with no tag is the Lite Instruct model at q4_0 (8.9 GB), which is what most people should start with. The tags below are the ones that matter; the full list with every quantization is on ollama.com/library/deepseek-coder-v2/tags.

Lite 16B with Ollama (one GPU)

# Lite Instruct at q4_0 (8.9 GB) — fits a 12 GB card at short context
ollama pull deepseek-coder-v2:16b
ollama run deepseek-coder-v2:16b

# Lite Instruct at Q8_0 (17 GB) — the 24 GB-card pick, room for the full 128K context
ollama pull deepseek-coder-v2:16b-lite-instruct-q8_0

Full 236B with Ollama (multi-GPU or RAM offload)

# q4_0, 133 GB on disk — see the 236B table above for what memory this needs
ollama pull deepseek-coder-v2:236b

# q2_K, 86 GB — the smallest 236B tag Ollama publishes
ollama pull deepseek-coder-v2:236b-instruct-q2_K

llama.cpp with a specific GGUF

If you want a quantization Ollama does not carry (the IQ1_M and IQ2_XS 236B files, or IQ4_XS for Lite), llama.cpp's quick start loads a Hugging Face repo directly with -hf; the GGUF repos are the two bartowski links in the hardware section.

# Lite Instruct GGUFs (pick the quant file on the repo page)
llama serve -hf bartowski/DeepSeek-Coder-V2-Lite-Instruct-GGUF

# 236B Instruct GGUFs — the IQ1_M (52.7 GB) and IQ2_XS (68.7 GB) files live here
llama serve -hf bartowski/DeepSeek-Coder-V2-Instruct-GGUF

The -hf syntax is from the llama.cpp README quick start. Which coding model to pull for each VRAM tier is covered in the Ollama coding-model guide.

Frequently asked questions

Sources and further reading