DeepSeek V4: Run a Frontier-Class Model on Your Own Hardware

Quick verdict

DeepSeek V4-Flash is the new default for serious self-hosters. If you have a multi-GPU rig — even a consumer 4× RTX 5090 build — V4-Flash gives you 1M context, MIT licensing, and benchmark scores close to closed frontier models on DeepSeek's published tables, at zero per-token cost.

V4-Pro is for research labs and infrastructure providers. The 1.6T MoE ships as 864.7 GB of weights, so it needs an 8× H200 or 8× B200-class server; most teams get more value from running V4-Flash and reaching for a closed model such as Claude Sonnet 4.6 or GPT-5.5 via API for the hardest problems.

Can an RTX 5090 run DeepSeek V4 Flash?

Not inside its own 32 GB of VRAM, at any quantization. The smallest DeepSeek-V4-Flash-0731 GGUF that exists is unsloth's UD-IQ1_S at 82.5 GB — 50 GB more than the card holds — and the native-precision UD-Q4_K_XL is 155.1 GB. A single 5090 runs V4 Flash only as an offload rig: the 256 routed experts per layer live in system RAM (llama.cpp's --n-cpu-moe) while the card keeps the attention layers, the shared expert and the KV cache. So the real requirement is 64 GB of DDR5 for the smallest build and 96–128 GB for the builds worth running. The 13B “active” figure sets compute per token, not memory: every expert must be resident before the first token, which is why the file size, not the active count, is the number to plan around.

Sizes are the shard sums the Hugging Face API reports for unsloth/DeepSeek-V4-Flash-0731-GGUF and bartowski/DeepSeek-V4-Flash-GGUF; the DDR5 column is the overflow plus about 10 GB for the OS and cache, rounded up to the next common kit. The optional DSpark speculative-decoding head is a separate 10.9 GB Q8_0 file in the unsloth repo and costs about that much extra memory if you enable it. Whether an offload rig is fast enough depends on your PCIe and DDR5 bandwidth, and we have not measured it, so no tokens-per-second figure is given here.

What context length fits next to the 5090?

Once the experts are in RAM, the card's job is the KV cache. Flash's config.json declares one KV head of 512 dimensions per layer across 43 layers, so the cache is at most 43 × 512 = 22,016 values per token — 22 KB at 1 byte per value (an fp8 cache) or 44 KB at f16. That is 0.7 GB at the 32K context unsloth recommends, 2.8 GB at 128K, 8.5 GB at the 384K output length DeepSeek recommends for its max reasoning effort, and 23 GB at the full 1,048,576-token window (46 GB at f16). So a 5090 comfortably holds 32K–384K beside the resident attention layers; the full 1M window is most of the card at fp8 and does not fit at f16. Treat these as ceilings: the config sets sliding_window: 128, so many layers store far less. The hardware requirements page extends this table to Strix Halo, Mac Studio and multi-GPU boxes, and dual-3090 versus single-5090 build math covers the case for spending the budget on system RAM instead of a second card.

Specs at a glance

V4-Pro vs V4-Flash: pick by hardware

Hardware requirements

Both V4 models are mixture-of-experts, so this page gives no tokens-per-second figure: speed depends on how the experts are placed across GPUs and system RAM and on the interconnect, not on file size alone, and none of these configurations has been measured here. The fit column is arithmetic from the Hugging Face file sizes (official release: 864.7 GB for V4-Pro and 159.6 GB for V4-Flash, read 2 November 2026). For multi-user serving, vLLM's continuous batching raises aggregate throughput over single-user runs.

Local setup

Option 1 · Ollama: cloud only, not local

Ollama lists DeepSeek V4, but only as a cloud model: no official tag downloads the weights. As of 2 November 2026 Ollama has retired its deepseek-v4-flash entry (retired 27 August 2026; no tags), and the live official tags are deepseek-v4-pro:cloud and deepseek-v4.1-flash:cloud. Running ollama run deepseek-v4-pro:cloud sends your prompt to Ollama's servers, which is not local inference. A community account, frob, has repackaged the V4-Flash-0731 GGUF builds as Ollama tags (for example frob/deepseek-v4-flash-0731:284b-a13b-ud-iq1_s, 83 GB); those download the weights, but they are not published by Ollama or DeepSeek. If the point is to keep your data on your own machine, Option 2 is the documented route.

Sources: ollama.com/library/deepseek-v4-flash, deepseek-v4-pro and frob/deepseek-v4-flash-0731 (read 2 November 2026). Details and the full size table: DeepSeek V4 hardware requirements.

Option 2 · llama.cpp (the local route)

Best for: when you need exact quantization control or are on Apple Silicon / non-CUDA hardware.

# Build llama.cpp with CUDA support
git clone https://github.com/ggml-org/llama.cpp && cd llama.cpp
make GGML_CUDA=1 -j

# Download a GGUF build (unsloth's recommended UD-IQ3_XXS is 104.2 GB in 4 shards)
huggingface-cli download unsloth/DeepSeek-V4-Flash-0731-GGUF \
  --include "UD-IQ3_XXS/*" \
  --local-dir models

# Serve OpenAI-compatible API on :8080
./llama-server \
  -m models/UD-IQ3_XXS/DeepSeek-V4-Flash-0731-UD-IQ3_XXS-00001-of-00004.gguf \
  -ngl 99 \        # attention layers on the GPU
  --n-cpu-moe 43 \ # keep the routed experts in system RAM on a single-GPU box; lower it as VRAM allows
  -c 32768 \       # context (unsloth's recommended default)
  --host 0.0.0.0 --port 8080

Option 3 · vLLM (production / multi-user)

Best for: serving many users. Continuous batching raises aggregate throughput well above single-user llama.cpp or Ollama runs. The official V4-Flash weights are 159.6 GB, so give vLLM four 80 GB cards: two hold the weights but leave almost nothing for the KV cache.

pip install vllm

# Serve V4-Flash on 4× H100 with tensor parallelism
python -m vllm.entrypoints.openai.api_server \
  --model deepseek-ai/DeepSeek-V4-Flash \
  --tensor-parallel-size 4 \
  --max-model-len 1000000 \
  --gpu-memory-utilization 0.92 \
  --port 8000

# OpenAI-compatible: use any OpenAI client with base_url=http://...:8000/v1

For full production deployment patterns including multi-GPU sharding, KV-cache management, and high-throughput serving, see our Local AI Deployment course — full GitHub repo included.

Benchmarks

Source: every number is from the DeepSeek-V4 model card on Hugging Face (read 2 November 2026): the “DeepSeek-V4-Pro-Max vs Frontier Models” table for the closed-model columns and the “Comparison across Modes” table for V4-Flash. These are DeepSeek's own reported scores at maximum reasoning effort, not independent tests; “—” means the card gives no score. The V4-Flash column is the April release; the official V4-Flash-0731 card reports only agentic benchmarks (for example Terminal Bench 2.1: 82.7 for 0731 against 61.8 for the April Flash and 72.1 for the V4-Pro preview). DeepSeek's card does not compare against Claude Sonnet 4.6 or GPT-5.5.

DeepSeek V4 cloud API pricing

Everything above is about self-hosting — but V4 is the rare frontier model you can also rent from the vendor cheaply. DeepSeek's pricing page, read 2 November 2026, lists two tiers, both with a 1M context and a 384K maximum output. Prices are per million tokens, shown as peak / off-peak; off-peak is half the peak rate, and peak hours are 01:00–04:00 and 06:00–10:00 UTC on weekdays:

Two things changed since this page first listed prices in June 2026 ($0.14 / $0.28 for V4-Flash and $0.435 / $0.87 for V4-Pro): DeepSeek introduced peak and off-peak rates, and it retired the original V4-Flash from the API. The pricing page says the legacy deepseek-v4-flash model name is still accepted but is now answered by DeepSeek-V4.1-Flash-1017 at the Flash price, so the cheap tier is no longer the same model as the V4-Flash weights on Hugging Face. Context caching is automatic, and cache hits cost 1/50th of the cache-miss input rate on the Flash tier ($0.006 vs $0.30) and 1/30th on V4-Pro ($0.044 vs $1.32), so repeated-prefix workloads (RAG, long system prompts, agentic loops) get cheaper still. Third-party hosts set their own rates; check their pages directly.

Self-host vs cheap API: which is actually cheaper?

Because V4 is both downloadable and rentable for pennies, the usual “self-host to escape API bills” math is different here than it is for closed models. The API is so cheap that, for most individuals and small teams, renting V4 beats buying GPUs on pure cost — self-hosting wins on privacy, control, and offline operation, not necessarily on dollars.

Rough break-even: at $1.20 per million output tokens, $10,000 buys about 8.3 billion Flash-tier output tokens at peak (about 16.7 billion off-peak), so a $10K V4-Flash rig only beats the API once you're producing billions of tokens before the hardware is obsolete — so for most people the API is the cheaperand easier option, and self-hosting earns its keep through privacy, latency control, and no third-party dependency rather than raw price. The same workload on GPT-5.5 ($30 output) or Claude Opus 4.7 ($25 output; see which Claude model to use for coding) costs roughly 6–8× more per output token than V4-Pro at peak ($3.96) and 13–15× more than V4-Pro off-peak ($1.98), which is why V4 is a common “cheap frontier” API even for teams that never plan to self-host.

When to pick DeepSeek V4

• ✓You need frontier-class capability without sending data to a third party.
• ✓You want predictable monthly costs (one-time hardware vs per-token API).
• ✓You need 1M-token context and don't want to pay Gemini 3.1 Pro's API rate.
• ✓You're building a product where the model is core IP and MIT licensing matters.

When to use a closed model instead

• →You need the absolute best coding quality → GPT-5.5 or a current Claude Opus model.
• →You want frontier-class quality at the lowest price → Claude Sonnet 4.6 ($3/$15 per Mtok, 1M context — best value of the closed flagships).
• →Hard math / reasoning where every percent counts → GPT-5.5.
• →You don't have multi-GPU hardware and don't want to manage infra → use any closed-API model.

Frequently asked questions