DeepSeek V4: Run a Frontier-Class Model on Your Own Hardware

Quick verdict

DeepSeek V4-Flash is the new default for serious self-hosters. If you have a multi-GPU rig — even a consumer 4× RTX 5090 build — V4-Flash gives you 1M context, MIT licensing, and within ~10% of frontier closed-model quality at zero per-token cost.

V4-Pro is for research labs and infrastructure providers. The 1.6T MoE needs 8× H100 minimum; most teams get more value from running V4-Flash and reaching for Claude Sonnet 4.6 / GPT-5.5 via API for the hardest 5-10% of problems.

Can an RTX 5090 run DeepSeek V4 Flash?

Not inside its own 32 GB of VRAM, at any quantization. The smallest DeepSeek-V4-Flash-0731 GGUF that exists is unsloth's UD-IQ1_S at 82.5 GB — 50 GB more than the card holds — and the native-precision UD-Q4_K_XL is 155.1 GB. A single 5090 runs V4 Flash only as an offload rig: the 256 routed experts per layer live in system RAM (llama.cpp's --n-cpu-moe) while the card keeps the attention layers, the shared expert and the KV cache. So the real requirement is 64 GB of DDR5 for the smallest build and 96–128 GB for the builds worth running. The 13B “active” figure sets compute per token, not memory: every expert must be resident before the first token, which is why the file size, not the active count, is the number to plan around.

Sizes are the shard sums the Hugging Face API reports for unsloth/DeepSeek-V4-Flash-0731-GGUF and bartowski/DeepSeek-V4-Flash-GGUF; the DDR5 column is the overflow plus about 10 GB for the OS and cache, rounded up to the next common kit. The optional DSpark speculative-decoding head is a separate 10.9 GB Q8_0 file in the unsloth repo and costs about that much extra memory if you enable it. Whether an offload rig is fast enough depends on your PCIe and DDR5 bandwidth, and we have not measured it, so no tokens-per-second figure is given here.

What context length fits next to the 5090?

Once the experts are in RAM, the card's job is the KV cache. Flash's config.json declares one KV head of 512 dimensions per layer across 43 layers, so the cache is at most 43 × 512 = 22,016 values per token — 22 KB at 1 byte per value (an fp8 cache) or 44 KB at f16. That is 0.7 GB at the 32K context unsloth recommends, 2.8 GB at 128K, 8.5 GB at the 384K output length DeepSeek recommends for its max reasoning effort, and 23 GB at the full 1,048,576-token window (46 GB at f16). So a 5090 comfortably holds 32K–384K beside the resident attention layers; the full 1M window is most of the card at fp8 and does not fit at f16. Treat these as ceilings: the config sets sliding_window: 128, so many layers store far less. The hardware requirements page extends this table to Strix Halo, Mac Studio and multi-GPU boxes, and dual-3090 versus single-5090 build math covers the case for spending the budget on system RAM instead of a second card.

Specs at a glance

V4-Pro vs V4-Flash: pick by hardware

Hardware requirements

Tokens/sec assumes single-user inference at 8K context. With KV-cache offloading, throughput drops 30-50% once you exceed VRAM. For multi-user serving, vLLM's continuous batching boosts aggregate throughput 5-15× depending on workload.

Local setup

Option 1 · Ollama: cloud only, not local

Ollama lists DeepSeek V4-Flash, but only as a cloud model: there is no tag that downloads the weights. Running ollama run deepseek-v4-flash:cloud sends your prompt to Ollama's servers, which is not local inference. If the point is to keep your data on your own machine, use Option 2.

Source: ollama.com/library/deepseek-v4-flash. Details and the full size table: DeepSeek V4 hardware requirements.

Option 2 · llama.cpp (the local route)

Best for: when you need exact quantization control or are on Apple Silicon / non-CUDA hardware.

# Build llama.cpp with CUDA support
git clone https://github.com/ggml-org/llama.cpp && cd llama.cpp
make GGML_CUDA=1 -j

# Download a GGUF build (unsloth's recommended UD-IQ3_XXS is 104.2 GB in 4 shards)
huggingface-cli download unsloth/DeepSeek-V4-Flash-0731-GGUF \
  --include "UD-IQ3_XXS/*" \
  --local-dir models

# Serve OpenAI-compatible API on :8080
./llama-server \
  -m models/UD-IQ3_XXS/DeepSeek-V4-Flash-0731-UD-IQ3_XXS-00001-of-00004.gguf \
  -ngl 99 \        # attention layers on the GPU
  --n-cpu-moe 43 \ # keep the routed experts in system RAM on a single-GPU box; lower it as VRAM allows
  -c 32768 \       # context (unsloth's recommended default)
  --host 0.0.0.0 --port 8080

Option 3 · vLLM (production / multi-user)

Best for: serving many users. Continuous batching gives 5-15× aggregate throughput vs Ollama/llama.cpp.

pip install vllm

# Serve V4-Flash on 2× H100 with tensor parallelism
python -m vllm.entrypoints.openai.api_server \
  --model deepseek-ai/DeepSeek-V4-Flash \
  --tensor-parallel-size 2 \
  --max-model-len 1000000 \
  --gpu-memory-utilization 0.92 \
  --port 8000

# OpenAI-compatible: use any OpenAI client with base_url=http://...:8000/v1

For full production deployment patterns including multi-GPU sharding, KV-cache management, and high-throughput serving, see our Local AI Deployment course — full GitHub repo included.

Benchmarks

Sources: DeepSeek V4 technical report (Apr 2026), SWE-Bench Verified leaderboard, Artificial Analysis, Aider public benchmarks. · DeepSeek-V4-Pro on Hugging Face

DeepSeek V4 cloud API pricing

Everything above is about self-hosting — but V4 is the rare frontier model you can also rent from the vendor for almost nothing. As of June 2026, DeepSeek's official API prices V4 in two tiers, both with the full 1M context included at no surcharge:

V4-Pro's $0.435 / $0.87 is the 75%-off launch price DeepSeek made permanent on May 22, 2026 (original list was $1.74 / $3.48). Context caching is automatic — cache hits drop input to roughly 1/50th of the cache-miss rate, so repeated-prefix workloads (RAG, long system prompts, agentic loops) get even cheaper. Third-party hosts vary: OpenRouter, for example, has listed V4-Flash around $0.09 / $0.18, but those are reseller rates, not DeepSeek's own.

Self-host vs cheap API: which is actually cheaper?

Because V4 is both downloadable and rentable for pennies, the usual “self-host to escape API bills” math is different here than it is for closed models. The API is so cheap that, for most individuals and small teams, renting V4 beats buying GPUs on pure cost — self-hosting wins on privacy, control, and offline operation, not necessarily on dollars.

Rough break-even: a $10K V4-Flash rig only beats the V4-Flash API once you're burning well over a billion output tokens before the hardware is obsolete — so for most people the API is the cheaperand easier option, and self-hosting earns its keep through privacy, latency control, and no third-party dependency rather than raw price. The same workload on GPT-5.5 ($30 output) or Claude Opus 4.7 ($25 output) costs roughly 28-34× more per output token than V4-Pro — which is exactly why V4 is the default “cheap frontier” API even for teams that never plan to self-host.

When to pick DeepSeek V4

• ✓You need frontier-class capability without sending data to a third party.
• ✓You want predictable monthly costs (one-time hardware vs per-token API).
• ✓You need 1M-token context and don't want to pay Gemini 3.1 Pro's API rate.
• ✓You're building a product where the model is core IP and MIT licensing matters.

When to use a closed model instead

• →You need the absolute best coding quality → GPT-5.5 (top SWE-Bench score).
• →You want frontier-class quality at the lowest price → Claude Sonnet 4.6 ($3/$15 per Mtok, 1M context — best value of the closed flagships).
• →Hard math / reasoning where every percent counts → GPT-5.5.
• →You don't have multi-GPU hardware and don't want to manage infra → use any closed-API model.

Frequently asked questions