DeepSeek V4: Run a Frontier-Class Model on Your Own Hardware

Quick verdict

DeepSeek V4-Flash is the new default for serious self-hosters. If you have a multi-GPU rig — even a consumer 4× RTX 5090 build — V4-Flash gives you 1M context, MIT licensing, and within ~10% of frontier closed-model quality at zero per-token cost.

V4-Pro is for research labs and infrastructure providers. The 1.6T MoE needs 8× H100 minimum; most teams get more value from running V4-Flash and reaching for Claude Sonnet 5 / GPT-5.5 via API for the hardest 5-10% of problems.

Specs at a glance

V4-Pro vs V4-Flash: pick by hardware

Hardware requirements

Tokens/sec assumes single-user inference at 8K context. With KV-cache offloading, throughput drops 30-50% once you exceed VRAM. For multi-user serving, vLLM's continuous batching boosts aggregate throughput 5-15× depending on workload.

Local setup

Option 1 · Ollama (easiest)

Best for: getting V4-Flash running in 10 minutes. Uses GGUF backend (llama.cpp under the hood).

# Install Ollama (skip if you have it)
curl -fsSL https://ollama.com/install.sh | sh

# Pull the Q4 quantization (~150 GB)
ollama pull deepseek-v4-flash:13b-q4

# Run it
ollama run deepseek-v4-flash

# Or expose an API endpoint
ollama serve
# curl http://localhost:11434/api/generate -d '{"model":"deepseek-v4-flash","prompt":"hello"}'

Option 2 · llama.cpp (max performance, manual)

Best for: when you need exact quantization control or are on Apple Silicon / non-CUDA hardware.

# Build llama.cpp with CUDA support
git clone https://github.com/ggml-org/llama.cpp && cd llama.cpp
make GGML_CUDA=1 -j

# Download GGUF (precomputed by community on HF)
huggingface-cli download bartowski/DeepSeek-V4-Flash-GGUF \
  DeepSeek-V4-Flash-Q4_K_M.gguf \
  --local-dir models

# Serve OpenAI-compatible API on :8080
./llama-server \
  -m models/DeepSeek-V4-Flash-Q4_K_M.gguf \
  -ngl 99 \        # offload all layers to GPU
  -c 32768 \       # context (set lower if VRAM-limited)
  --host 0.0.0.0 --port 8080

Option 3 · vLLM (production / multi-user)

Best for: serving many users. Continuous batching gives 5-15× aggregate throughput vs Ollama/llama.cpp.

pip install vllm

# Serve V4-Flash on 2× H100 with tensor parallelism
python -m vllm.entrypoints.openai.api_server \
  --model deepseek-ai/DeepSeek-V4-Flash \
  --tensor-parallel-size 2 \
  --max-model-len 1000000 \
  --gpu-memory-utilization 0.92 \
  --port 8000

# OpenAI-compatible: use any OpenAI client with base_url=http://...:8000/v1

For full production deployment patterns including multi-GPU sharding, KV-cache management, and high-throughput serving, see our Local AI Deployment course — full GitHub repo included.

Benchmarks

Sources: DeepSeek V4 technical report (Apr 2026), SWE-Bench Verified leaderboard, Artificial Analysis, Aider public benchmarks.

When to pick DeepSeek V4

• ✓You need frontier-class capability without sending data to a third party.
• ✓You want predictable monthly costs (one-time hardware vs per-token API).
• ✓You need 1M-token context and don't want to pay Gemini 3.1 Pro's API rate.
• ✓You're building a product where the model is core IP and MIT licensing matters.

When to use a closed model instead

• →You need the absolute best coding quality → Claude Sonnet 5.
• →Hard math / reasoning where every percent counts → GPT-5.5.
• →You don't have multi-GPU hardware and don't want to manage infra → use any closed-API model.

Frequently asked questions