Aug 14, 2026

DeepSeek V4 Self-Hosting 2026: Flash Locally, Pro on API

By AIFoss · 10 min read

deepseekvllmllamacppggufselfhosted

TL;DR: Both DeepSeek V4 models are plain MIT — no MAU caps, no attribution strings. V4-Flash (284B MoE, 13B active) is genuinely self-hostable: ~175GB VRAM in vLLM, or a 103GB 3-bit GGUF on a 128GB-RAM box with llama.cpp. V4-Pro (1.6T) needs ~800GB at 4-bit — use the API and stop pretending.

	V4-Flash local (GGUF)	V4-Flash vLLM (full)	V4-Pro via API
Best for	Privacy-first home lab, single user	Team serving, 1M context	Everyone else
Memory floor	~110GB RAM (3-bit)	~175GB VRAM (4×A100 / 2×H200)	None
Speed	Single-digit tok/s on CPU+one GPU	Production-grade, batched	Fast, but peak-hour pricing looms
Cost	Hardware you may already own	~$5.60/hr on RunPod	$0.435/M in, $0.87/M out
The catch	Quant quality loss below 3-bit	Nothing consumer-grade fits it	Your prompts leave your network

Honest take: If you can’t name the privacy or compliance requirement forcing you local, use the API — V4-Flash costs $0.14 per million input tokens there. If you can name it, V4-Flash 3-bit on a 128GB-RAM workstation is the cheapest fully-local frontier-class coding model available right now.

What actually shipped, and under what license

The timeline matters because half the guides floating around were written against the preview and are already stale.

DeepSeek released V4 as a preview with open MIT weights on April 24, 2026. On July 24 the API dropped the legacy deepseek-chat/deepseek-reasoner names in favor of deepseek-v4-flash and deepseek-v4-pro (we covered that cutover in the API migration guide). Then on July 31, 2026, DeepSeek open-sourced the production Flash checkpoint — deepseek-ai/DeepSeek-V4-Flash-0731 on Hugging Face — under the plain MIT license, same as V3 before it.

That license check is the part worth being precise about, because 2026 has trained us all to read the fine print. This is not a “Community License” with a 700M-MAU cap like Llama 4, not a revenue-capped Apache derivative like LiquidAI’s LFM Open License, and not non-commercial like MiniMax M3. It’s MIT: commercial self-hosting, modification, redistribution, fine-tuning — all allowed, for both V4-Pro and V4-Flash weights. If your legal team asks, the LICENSE file in the Hugging Face repo is one paragraph. See our license shootout for how rare that’s becoming at the frontier tier.

The 0731 release also isn’t just a re-tag. It went through additional RL, SFT, and distillation passes over the April preview, and it ships with DSpark, a small speculative-decoding draft module attached to the main model (the repo totals ~304B parameters with the drafter included). More on why that matters for tokens-per-second below.

Two model sizes, one honest split:

V4-Flash: 284B total parameters, 13B active per token, MoE, 1M context window, up to 384K output tokens. Vendor-reported 79.0% on SWE-bench Verified for the preview, with the production 0731 build reported higher — treat both numbers as vendor-published until independent runs land.
V4-Pro: 1.6T total, 49B active, 80.6% SWE-bench Verified (vendor-reported). At 4-bit this is roughly 800GB of weights.

Why V4-Pro is API-only for anything smaller than a datacenter

Run the arithmetic once and move on. 1.6T parameters at 4-bit quantization is ~800GB before KV cache. That’s ten H100 80GB cards minimum, and realistically 8×H200 141GB for headroom — a six-figure capex or roughly $30+/hr rented. Meanwhile the API charges $0.435 per million input tokens (and $0.003625 on cache hits) and $0.87 per million out.

At those prices, a self-hosted V4-Pro rig has to push tens of millions of tokens per day, every day, before it beats the API on cost — and that’s before electricity and the weekend you’ll lose to NCCL errors. The MIT license makes V4-Pro self-hosting legal; nothing makes it sensible for a home lab. Our V4-Pro review covers what the model is actually like to use.

The interesting self-hosting story is entirely V4-Flash. Two real paths follow.

Path 1: V4-Flash on vLLM (the serving-grade option)

Full-quality V4-Flash needs about 175GB of VRAM: ~158GB of weights plus ~10GB of KV cache and runtime overhead. In practice that means 4×A100 80GB or 2×H200 — the checkpoint is quantization-aware-trained, storing its routed experts natively in MXFP4 with the rest in FP8/BF16, so “full quality” is already compact for a 284B model.

The launch command, using the DSpark-equipped repo:

vllm serve deepseek-ai/DeepSeek-V4-Flash-DSpark \
  --tensor-parallel-size 4 \
  --trust-remote-code \
  --kv-cache-dtype fp8 \
  --max-model-len 131072 \
  --speculative-config '{"method": "dspark", "num_speculative_tokens": 2}'

The --speculative-config flag is the new part: it activates the bundled DSpark drafter, which proposes tokens the big model verifies in batches — same output, fewer full forward passes. Keep --max-model-len at 128K unless you genuinely need the 1M window; KV cache for a million tokens is its own memory line-item. Flag-by-flag tuning (--gpu-memory-utilization, scheduling policy, the NCCL consumer-board workaround) is in our vLLM multi-GPU guide.

The rental math. Nobody has 4×A100 at home. On RunPod, 4×A100 80GB community-cloud pods run about $5.60/hr as of August 2026 — call it $17 for a three-hour evening session, or ~~$90/month at part-time hobby usage. Add a network volume (~~$20/month) so you’re not re-downloading 160GB of weights every session. That’s a legitimate middle ground: your prompts hit hardware you control for the hours you rent it, without $40k of capex. But be honest with yourself — at 24/7 usage it’s ~$4,000/month, which buys a lot of API tokens.

Path 2: V4-Flash GGUF on llama.cpp (the actually-local option)

This is the path most home labs should care about. Unsloth published GGUF conversions of the 0731 checkpoint, and because the model is natively MXFP4/FP8, the quant ladder is unusually friendly:

Quant	Size on disk	Runs on	Quality
UD-Q8_K_XL (8-bit)	~162GB	192GB RAM / Mac Studio 256GB+	Lossless — bit-checked against official weights
UD-Q4_K_XL (4-bit)	~155GB	~168GB RAM	Near-lossless; only 7GB smaller than Q8
3-bit (UD-Q3_K_XL)	~103GB	~110–128GB RAM	The sweet spot for 128GB machines
1-bit dynamic	smaller still	~96GB RAM	Works; noticeable degradation — last resort

Read that table twice, because it contains a trap: the 4-bit quant saves you almost nothing over 8-bit (155GB vs 162GB) since the experts are already 4-bit-native in the original checkpoint. Either run Q8 if you have ~192GB, or drop straight to 3-bit for a 128GB machine. The 4-bit tier mostly exists for the ~168GB edge case. Background on why dynamic quants behave this way is in our GGUF quantization guide.

A working launch on a 128GB-RAM box with a single 24GB GPU:

llama-server \
  -m DeepSeek-V4-Flash-0731-UD-Q3_K_XL-00001-of-00003.gguf \
  --ctx-size 32768 \
  --temp 1.0 --top-p 1.0 \
  -ngl 99 \
  -ot ".ffn_.*_exps.=CPU" \
  --draft-max 2

Temperature 1.0 and top-p 1.0 are DeepSeek’s official sampling recommendation for V4 — don’t reflexively drop temp to 0.2 like it’s 2024. And note llama.cpp merged MTP/DSpark support for V4-Flash on August 2, 2026; early adopters report 30–50% faster generation with the drafter enabled, with the sweet spot at --draft-max 2-3.

The problem you will actually hit: instant OOM on load

Here’s the failure that generates the GitHub issues: you run llama-server -m <quant> -ngl 99 on your RTX 4090 and it dies allocating memory, because -ngl 99 tries to push all layers — experts included — onto a 24GB card that’s being asked to hold 100GB+ of tensors.

The fix is the -ot ".ffn_.*_exps.=CPU" override in the command above: it pins the MoE expert tensors (the bulk of the weights) in system RAM while attention and shared layers run on the GPU. Since only 13B parameters activate per token, this is far less punishing than CPU-offloading a dense model. If you have VRAM to spare, claw speed back progressively — ".ffn_(up|down)_exps.=CPU" keeps gate experts on-GPU, ".ffn_(up)_exps.=CPU" keeps more still. Expect single-digit tokens per second on DDR5 + one GPU: fine for agentic coding runs you fire and forget, frustrating for interactive chat.

Apple Silicon deserves a mention: a Mac Studio M3 Ultra with 256GB+ unified memory runs the Q8 quant without any offload gymnastics, and the 512GB configuration holds it with room for a serious context window. GPU-buying trade-offs for exactly this workload live at runaihome.com.

Once the server is up, both paths speak OpenAI-compatible HTTP. Point Open WebUI at it (Settings → Connections → add http://<host>:8000/v1 for vLLM or :8080/v1 for llama-server), or wire it into a coding agent — pairing notes for Cline and friends are at aicoderscope.com.

When NOT to self-host V4-Flash

You can’t articulate the requirement. “Privacy” as a vibe isn’t a requirement. The API costs $0.14/M in, $0.28/M out, with cache hits at $0.0028/M — a heavy month of personal coding use is single-digit dollars.
You need interactive speed on consumer hardware. 3-bit on RAM offload is a patient person’s setup. If you want 40+ tok/s chat, run a 24–35B model fully in VRAM instead — our Qwen3.6-35B setup guide covers the best of that class.
You have 64GB of RAM. No current quant fits honestly. Don’t run the 1-bit on swap; you’ll hate it.
You’re chasing benchmarks, not workloads. The vendor’s SWE-bench numbers are impressive but still vendor-published for the 0731 build. Wait for independent runs before re-architecting anything around them.

One more caution flag: DeepSeek has announced peak-hour API pricing (2× during Beijing business hours) that wasn’t yet being billed as of mid-August 2026. If that lever gets pulled hard, the self-hosting math above shifts in local’s favor — recheck it before committing either way.

Verdict

V4-Flash-0731 is the most self-hostable frontier-class model of 2026 so far: genuine MIT, a QAT checkpoint that makes 8-bit lossless at 162GB, a 103GB 3-bit that fits prosumer hardware, and a bundled speculative decoder that both vLLM and llama.cpp already support. Self-host it if you have 128GB+ of RAM and a real reason. Rent 4×A100s on RunPod for the in-between case. And leave V4-Pro on the API, where a 1.6T model belongs.

FAQ

Is DeepSeek V4 really MIT-licensed, including commercial use? Yes. Both the V4-Pro and V4-Flash weight repos on Hugging Face carry the plain MIT license, matching the V3 precedent — no user caps, no attribution requirement, no non-commercial clause. Fine-tuning and redistribution are permitted.

What’s the minimum hardware for running DeepSeek V4-Flash locally? Realistically a 128GB-RAM machine for the ~103GB 3-bit GGUF, with any modern GPU accelerating attention via llama.cpp’s -ot expert offload. A 1-bit quant squeezes into ~96GB RAM with visible quality loss. Full-quality serving needs ~175GB of VRAM (4×A100 80GB or 2×H200).

What is DSpark and should I enable it? DSpark is a small draft model bundled with the 0731 release for speculative decoding: it proposes tokens, V4-Flash verifies them in batches, and output is identical. Enable it — vLLM via --speculative-config, llama.cpp since the August 2, 2026 merge — for a reported 30–50% generation speedup at --draft-max 2-3.

Sources

Recommended Gear

RTX 4090 — 24GB card that carries attention layers while experts sit in system RAM
Mac Studio M3 Ultra — 256GB+ unified memory runs the lossless Q8 quant with no offload tricks

Was this article helpful?