Aug 5, 2026

Qwen3-Coder 480B Self-Hosting Guide 2026: Which Quant Fits, and the Honest RunPod vs Free API Math

By AIFoss · 9 min read

qwen3-codervllmollamallmselfhosted

TL;DR: Qwen3-Coder 480B-A35B-Instruct is a genuinely Apache 2.0, 480B-parameter MoE coding model that trades blows with frontier cloud agents on agentic benchmarks. Self-hosting it means 150–290GB of memory minimum, so for most people the honest answer is a hosted API or a rented GPU pod, not a home rig.

	Free/cheap API	RunPod pod (rented GPUs)	Local rig
Best for	Trying it, light daily use	Privacy + bursts of heavy agent work	Always-on, air-gapped, no per-hour anxiety
Hardware needed	None	None owned — 4×A100 80GB class, rented	256GB+ unified memory or multi-GPU server
Cost	$0 on free tiers, cents per task after	~$5.50–$6/hr (community tier, 4×A100)	$5,000+ up front (used server) to $10k+ (Mac Studio 512GB)
The catch	Your code leaves your machine	Meter runs while you think; weights re-download on pod start	The 150GB floor rules out every consumer GPU

Honest take: Use the free or metered API unless you have a privacy requirement you can name. If you do, rent a RunPod pod for work sessions before you spend five figures on hardware — and if you just want a strong local coding model on one GPU, run a smaller Qwen3-Coder variant instead.

What Qwen3-Coder 480B actually is

Qwen3-Coder-480B-A35B-Instruct is Alibaba’s flagship open-weights coding model: 480B total parameters in a mixture-of-experts layout that activates 35B per token (8 of 160 experts), with 256K native context extendable to 1M. It was built for agentic coding — long tool-calling loops, repo-scale edits, terminal use — rather than one-shot snippet completion, and on the Aider Polyglot benchmark the BF16 weights score 61.8%, which put it in the same conversation as Claude Sonnet 4 when it landed.

A year after release it’s no longer the newest thing in the Qwen lineup — Qwen3-Coder-Next exists for smaller hardware — but the 480B remains the largest open coding model most people can realistically get access to, and it has aged into mature support across vLLM, llama.cpp, Ollama, and every major agent harness.

License check: it’s real Apache 2.0

This deserves its own section because there’s persistent confusion about it. Several Qwen releases over the years shipped under the custom “Qwen License” with a 100M-monthly-active-user clause and a “Built with Qwen” attribution requirement, and plenty of 2026 commentary still assumes the 480B coder carries those terms.

It doesn’t. The LICENSE file in the official Qwen/Qwen3-Coder-480B-A35B-Instruct HuggingFace repo is plain Apache 2.0, same as the FP8 variant. That means:

Commercial use, modification, and redistribution with no user cap
No attribution branding requirement beyond the standard Apache notice
Fine-tunes and derivatives can be relicensed as your own product allows

For self-hosters this is the clean case: no MAU math, no lawyer call. If you’ve read our LLM license shootout, this sits in the same tier as DeepSeek’s MIT releases — true FOSS weights, unlike Llama 4’s community license.

Which quant fits your hardware

The full BF16 checkpoint is roughly 960GB. Nobody self-hosts that. What you actually deploy is one of Unsloth’s dynamic GGUF quants (or the official FP8 on vLLM), and the size tiers decide everything:

Quant	Size on disk	Memory floor (VRAM + RAM)	Realistic hardware	Quality
UD-IQ1_M	~150GB	~160GB	192GB Mac Studio, 4×48GB GPUs	Usable, noticeably degraded
UD-Q2_K_XL	~180GB	~190GB	256GB unified memory, 2×H100 + RAM offload	Good for its size
Ollama `qwen3-coder:480b`	~290GB	~300GB	512GB Mac Studio, 4×A100 80GB + offload	Solid default
UD-Q4_K_XL	~276GB	~290GB	4×A100/H100 80GB	60.9% Aider Polyglot — near BF16
Official FP8 (vLLM)	~480GB	8×80GB GPUs	Datacenter / rented pod	Full quality, production serving

The headline number in that table is the UD-Q4_K_XL result: 276GB of weights scoring 60.9% on Aider Polyglot against the 960GB original’s 61.8%. Unsloth’s dynamic quantization keeps the layers that matter at higher precision, and at Q4 the quality loss is close to noise. Below Q2 the degradation is real — the 150GB UD-IQ1_M exists so the model runs at all on 192GB machines, not because it’s a good experience for agentic work, where small per-step errors compound across a long tool loop.

Practical floor: you need roughly 250GB of memory — VRAM, unified memory, or VRAM plus system RAM with llama.cpp offloading — for a quant you’d trust with real agent tasks. On Apple Silicon that means the 256GB or 512GB Mac Studio M4 Ultra configurations. On the PC side there is no consumer path: a single RTX 5090 has 32GB, and even a quad-3090 rig at 96GB total doesn’t reach the smallest quant without leaning heavily on system RAM, at which point tokens per second fall off a cliff. Our sister site has the full hardware breakdown at runaihome.com, including what CPU-offload speeds actually look like.

Running it: Ollama and llama.cpp

If you have the memory, the Ollama path is one line:

ollama pull qwen3-coder:480b   # ~290GB download
ollama run qwen3-coder:480b

Two traps, both familiar if you’ve read our other setup guides:

The :cloud tag is not local. ollama run qwen3-coder:480b-cloud works on any laptop because it routes to Ollama’s hosted datacenters — your prompts leave your machine. It’s a legitimate way to try the model, but it is remote inference with local syntax. If your goal is self-hosting, the suffix-free tag is the only one that counts; our Ollama cloud-tags guide covers how to verify with ollama ps (a cloud model shows no local size and cloud in the PROCESSOR column).

Raise the context window. Ollama’s default context silently truncates agent sessions. A 256K-context model behind a 4K window will look broken in Aider or Cline. Set num_ctx to at least 32768 in a Modelfile or via OLLAMA_CONTEXT_LENGTH.

For llama.cpp with a specific Unsloth quant:

llama-server \
  --model Qwen3-Coder-480B-A35B-Instruct-UD-Q4_K_XL-00001-of-00006.gguf \
  --ctx-size 65536 --n-gpu-layers 99 --jinja \
  --host 0.0.0.0 --port 8080

--jinja matters — the model’s tool-calling relies on its chat template, and without it agent harnesses fail in confusing ways. For multi-GPU vLLM serving of the FP8 weights, the flags and tensor-parallel logic are the same as our vLLM multi-GPU guide, scaled up to --tensor-parallel-size 8.

The RunPod math, honestly

Say you don’t own 250GB of memory. Renting is the middle path, and this is where most “self-host a 480B model!” posts get hand-wavy, so here are real numbers.

A 4×A100 80GB pod on RunPod community cloud runs about $1.39 per GPU-hour — roughly $5.60/hour for the pod. Secure cloud (vetted datacenters) is closer to double; an 8×H200 pod for FP8 serving is in the $35/hour range. Billing is per-second, which is what makes this workable:

Evening agent session (3 hrs): ~$17
Two work-days a week (16 hrs/mo… × 4): ~$90/month
24/7 availability: ~$4,000/month — at which point buy hardware or use an API

Three costs the hourly rate hides. First, the weights: 276GB re-downloads on every fresh pod unless you pay for a persistent network volume (~$0.07/GB/month ≈ $20/month for this model — worth it, cold-starts drop from an hour-plus to minutes). Second, idle time: the meter runs while you read the model’s output, and agentic coding is mostly you thinking. Third, availability: community-cloud 4×A100 pods aren’t always instantly available in your region.

The comparison that actually decides it: hosted Qwen3-Coder access via OpenRouter’s free tier or Alibaba’s own API costs $0 for light use, and even metered pricing means a heavy day of agent work costs less than one pod-hour. Renting only wins when the requirement is your code never touches a third-party inference provider — a real requirement for some teams, and RunPod with your own pod is a defensible middle ground: the provider hosts your GPUs, not your model API.

When NOT to self-host this model

You have one GPU. No quant fits. Run Qwen3-Coder-Next or a 32B-class coder locally instead — on a 24GB card they’re a far better experience than a 1-bit 480B.
You wanted it “because it’s the biggest.” At Q1 quantization, a well-run 35B often produces more reliable agent sessions than a starved 480B.
You have no privacy requirement. The free API tier exists. Use it, keep the $90/month.
You need production SLAs. Rent dedicated capacity and serve the FP8 weights with vLLM, or pay a serverless provider per token — hobbyist pods aren’t uptime infrastructure.

FAQ

Can I run Qwen3-Coder 480B on an RTX 4090 or 5090? No — not in any configuration worth using. The smallest dynamic quant is ~150GB and a 5090 has 32GB of VRAM. Aggressive CPU offload will technically load it on a 24–32GB card with 256GB of system RAM, but expect low single-digit tokens per second: unusable for agentic loops. Use a smaller Qwen3-Coder variant on consumer cards.

Is the license really unrestricted Apache 2.0, even commercially? Yes. The official HuggingFace repo’s LICENSE file is standard Apache 2.0, with none of the 100M-MAU or attribution clauses from the older custom Qwen License. You can build a commercial product on it or fine-tune and redistribute.

Which quant should I pick if I have the hardware? UD-Q4_K_XL (~276GB) if you can fit it — third-party Aider Polyglot testing puts it within one point of full BF16. If you’re on a 256GB machine, UD-Q2_K_XL (~180GB) leaves headroom for context. Treat anything below Q2 as a demo, not a daily driver.

Recommended Gear

Mac Studio M4 Ultra — the 256/512GB unified-memory configs are the only desktop-class single boxes that fit the usable quants
RTX 5090 — not for this model, but the right ceiling for the smaller Qwen3-Coder variants this guide points single-GPU users toward

Sources

Was this article helpful?