TL;DR: Tencent’s Hy3 is a 295B-total / 21B-active MoE under a clean Apache 2.0 license — the largest permissively licensed model you can realistically run on a single 96–128GB box. The catch is memory, not compute: even the 1-bit GGUF needs ~86GB, and the good quants want 128GB+. If you have 24GB of VRAM, this model is not for you — stay on Qwen3.6-35B-A3B.

	Hy3 (295B-A21B)	Qwen3.6-35B-A3B	DeepSeek V4-Flash (284B-A13B)
Best for	96–192GB unified/multi-GPU rigs that want frontier-adjacent depth	Single 24GB GPU, the practical champion	128GB+ rigs that want the strongest coding scores
License	Apache 2.0, no strings	Apache 2.0	MIT
Smallest usable local footprint	~86GB (IQ1_M, degraded reasoning) / ~107GB resident (IQ2_M)	~22GB (Q4_K_M)	~96GB RAM (1-bit) / ~103GB (3-bit)
The catch	Nothing fits in 24GB; 1-bit reasoning mode loops	Less expert depth than the big MoEs	Q4 needs ~155GB+, so same memory class as Hy3

Honest take: If you already own a 128GB Strix Halo box, a 96GB RTX PRO 6000, or a big-RAM server, Hy3 IQ2_M is one of the best Apache 2.0 models you can run entirely offline today. Everyone else should rent the hours or run Qwen3.6 and stop scrolling.

Tencent released Hy3 on July 6, 2026 (HuggingFace tencent/Hy3, GitHub Tencent-Hunyuan/Hy3) as a full open-weight release under plain Apache 2.0, superseding the more limited Hy3 Preview from late April. Mainline llama.cpp merged the hy_v3 architecture with MTP speculative decoding in mid-July (PR #25395), which turned Hy3 from a datacenter curiosity into something a serious home lab can actually load. Below: the license check, the real quant-by-quant memory math, the llama.cpp setup including the two errors everyone hits, and where the line sits between “self-host it” and “rent it.”

License check: Apache 2.0, and this time it’s clean

Verified against the Tencent-Hunyuan/Hy3 repo in September 2026: Hy3 is released under the Apache License 2.0 with no geographic restrictions, no MAU cap, no revenue clause, and no attribution requirement. That matters because the recent big-MoE releases have been a licensing minefield — Kimi K3 is MIT-derived with a revenue clause, MiniMax M3 is non-commercial, Llama 4 carries the 700M MAU cap. Hy3 sits with Qwen and Gemma 4 in the genuinely-free column, at a much larger scale.

If you want the full license taxonomy, our open-source LLM license shootout covers how these clauses bite in practice.

What 295B-A21B actually means for your hardware

Hy3’s architecture, from the official model card: 80 transformer layers plus one multi-token-prediction (MTP) layer, 64 attention heads (GQA, 8 KV heads), 192 routed experts with top-8 routing plus one always-active shared expert, 120,832-token vocabulary, and a 256K context window. 295B parameters total, ~21B active per token, with the MTP layer adding ~3.8B on top.

The MoE math cuts two ways, and most coverage only mentions the nice half:

• Compute scales with active parameters. Per token, Hy3 does roughly the work of a 21B dense model. That’s why a machine with enough memory gets double-digit tok/s out of a 295B model on consumer silicon.
• Memory scales with total parameters. All 295B must sit somewhere the inference engine can reach — VRAM, unified memory, or system RAM. Full-precision weights are 598GB; Tencent’s own FP8 build is 300GB. There is no router trick that lets a 24GB card hold this model.

So the entire self-hosting question for Hy3 is: how much memory do you have, and which quant fits?

GGUF quant table: what fits where

Sizes below are from the community GGUF repos that appeared within days of release (satgeze, AngelSlim, and others on HuggingFace) plus reported resident footprints — file size on disk and what the process actually occupies diverge once you add KV cache and the MTP layer:

Quant	Disk size	Realistic memory floor	Fits on	Quality notes
IQ1_M (1-bit)	~86GB (~92GB with MTP)	~96GB	Single 96GB card, 128GB unified	Chat and short answers hold up; reasoning mode collapses into loops
IQ2_M (2-bit)	~90–100GB	~107GB resident	128GB unified (Strix Halo, Mac), 4×3090	The sweet spot: coherent reasoning, fits 128GB-class boxes
Q4_K_M (4-bit)	~170GB	~192GB	192GB+ unified, 2×96GB cards, big-RAM server	Near-full quality; community needle tests pass at extreme context
FP8 (official)	300GB	8-GPU server	Datacenter	Tencent’s recommended serving build
BF16	598GB	8×H20-3e class	Datacenter	Full precision, vLLM/SGLang territory

Two honest flags on that table. First, the 1-bit quality warning is real: community testing found IQ1_M fine for chat but prone to circular loops the moment you trigger reasoning mode, even on trivial prompts. A 295B model at 1-bit is not a free lunch. Second, these are community quants — Tencent’s official quantized releases (via their AngelSlim tooling) cover 1-bit and 4-bit, and the numbers land in the same ranges, but exact file sizes shift between uploads, so check the model card before committing a download that large. Our GGUF quantization guide explains what each quant level trades away.

Storage note: even the small quants are ~100GB downloads. A Samsung 990 Pro 4TB or similar NVMe drive stops you from re-downloading 170GB every time you swap quants, and model load time off NVMe is minutes instead of tens of minutes off spinning rust.

llama.cpp setup, and the error everyone hits

The problem you will actually encounter: any llama.cpp build from before mid-July 2026 fails to load Hy3 with unknown model architecture: 'hy_v3'. This isn’t a corrupt download — the architecture (plus its MTP support) was merged into mainline in PR #25395 on July 14, 2026. The fix is simply a current build:

git clone https://github.com/ggml-org/llama.cpp
cd llama.cpp
cmake -B build -DGGML_CUDA=ON   # or -DGGML_METAL=ON / Vulkan for Strix Halo
cmake --build build --config Release -j

Then serve the model. This is the community-tested command shape for an IQ2_M on a 128GB-class machine:

./build/bin/llama-server \
  -m Hy3-IQ2_M.gguf \
  -ngl 99 -fa on -c 32768 --jinja \
  --spec-type draft-mtp --spec-draft-n-max 2 --spec-draft-p-min 0.75 \
  --port 8080

Expected output ends with main: server is listening on http://127.0.0.1:8080, and you get an OpenAI-compatible endpoint at /v1 that Open WebUI or any client can point at. --jinja matters — Hy3’s chat template needs it for tool calling to work, same trap as Qwen3-Coder.

MTP speculative decoding: the free 27%

Hy3 ships its own draft layer, which llama.cpp exposes as --spec-type draft-mtp. Community benchmarks on IQ2_M measured 18.0 → 22.8 tok/s with MTP enabled — a +27% speedup at ~90% draft acceptance, essentially free because the draft layer was trained with the model. On a 128GB Strix Halo mini PC, the 1-bit build reportedly reaches ~24.3 tok/s with MTP.

The catch, verified the hard way by Strix Halo owners: MTP costs memory too (~4GB extra for the layer plus its cache). On a 128GB box running IQ2_M at ~107GB resident, the MTP layer may not fit alongside a useful context size — one published Strix Halo test ran IQ2_M at a stable 10.2–10.5 tok/s without MTP for exactly this reason. Rule of thumb: 1-bit + MTP or 2-bit without, pick one at 128GB. At 192GB+ you stop choosing.

The vLLM path: full quality, rented or racked

If you want Hy3 at FP8/BF16 quality, the official path is vLLM or SGLang across 8 GPUs:

vllm serve tencent/Hy3 --tensor-parallel-size 8 --speculative-config.method mtp

Tencent recommends H20-3e-class GPUs or anything with more memory for the full model. Nobody’s home lab does this, which is where renting comes in: a multi-A100/H100 pod on RunPod lets you benchmark full-quality Hy3 against your actual workload for a few dollars an hour before you decide whether the local quant is good enough — the same evaluate-before-you-buy math we ran in the Qwen3-Coder 480B guide.

If you only need API access, hosted endpoints exist — but if you’re reading this site you probably care that prompts stay on your hardware, and that’s the whole point of the Apache 2.0 + GGUF combination.

Hardware tiers: the short version

• 24GB VRAM (RTX 4090 or RTX 3090): Hy3 does not fit at any quant. Run Qwen3.6-35B-A3B (~22GB Q4_K_M) and lose surprisingly little — our August leaderboard has the 24GB rankings.
• 96GB (RTX PRO 6000, 4×3090): IQ1_M fits with headroom for context; expect degraded reasoning. 4×3090 can stretch to IQ2_M split across cards.
• 128GB unified (Strix Halo, Mac M-series): IQ2_M without MTP (~10 tok/s) or 1-bit with MTP (~24 tok/s). This is the tier Hy3 was seemingly made for — the sister-site Hy3 hardware guide on runaihome.com covers the actual boxes.
• 192GB+ (Mac Studio, dual-socket server): Q4_K_M with MTP. Near-full quality, fully local.

When NOT to self-host Hy3

• You have one consumer GPU. No quant fits in 24GB or even 48GB. This is not a “someday with a better quant” situation — 295B total parameters is a memory floor, not a suggestion.
• You need reasoning on a 96GB budget. The only quant that fits degrades exactly the capability (deep reasoning) that justifies a 295B model. At that budget, DeepSeek V4-Flash’s 3-bit (~103GB, borderline) or a smaller model served properly is the better spend — see the DeepSeek V4 self-hosting guide.
• You want Ollama one-command convenience. As of early September 2026, the supported local path is llama.cpp directly (or vLLM for full weights). Check ollama pull availability before assuming; the arch is new enough that wrapper-tool support lags mainline.
• Interactive latency matters more than depth. 10–24 tok/s is fine for chat and batch work, sluggish for agent loops that burn thousands of tokens per step.

FAQ

Is Hy3 really Apache 2.0 for commercial use? Yes. The July 6, 2026 release is plain Apache 2.0 per the GitHub repo — no MAU threshold, no revenue clause, no geographic restriction anywhere in the license text. As always, the model weights’ license says nothing about what your fine-tuning data allows.

How fast is Hy3 on a 128GB machine? Community-published numbers: ~10.2–10.5 tok/s on IQ2_M without MTP on a Strix Halo 128GB box, ~24.3 tok/s on the 1-bit build with MTP speculative decoding enabled. Apple Silicon lands in a similar band, scaled by memory bandwidth. Treat all of these as single-user llama.cpp numbers, not batch-serving throughput.

Why does my llama.cpp say unknown model architecture: 'hy_v3'? Your build predates July 14, 2026. The hy_v3 architecture and its MTP speculative decoding merged in PR #25395; pull mainline and rebuild. If you’re on a distro package or a wrapper app, it may still ship an older build — compile from source until the ecosystem catches up.

Sources

• Tencent-Hunyuan/Hy3 — official GitHub repo (license, architecture, vLLM/SGLang commands)
• tencent/Hy3 on HuggingFace (model card, official weights)
• MarkTechPost: Tencent Releases Hy3 (release coverage, July 6 2026)
• satgeze/Hy3-1M-GGUF (community GGUF quants, MTP benchmark numbers)
• Sleeping Robots: Hy3 IQ2_M on Strix Halo (128GB memory-fit testing)

Recommended Gear

• RTX 4090 — 24GB flagship; ironically not enough for Hy3, ideal for the Qwen3.6 tier
• Samsung 990 Pro 4TB — NVMe headroom for 100–170GB model files