TL;DR: Inkling, released July 15, 2026 by Thinking Machines Lab, is a 975B-parameter multimodal MoE under a clean Apache 2.0 license — no MAU caps, no attribution clauses. Self-hosting the flagship requires roughly 290GB of combined RAM+VRAM at 1-bit, and about 2TB of GPU memory at native precision. For almost everyone, the real self-hosting story is Inkling-Small at 276B.

	Inkling 975B (vLLM)	Inkling 975B (GGUF offload)	Inkling-Small 276B
Best for	Teams with datacenter GPUs	High-RAM workstation owners	Single-GPU / unified-memory home labs
Memory needed	~2TB BF16 (8×B300 / 16×H200); NVFP4 halves it	~290GB RAM+VRAM (UD-IQ1_S, 270GB on disk)	~88GB at 2-bit dynamic
The catch	Hardware costs more than a house	Single-digit-to-low-teens tok/s	2-bit quant trades some quality

Honest take: Unless you already own a 512GB-RAM server or a multi-GPU rig, use Inkling through a hosted API and self-host Inkling-Small instead — it’s the first frontier-class model with a real single-card path.

Thinking Machines Lab spent a year and a half as the most-watched stealth startup in AI. When Mira Murati’s team finally shipped its first production model on July 15, 2026, it did something none of the other US frontier labs have done recently: it published the full weights on Hugging Face under Apache 2.0. Not a community license with a monthly-active-user trigger, not “open weights, restricted use” — the same license Qwen and Gemma 4 use, with commercial use, modification, and redistribution all permitted.

That makes Inkling the largest permissively licensed US-origin model you can download today. This guide covers what it actually takes to run it yourself — and where the honest answer is “don’t.”

What Inkling is

Inkling is a Mixture-of-Experts transformer with 975B total parameters and 41B active per token, trained on 45 trillion tokens of text, images, audio, and video. It accepts text, image, and audio input natively and supports a context window of up to 1M tokens.

The headline numbers from the release: 97.1% on AIME 2026, 77.6% on SWE-bench Verified, 54.3% on SWE-bench Pro (Public), 63.8% on Terminal-Bench 2.1, 88.7% on Global-MMLU-Lite, and 73.3% on MMMU Pro. Those are vendor-reported figures — independent replication was still thin as of mid-August — but they place Inkling in the same conversation as GLM-5.2 and DeepSeek V4 Pro on coding, with a multimodal breadth neither of those has.

The most interesting architectural feature for self-hosters is controllable thinking effort. Instead of a binary think/no-think toggle, Inkling takes a continuous numeric effort level (roughly 0.2 to 0.99) injected through the chat template. The model was trained with a per-token reinforcement-learning cost, so it learns to spend more or fewer chain-of-thought tokens depending on the budget you request. On your own hardware, where every reasoning token costs you wall-clock time, being able to dial effort to 0.3 for easy queries and 0.9 for hard ones matters more than it does on someone else’s API.

The license check

Apache 2.0, verified on the Hugging Face model card and confirmed across release coverage. That means:

• Commercial self-hosting: allowed, no revenue or user thresholds.
• Fine-tuning and redistribution of derivatives: allowed.
• No “built with” attribution requirement, no EU carve-outs.

Compare that to Kimi K2.7’s Modified MIT (restrictions above 100M MAU / $20M monthly revenue) or Llama 4’s Community License (700M MAU cap, attribution, EU multimodal restriction), and Inkling sits in the clean column with GLM-5.2 (MIT), DeepSeek V4 (MIT), and Qwen3.6 (Apache 2.0). Our license shootout has the full map.

The hardware reality

Now the part most coverage glosses over. At native 16-bit precision, Inkling needs about 2TB of GPU memory — around 1.89TB just for the BF16 weights. Thinking Machines’ own guidance is roughly eight NVIDIA B300 accelerators or sixteen H200s. That is a datacenter deployment, full stop.

The official escape hatch is the NVFP4 checkpoint, which quantizes only the routed experts to 4-bit (block size 16, FP8 scale metadata) while keeping shared experts and attention projections in BF16. It’s hardware-aligned to Blackwell FP4 tensor cores and cuts the GPU count roughly in half — call it 8×H200-class instead of 16. Still datacenter territory.

For everyone else, Unsloth published dynamic GGUF quants within days of release:

Quant	Disk size	Practical requirement
BF16	~1.89TB	Multi-node cluster
UD-Q4_K_XL	~587GB	512GB+ RAM server + GPU, or Mac cluster
UD-IQ1_S (1-bit dynamic)	~270GB	~290GB combined RAM+VRAM

Two honest caveats before you get excited about the 1-bit number. First, quality: community testing puts aggressive 1-bit dynamic quants at roughly 74% of full-model accuracy on aggregate intelligence scores — usable, but you are no longer running the model on the leaderboard. Second, speed: because only 41B parameters are active per token, CPU+GPU expert offloading works, but a 512GB DDR5 workstation running llama.cpp with expert-offload flags lands in the single-digit-to-low-teens tokens per second. Fine for batch jobs and agents you leave running overnight; painful for interactive chat.

If you want to feel the difference GPU memory makes, the reference numbers from vLLM’s day-0 post: up to 380 tok/s per user with multi-token prediction on 4×GB200, and about 140 tok/s without MTP. The gap between that and a home offload rig is two orders of magnitude.

Path 1: llama.cpp with expert offload

This is the realistic local path for the 975B model, and it requires serious RAM. A Mac Studio M3 Ultra with 512GB unified memory can hold UD-IQ1_S entirely; on the PC side you want a workstation or server board with 384–512GB of DDR5 plus any 24GB CUDA card to accelerate the shared experts and KV cache.

The flow, per Unsloth’s guide:

1. Pull the GGUF from the unsloth/inkling-GGUF repo on Hugging Face (the 1-bit build is split across multiple files — expect a 270GB download, so do it on a wired connection).
2. Build llama.cpp from a current release. At launch, Inkling support lived in a dedicated branch; check llama-cli --list-arch or the release notes for inkling before assuming your distro package is new enough — an old build fails with an unknown-architecture error, the same trap Hunyuan and GLM users hit at their launches.
3. Run llama-server with MoE expert offload flags (-ot regex patterns pushing routed experts to CPU) and as many GPU layers as your VRAM allows, then point any OpenAI-compatible client — Open WebUI, Continue.dev, opencode — at the local endpoint.

One trap carried over from every big MoE we’ve covered: watch your context allocation. At 1M maximum context, the KV cache alone can eat more memory than you budgeted — start at 32K and grow it deliberately.

What about ik_llama.cpp, the CPU-optimized fork that gets remarkable throughput from old Xeons? Its expert-offload tuning makes it a natural fit for exactly this kind of RAM-heavy MoE deployment, and multi-GPU offload strategies for it are an active discussion topic. But confirm Inkling architecture support in the fork’s release notes before building your plan around it — fork support typically lands after mainline, and we could not verify a merged Inkling implementation at publish time.

Path 2: vLLM for real serving

If you have the GPUs — or rent them — vLLM shipped day-0 support with full feature parity: tensor, data, expert, and pipeline parallelism, LoRA, prefix caching, and disaggregated serving. The vLLM recipes site has the canonical vllm serve thinkingmachines/Inkling invocations per GPU configuration; the NVFP4 checkpoint is the one to serve on Blackwell.

Nobody should buy 16 H200s for a hobby. Renting, though, is worth pricing out: on RunPod, an 8×H100/H200-class pod runs on-demand by the second, which makes “self-hosted Inkling for a privacy-sensitive batch job” a per-evening cost rather than a capital expense. The same math we ran for Qwen3-Coder 480B applies, scaled up: spin up a pod, attach a network volume so you don’t re-download 600GB of weights every session, and shut it down when you’re done. If you don’t have a named privacy or compliance requirement, the hosted APIs (Together, Fireworks, Baseten, and Thinking Machines’ own Tinker platform for fine-tuning) will beat your rental math for interactive use.

The real answer for home labs: Inkling-Small

Thinking Machines followed the flagship with Inkling-Small, a 276B version whose weights are now live, also with Unsloth GGUF support. The 2-bit dynamic quant is 87.9GB — which means a single RTX PRO 6000 96GB card, or a 128GB unified-memory box like a Strix Halo mini-PC or Mac Studio, holds a frontier-class model on one machine. That is the first time a US frontier lab’s current-generation model has had a genuine single-card path.

For most readers, the decision tree is: Inkling-Small locally, flagship Inkling via API or rented pod when a task demands it. Our sister site has the hardware guide for the 975B flagship and a dedicated Inkling-Small home-lab build guide with per-machine speed expectations.

When NOT to self-host Inkling

• You want interactive chat speed on consumer hardware. A 1-bit 975B at 8 tok/s is a worse daily driver than Qwen3.6-35B-A3B at 50+ tok/s on a 24GB card. Size isn’t the only axis of quality-per-dollar.
• You’d be quoting leaderboard numbers from a 1-bit quant. The model that scored 77.6% on SWE-bench Verified is the BF16 one. Budget your expectations accordingly.
• Your privacy requirement is vague. “I’d rather it stayed local” doesn’t justify a $6,000 RAM build or a rental bill. A named compliance constraint does.

FAQ

Is Inkling really Apache 2.0, including commercial use? Yes. The weights on Hugging Face ship under Apache 2.0 with no user caps, revenue triggers, or attribution requirements — verified on the model card at release and unchanged as of late August 2026.

What’s the cheapest way to run the full 975B model locally? Unsloth’s UD-IQ1_S dynamic GGUF: 270GB on disk, roughly 290GB of combined RAM+VRAM, via llama.cpp with expert offload. A 512GB Mac Studio or DDR5 workstation qualifies. Expect single-digit-to-low-teens tokens per second and a measurable quality drop versus full precision.

Should I run Inkling or Inkling-Small? Inkling-Small for anything interactive on a home lab — its 2-bit quant fits in 88GB, so one 96GB card or a 128GB unified-memory machine handles it. Reserve the 975B flagship for rented multi-GPU pods or API calls on tasks where the extra capability provably matters.

Sources

• Thinking Machines Lab Releases Inkling: A 975B-Parameter Open-Weights Multimodal MoE — MarkTechPost
• Welcome Inkling by Thinking Machines — Hugging Face Blog
• Inkling: How to Run Locally — Unsloth Documentation
• TML Inkling on vLLM: Day-0 Support with Optimized Performance — vLLM Blog
• Inkling 975B Open Weights Model Under Apache 2.0 — gHacks

Recommended Gear

• Mac Studio M3 Ultra — 512GB unified memory holds the 1-bit 975B quant in one box
• NVIDIA RTX PRO 6000 — 96GB VRAM, single-card home for Inkling-Small’s 2-bit GGUF