Ornith-1.0 Self-Hosting Guide 2026: Ollama and vLLM

ornithollamavllmcoding-agentselfhostedai

TL;DR: Ornith-1.0 is DeepReinforce’s MIT-licensed coding model family (released June 25, 2026) in four sizes: 9B and 31B dense, 35B MoE, and a 397B MoE flagship. The 9B runs on any 8GB card; the 35B MoE just fits a 24GB GPU at Q4_K_M. Benchmarks are vendor-reported for now, so treat the 397B’s 82.4% SWE-bench Verified claim as promising, not proven.

Ornith-1.0-9BOrnith-1.0-35B MoEOrnith-1.0-397B MoE
Best for8–12GB GPUs, laptop codingSingle 24GB card, best local valueMulti-GPU nodes, cloud rental
VRAM at Q4~6GB21.2GB (tight on 24GB)~225GB+ (FP8)
The catchWeakest of the familyKV cache forces 8–16K context on 24GBNot realistic on home hardware

Honest take: run the 35B MoE if you have a 24GB card — it decodes like a small model because only ~3B parameters are active per token. Everyone else should start with the 9B and keep expectations calibrated until independent benchmarks land.

What Ornith-1.0 actually is

DeepReinforce released Ornith-1.0 on June 25, 2026: a family of coding models trained with a self-scaffolding reinforcement learning loop. Instead of humans hand-writing the RL training harness, the model writes and revises its own scaffold — the test runners, reward checks, and task decomposition it trains against. Whether that mechanism is why the numbers look good is impossible to say from outside, but the release itself is unusually complete: weights for every size, GGUF quants for the two smaller tiers, FP8 for the big MoEs, and day-one vLLM support.

The four variants, per the release notes and Hugging Face model cards:

  • Ornith-1.0-9B — dense, post-trained from Qwen 3.5
  • Ornith-1.0-31B — dense, post-trained from Gemma 4
  • Ornith-1.0-35B — mixture-of-experts, ~3B active parameters per token
  • Ornith-1.0-397B — MoE flagship, shipped in BF16 and FP8

All four share a 256K (262,144-token) context window, which matters more for agentic coding than raw benchmark scores — long agent sessions eat context fast.

Vendor-reported numbers: the 397B posts 82.4% on SWE-bench Verified and 77.5 on Terminal-Bench 2.1; the 9B claims 69.4% on SWE-bench Verified. That 9B figure would be remarkable if it holds up — as of early August 2026 there is no independent verification, and vendor SWE-bench harnesses have flattered models before. Plan around the hardware math, not the leaderboard.

License check: actually MIT

This is the part worth pausing on. Ornith-1.0 ships under plain MIT — not a community license with MAU caps (Llama 4), not a non-commercial restriction (Flux.1 Dev, MiniMax M3), not attribution-required-above-a-threshold (Kimi K2.7’s modified MIT). MIT means you can self-host it commercially, fine-tune it, redistribute quants, and embed it in a product without asking anyone. There are no regional restrictions either, which has become a real differentiator since some 2026 releases started carving out the EU.

The upstream bases have their own licenses (Qwen 3.5 is Apache 2.0; Gemma 4 moved to Apache 2.0 with the March 2026 generation), so there’s no license conflict buried underneath. For a deeper tour of what these license differences cost you in practice, see our LLM license shootout.

Which variant fits your hardware

VariantQuantDisk/VRAMMinimum realistic hardwareContext you can afford
9BQ4_K_M GGUF~6GBAny 8GB GPU, or CPU-only with 16GB RAM32K+ on 12GB
9BBF16~19GB24GB GPU128K+
31BQ4_K_M GGUF~19GB24GB GPU16–32K
35B MoEQ4_K_M GGUF21.2GB24GB GPU8–16K only
35B MoEQ5_K_M GGUF~25GB32GB (or 24GB + CPU offload)16K with offload
397B MoEFP8~225GB+3–4× H100 80GB64K+

Two things jump out. First, the 35B MoE at Q4_K_M is 21.2GB — it fits a 24GB card like an RTX 4090 or a used RTX 3090, but only barely. The remaining ~2.8GB has to hold the KV cache, so you’re capped around 8–16K context. That’s fine for single-file work and short agent loops; it’s not enough for the 256K window the model card advertises. Second, the 9B is the only variant that makes sense below 16GB — a Q4_K_M on an RTX 3060 12GB leaves comfortable headroom for 32K context.

For a GPU-by-GPU breakdown, our sister site has a dedicated guide: Which GPU runs Ornith-1.0?

Ollama setup (9B and 35B)

DeepReinforce publishes official GGUF repos, so you can pull straight from Hugging Face without waiting for an Ollama library entry:

# 9B — any 8GB+ GPU
ollama run hf.co/deepreinforce-ai/Ornith-1.0-9B-GGUF

# 35B MoE — 24GB GPU
ollama run hf.co/deepreinforce-ai/Ornith-1.0-35B-GGUF

First pull of the 35B downloads ~21GB, so give it a few minutes. Verify it actually landed on your GPU rather than spilling to CPU:

$ ollama ps
NAME                                          SIZE     PROCESSOR    UNTIL
hf.co/deepreinforce-ai/Ornith-1.0-35B-GGUF    23 GB    100% GPU     4 minutes from now

If PROCESSOR shows a CPU/GPU split on a 24GB card, another process (usually your desktop compositor or a browser) is holding VRAM — close it or drop to a lower quant.

Two config details the quick-start skips:

Sampling. DeepReinforce recommends temperature 0.6, top_p 0.95, top_k 20. Ollama’s defaults are close but not identical; set them explicitly in a Modelfile or per-request.

Context. Ollama still defaults to a small num_ctx, and an agentic coding model with a truncated context fails silently — it just forgets your earlier files. Raise it deliberately:

FROM hf.co/deepreinforce-ai/Ornith-1.0-9B-GGUF
PARAMETER num_ctx 32768
PARAMETER temperature 0.6
PARAMETER top_p 0.95
PARAMETER top_k 20

Then ollama create ornith-9b -f Modelfile. On the 35B at 24GB, stay at 8192–16384 or the KV cache alone will push you into offload territory. The same trap breaks Goose, OpenCode, and Cline when they’re wired to local models — we cover it in the OpenCode + Ollama guide.

A problem you will hit: reasoning blocks in your diffs

Ornith-1.0 emits reasoning blocks by default — the model thinks out loud before answering. Run it bare in Ollama and that reasoning can leak into whatever your editor or agent captures, which means chain-of-thought commentary pasted into the middle of a code patch. The fix depends on your client: recent Ollama builds separate the reasoning field automatically for models that declare it; if yours doesn’t, update Ollama first (ollama -v, then upgrade if you’re below the June 2026 releases), and in API calls read message.content, not the raw stream. In Continue.dev and Cline, enable the reasoning/thinking toggle for the model so the block gets folded instead of inlined.

vLLM setup (35B FP8 and 397B)

For serving more than one developer, use vLLM. Ornith-1.0 requires vLLM ≥ 0.19.1 (Transformers ≥ 5.8.1 if you’re loading via HF directly):

pip install "vllm>=0.19.1"

# 35B FP8 on a single 48GB card (A6000, RTX PRO 6000)
vllm serve deepreinforce-ai/Ornith-1.0-35B-FP8 \
  --max-model-len 65536

# 397B flagship — 8-way tensor parallel, per the official README
vllm serve deepreinforce-ai/Ornith-1.0-397B \
  --tensor-parallel-size 8 \
  --max-model-len 262144

The FP8 variants cut VRAM roughly in half versus BF16, which is what makes the 35B servable on one workstation card with real context. Our vLLM setup guide covers the OpenAI-compatible endpoint config that the rest of your tooling plugs into.

The 397B math: RunPod, not your basement

At ~225GB+ in FP8 before KV cache, the 397B needs three to four H100 80GB GPUs minimum, and the official command assumes eight. Nobody’s home lab does this. If you want to evaluate the flagship, rent it: an 8×H100 pod on RunPod runs in the low tens of dollars per hour, which prices a weekend of serious evaluation at less than a month of most coding-agent subscriptions. If the 35B turns out to cover your workload — and for single-repo agent tasks it likely will — you never rent again.

How it stacks up against the local coding incumbents

Ornith-1.0-35B MoEDevstral Small 2 (24B)Qwen3-Coder-Next
LicenseMITApache 2.0Qwen License (attribution)
Fits 24GB at Q4Yes (21.2GB, tight)Yes (comfortable)Yes
ArchitectureMoE, ~3B active (fast decode)DenseMoE
SWE-bench Verified69.4% (9B) / n.a. (35B) — vendor-reported68% — independently reproducedStrong, partly verified
Track record5 weeks oldProven since springProven

The honest comparison: Devstral Small 2 is still the safer 24GB pick because its numbers have been independently reproduced and its dense 24B leaves VRAM headroom for longer context. Ornith’s 35B is the more interesting pick — MoE decode speed means it feels dramatically faster in agent loops, where the model generates far more tokens than you read. If the community reproduces even most of the vendor’s SWE-bench claims, the recommendation flips. Qwen3-Coder-Next remains the pick if you want the largest fine-tune ecosystem, at the cost of a non-MIT license, and Codestral 2 covers the 16GB-VRAM tier the others don’t reach.

If you’re choosing an agent front-end to put in front of any of these, our sister site aicoderscope.com covers the editor-integration side, including Cline and Cursor-style workflows.

When not to run Ornith-1.0

  • You need proven accuracy today. Every Ornith benchmark is vendor-reported as of August 2026. If your use case can’t tolerate a mis-ranked model, wait for independent SWE-bench runs or stick with Codestral 2 / Devstral.
  • You have 24GB and need long context. The 35B Q4_K_M’s 8–16K ceiling undercuts its 256K design. A dense 14B–24B model with room for 64K KV cache serves long agent sessions better.
  • You want a general assistant. Ornith is post-trained hard toward agentic coding. For chat, RAG, and writing, the base Qwen 3.5 / Gemma 4 checkpoints it descends from are better all-rounders.
  • Fine-tuning on day one. The self-scaffolding post-training recipe is new; community LoRA recipes and quant-aware fine-tune configs barely exist yet. Give the ecosystem a quarter.

Verdict

Ornith-1.0 is the most permissively licensed serious coding model family of mid-2026, and the 35B MoE is the first model in a while that makes a single 24GB card feel fast in agent loops. The vendor benchmarks are the asterisk on everything. Pull the 9B today (it costs you 6GB to find out), run the 35B if you have the VRAM and can live with 16K context, and rent before you believe the 397B numbers.

FAQ

Is Ornith-1.0 really MIT-licensed, including for commercial use? Yes. The Hugging Face model cards under the deepreinforce-ai org and the GitHub repo both state MIT with no regional restrictions, MAU caps, or attribution requirements. That’s cleaner than Llama 4, Kimi K2.7, or Qwen’s large-model license.

Can I run the 35B on a 12GB GPU? Technically, with CPU offload — Ollama will split the 21.2GB Q4_K_M between VRAM and system RAM. Expect single-digit tokens per second. The MoE’s ~3B active parameters help less under offload than you’d hope because expert weights still stream over the PCIe bus. On 12GB, run the 9B instead.

Why does my Ornith output include its thinking process? The model emits reasoning blocks by default. Update Ollama to a June 2026+ build so the reasoning field is parsed separately, read message.content in API responses, and enable the thinking toggle in Continue.dev/Cline so the block folds instead of landing in your code.

Sources

  • RTX 4090 — 24GB VRAM, runs the 35B MoE Q4_K_M fully on-GPU
  • RTX 3060 12GB — budget pick for the 9B at 32K context

Was this article helpful?