Ornith-1.0 Self-Hosting Guide 2026: Ollama and vLLM
TL;DR: Ornith-1.0 is DeepReinforce’s MIT-licensed coding model family (released June 25, 2026) in four sizes: 9B and 31B dense, 35B MoE, and a 397B MoE flagship. The 9B runs on any 8GB card; the 35B MoE just fits a 24GB GPU at Q4_K_M. Benchmarks are vendor-reported for now, so treat the 397B’s 82.4% SWE-bench Verified claim as promising, not proven.
| Ornith-1.0-9B | Ornith-1.0-35B MoE | Ornith-1.0-397B MoE | |
|---|---|---|---|
| Best for | 8–12GB GPUs, laptop coding | Single 24GB card, best local value | Multi-GPU nodes, cloud rental |
| VRAM at Q4 | ~6GB | 21.2GB (tight on 24GB) | ~225GB+ (FP8) |
| The catch | Weakest of the family | KV cache forces 8–16K context on 24GB | Not realistic on home hardware |
Honest take: run the 35B MoE if you have a 24GB card — it decodes like a small model because only ~3B parameters are active per token. Everyone else should start with the 9B and keep expectations calibrated until independent benchmarks land.
What Ornith-1.0 actually is
DeepReinforce released Ornith-1.0 on June 25, 2026: a family of coding models trained with a self-scaffolding reinforcement learning loop. Instead of humans hand-writing the RL training harness, the model writes and revises its own scaffold — the test runners, reward checks, and task decomposition it trains against. Whether that mechanism is why the numbers look good is impossible to say from outside, but the release itself is unusually complete: weights for every size, GGUF quants for the two smaller tiers, FP8 for the big MoEs, and day-one vLLM support.
The four variants, per the release notes and Hugging Face model cards:
- Ornith-1.0-9B — dense, post-trained from Qwen 3.5
- Ornith-1.0-31B — dense, post-trained from Gemma 4
- Ornith-1.0-35B — mixture-of-experts, ~3B active parameters per token
- Ornith-1.0-397B — MoE flagship, shipped in BF16 and FP8
All four share a 256K (262,144-token) context window, which matters more for agentic coding than raw benchmark scores — long agent sessions eat context fast.
Vendor-reported numbers: the 397B posts 82.4% on SWE-bench Verified and 77.5 on Terminal-Bench 2.1; the 9B claims 69.4% on SWE-bench Verified. That 9B figure would be remarkable if it holds up — as of early August 2026 there is no independent verification, and vendor SWE-bench harnesses have flattered models before. Plan around the hardware math, not the leaderboard.
License check: actually MIT
This is the part worth pausing on. Ornith-1.0 ships under plain MIT — not a community license with MAU caps (Llama 4), not a non-commercial restriction (Flux.1 Dev, MiniMax M3), not attribution-required-above-a-threshold (Kimi K2.7’s modified MIT). MIT means you can self-host it commercially, fine-tune it, redistribute quants, and embed it in a product without asking anyone. There are no regional restrictions either, which has become a real differentiator since some 2026 releases started carving out the EU.
The upstream bases have their own licenses (Qwen 3.5 is Apache 2.0; Gemma 4 moved to Apache 2.0 with the March 2026 generation), so there’s no license conflict buried underneath. For a deeper tour of what these license differences cost you in practice, see our LLM license shootout.
Which variant fits your hardware
| Variant | Quant | Disk/VRAM | Minimum realistic hardware | Context you can afford |
|---|---|---|---|---|
| 9B | Q4_K_M GGUF | ~6GB | Any 8GB GPU, or CPU-only with 16GB RAM | 32K+ on 12GB |
| 9B | BF16 | ~19GB | 24GB GPU | 128K+ |
| 31B | Q4_K_M GGUF | ~19GB | 24GB GPU | 16–32K |
| 35B MoE | Q4_K_M GGUF | 21.2GB | 24GB GPU | 8–16K only |
| 35B MoE | Q5_K_M GGUF | ~25GB | 32GB (or 24GB + CPU offload) | 16K with offload |
| 397B MoE | FP8 | ~225GB+ | 3–4× H100 80GB | 64K+ |
Two things jump out. First, the 35B MoE at Q4_K_M is 21.2GB — it fits a 24GB card like an RTX 4090 or a used RTX 3090, but only barely. The remaining ~2.8GB has to hold the KV cache, so you’re capped around 8–16K context. That’s fine for single-file work and short agent loops; it’s not enough for the 256K window the model card advertises. Second, the 9B is the only variant that makes sense below 16GB — a Q4_K_M on an RTX 3060 12GB leaves comfortable headroom for 32K context.
For a GPU-by-GPU breakdown, our sister site has a dedicated guide: Which GPU runs Ornith-1.0?
Ollama setup (9B and 35B)
DeepReinforce publishes official GGUF repos, so you can pull straight from Hugging Face without waiting for an Ollama library entry:
# 9B — any 8GB+ GPU
ollama run hf.co/deepreinforce-ai/Ornith-1.0-9B-GGUF
# 35B MoE — 24GB GPU
ollama run hf.co/deepreinforce-ai/Ornith-1.0-35B-GGUF
First pull of the 35B downloads ~21GB, so give it a few minutes. Verify it actually landed on your GPU rather than spilling to CPU:
$ ollama ps
NAME SIZE PROCESSOR UNTIL
hf.co/deepreinforce-ai/Ornith-1.0-35B-GGUF 23 GB 100% GPU 4 minutes from now
If PROCESSOR shows a CPU/GPU split on a 24GB card, another process (usually your desktop compositor or a browser) is holding VRAM — close it or drop to a lower quant.
Two config details the quick-start skips:
Sampling. DeepReinforce recommends temperature 0.6, top_p 0.95, top_k 20. Ollama’s defaults are close but not identical; set them explicitly in a Modelfile or per-request.
Context. Ollama still defaults to a small num_ctx, and an agentic coding model with a truncated context fails silently — it just forgets your earlier files. Raise it deliberately:
FROM hf.co/deepreinforce-ai/Ornith-1.0-9B-GGUF
PARAMETER num_ctx 32768
PARAMETER temperature 0.6
PARAMETER top_p 0.95
PARAMETER top_k 20
Then ollama create ornith-9b -f Modelfile. On the 35B at 24GB, stay at 8192–16384 or the KV cache alone will push you into offload territory. The same trap breaks Goose, OpenCode, and Cline when they’re wired to local models — we cover it in the OpenCode + Ollama guide.
A problem you will hit: reasoning blocks in your diffs
Ornith-1.0 emits reasoning blocks by default — the model thinks out loud before answering. Run it bare in Ollama and that reasoning can leak into whatever your editor or agent captures, which means chain-of-thought commentary pasted into the middle of a code patch. The fix depends on your client: recent Ollama builds separate the reasoning field automatically for models that declare it; if yours doesn’t, update Ollama first (ollama -v, then upgrade if you’re below the June 2026 releases), and in API calls read message.content, not the raw stream. In Continue.dev and Cline, enable the reasoning/thinking toggle for the model so the block gets folded instead of inlined.
vLLM setup (35B FP8 and 397B)
For serving more than one developer, use vLLM. Ornith-1.0 requires vLLM ≥ 0.19.1 (Transformers ≥ 5.8.1 if you’re loading via HF directly):
pip install "vllm>=0.19.1"
# 35B FP8 on a single 48GB card (A6000, RTX PRO 6000)
vllm serve deepreinforce-ai/Ornith-1.0-35B-FP8 \
--max-model-len 65536
# 397B flagship — 8-way tensor parallel, per the official README
vllm serve deepreinforce-ai/Ornith-1.0-397B \
--tensor-parallel-size 8 \
--max-model-len 262144
The FP8 variants cut VRAM roughly in half versus BF16, which is what makes the 35B servable on one workstation card with real context. Our vLLM setup guide covers the OpenAI-compatible endpoint config that the rest of your tooling plugs into.
The 397B math: RunPod, not your basement
At ~225GB+ in FP8 before KV cache, the 397B needs three to four H100 80GB GPUs minimum, and the official command assumes eight. Nobody’s home lab does this. If you want to evaluate the flagship, rent it: an 8×H100 pod on RunPod runs in the low tens of dollars per hour, which prices a weekend of serious evaluation at less than a month of most coding-agent subscriptions. If the 35B turns out to cover your workload — and for single-repo agent tasks it likely will — you never rent again.
How it stacks up against the local coding incumbents
| Ornith-1.0-35B MoE | Devstral Small 2 (24B) | Qwen3-Coder-Next | |
|---|---|---|---|
| License | MIT | Apache 2.0 | Qwen License (attribution) |
| Fits 24GB at Q4 | Yes (21.2GB, tight) | Yes (comfortable) | Yes |
| Architecture | MoE, ~3B active (fast decode) | Dense | MoE |
| SWE-bench Verified | 69.4% (9B) / n.a. (35B) — vendor-reported | 68% — independently reproduced | Strong, partly verified |
| Track record | 5 weeks old | Proven since spring | Proven |
The honest comparison: Devstral Small 2 is still the safer 24GB pick because its numbers have been independently reproduced and its dense 24B leaves VRAM headroom for longer context. Ornith’s 35B is the more interesting pick — MoE decode speed means it feels dramatically faster in agent loops, where the model generates far more tokens than you read. If the community reproduces even most of the vendor’s SWE-bench claims, the recommendation flips. Qwen3-Coder-Next remains the pick if you want the largest fine-tune ecosystem, at the cost of a non-MIT license, and Codestral 2 covers the 16GB-VRAM tier the others don’t reach.
If you’re choosing an agent front-end to put in front of any of these, our sister site aicoderscope.com covers the editor-integration side, including Cline and Cursor-style workflows.
When not to run Ornith-1.0
- You need proven accuracy today. Every Ornith benchmark is vendor-reported as of August 2026. If your use case can’t tolerate a mis-ranked model, wait for independent SWE-bench runs or stick with Codestral 2 / Devstral.
- You have 24GB and need long context. The 35B Q4_K_M’s 8–16K ceiling undercuts its 256K design. A dense 14B–24B model with room for 64K KV cache serves long agent sessions better.
- You want a general assistant. Ornith is post-trained hard toward agentic coding. For chat, RAG, and writing, the base Qwen 3.5 / Gemma 4 checkpoints it descends from are better all-rounders.
- Fine-tuning on day one. The self-scaffolding post-training recipe is new; community LoRA recipes and quant-aware fine-tune configs barely exist yet. Give the ecosystem a quarter.
Verdict
Ornith-1.0 is the most permissively licensed serious coding model family of mid-2026, and the 35B MoE is the first model in a while that makes a single 24GB card feel fast in agent loops. The vendor benchmarks are the asterisk on everything. Pull the 9B today (it costs you 6GB to find out), run the 35B if you have the VRAM and can live with 16K context, and rent before you believe the 397B numbers.
FAQ
Is Ornith-1.0 really MIT-licensed, including for commercial use? Yes. The Hugging Face model cards under the deepreinforce-ai org and the GitHub repo both state MIT with no regional restrictions, MAU caps, or attribution requirements. That’s cleaner than Llama 4, Kimi K2.7, or Qwen’s large-model license.
Can I run the 35B on a 12GB GPU? Technically, with CPU offload — Ollama will split the 21.2GB Q4_K_M between VRAM and system RAM. Expect single-digit tokens per second. The MoE’s ~3B active parameters help less under offload than you’d hope because expert weights still stream over the PCIe bus. On 12GB, run the 9B instead.
Why does my Ornith output include its thinking process?
The model emits reasoning blocks by default. Update Ollama to a June 2026+ build so the reasoning field is parsed separately, read message.content in API responses, and enable the thinking toggle in Continue.dev/Cline so the block folds instead of landing in your code.
Sources
- DeepReinforce Ornith-1 GitHub repository — license, variants, vLLM commands, sampling settings
- Ornith-1.0-35B-GGUF on Hugging Face — quant file sizes
- MarkTechPost: DeepReinforce Releases Ornith-1.0 — release coverage, benchmark claims
- TechTimes: Ornith-1.0 Writes Its Own Training Scaffold — self-scaffolding RL background
- runaihome.com Ornith-1.0 GPU guide — per-GPU VRAM testing
Recommended Gear
- RTX 4090 — 24GB VRAM, runs the 35B MoE Q4_K_M fully on-GPU
- RTX 3060 12GB — budget pick for the 9B at 32K context
Was this article helpful?
Thanks for the feedback — it helps improve future articles.
Need hands-on help?
I offer 1-on-1 technical consulting for local AI setup, GPU selection, and AI coding tool configuration — same topics covered on this site.
Book a session — $49 / hour →