llama.cpp v1.0.0: the API is stable now, and what changed for self-hosters

self-hostingllama.cppreleaseopen-source

TL;DR: llama.cpp v1.0.0 was published on November 13, 2026, and the headline is not a feature — it is a promise. The release removes the deprecated public API (35 functions plus llama_set_warmup, defrag_thold and the --defrag-thold flag), promotes the NextN/speculative API into llama.h, and the official announcement states the rule for everything that follows: a breaking change to the public C API requires a major version bump. Measured against v0.9.0 (November 4, 2026) the release is 249 commits, 300 files changed, +7,597 / −5,536 lines. Underneath the version-policy news there are real changes for self-hosters: FP8 KV caches (-ctk / -ctv f8_e4m3), a host-memory MoE expert cache with a configurable CPU share, external PLE shards, a hardened llama-server (SSRF, CORS, DNS rebinding, MCP tool cancellation) and a new OpenAI-compatible /v1/audio/speech endpoint.

What you’ll be able to do:

  • Tell whether upgrading to v1.0.0 breaks the tool you built against llama.h, and how to check in one grep.
  • Know which parts of this release change your day-to-day server usage and which change nothing.
  • Use the new FP8 KV cache to halve the memory cost of a long context, and understand what it does not fix.
  • Decide whether to pin a version or track the tags, now that minor releases promise not to break the public API.

Honest take: this is a library release, and for most people running llama-server — or running Ollama, which vendors its own copy of llama.cpp — nothing changes. The two releases that matter are for embedders (your build may break) and for anyone exposing a server to a network (the hardening is the point). I did not build or benchmark v1.0.0 on this machine; everything below is taken from the release notes, the official announcement, and the GitHub API, all read on November 19, 2026.

The release cadence that got us here

The tag list on the upstream repository shows how fast this project moves:

TagPublishedKind
v0.7.02026-10-15stable
v0.8.02026-10-28stable
v0.9.02026-11-04stable
v1.0.02026-11-13stable

Four stable tags in a month. The release cadence article covers what that does to your pinning strategy in general; the specific change in v1.0.0 is that the rules are now written down.

From the official announcement on Hugging Face (published November 13, 2026 by the ggml-org team): the release “removes the deprecated public API and promotes the NextN/speculative API to llama.h”, and it lays out the semantic versioning policy for all ggml-org projects — MAJOR for “Breaking change to the public C API in ./include”, MINOR for “Backward-compatible feature, backend updates, model support, or API addition”, PATCH for “Bug fix with no API change”. The announcement also says that “both llama.cpp and ggml will reflect API and ABI incompatibilities with major-version bumps”, and that the project will “be stricter about making those boundaries visible to downstream consumers”.

That is the actual headline of v1.0.0: a downstream project can now bump a minor version without fearing that a function you call was renamed out from under you.

What was removed from the public header

The API changes section of the release notes is explicit:

  • llama.h: remove llama_set_warmup, 35 deprecated functions, defrag_thold and --defrag-thold (PR #30565).
  • llama.h: promote the NextN API from llama-ext.h, and rename llama_set_embd_state_src to llama_set_ctx_state_src.
  • llama.h: add moe_cache_size and moe_cache_cpu_share to llama_context_params (PR #30966).

If you build against the library, the check is one grep:

$ grep -rn "llama_set_warmup\|defrag_thold\|llama_set_embd_state_src" src/

Anything it prints is code that needs updating before you move to v1.0.0. The --defrag-thold removal also bites command-line users: any script that passes --defrag-thold to llama-server will now get an unknown-argument error instead of silently doing something.

The server changes that matter in production

For anyone running llama-server as a service, the release notes list hardening as a first-class item: “hardens llama-server (SSRF, CORS, DNS rebinding, MCP tool cancellation)”. Those four are exactly the classes of bug that turn “my local server” into “someone else’s server” when you bind it to 0.0.0.0 or put it behind a proxy. If you expose a llama.cpp server on a network, this release is a security release, and the practical advice is to take it.

The other server-side additions:

  • An OpenAI-compatible /v1/audio/speech endpoint, plus a temperature option for /tts (PRs #31763, #31655). If you have a TTS model running, you can now point an OpenAI-SDK client at it for speech synthesis.
  • A redesigned web UI with chat groups and a markdown editor.
  • MCP tool cancellation — long-running tool calls in a Model Context Protocol setup can now be cancelled instead of running to completion.

FP8 KV caches, and why it matters for long context

Under core changes the release lists: “FP8 KV caches (-ctk/-ctv f8_e4m3) with optional static per-tensor scales” (PR #28392). It also lists a fix for “on-device state copy sizes for quantized KV caches” (PR #30541) — a bug that only shows up when the KV cache is quantized and the tensors live on a device other than the host, which is the iGPU and Metal case.

Why this matters in practice: the KV cache grows linearly with context length, and at f16 it costs 2 bytes per stored element. An FP8 cache costs 1 byte, so it halves the cache footprint — the same order of saving as q8_0, with a format that has an explicit scale model instead of block scales.

If you want the actual numbers per model and per context length, the arithmetic is worked out on the hardware side: KV cache VRAM cost: what 128K context actually costs on a consumer GPU.

MoE expert weights in host memory

The release adds moe_cache_size and moe_cache_cpu_share to llama_context_params, and the notes describe the behaviour as “CPU share for misses, LRFU/ClassLRU replacement, no CPU wait on cached experts” (PRs #30966, #31233, #31888).

Read that carefully: it is a cache for MoE expert weights with a configurable share of host memory, an LRU-family replacement policy, and no blocking wait on experts that are already cached. The practical shape is that a mixture-of-experts model can keep some expert weights in system RAM and still serve cached experts without a CPU round trip. There is no benchmark in the release notes, so treat the performance benefit as unmeasured — the API is new, and what it costs on your workload is yours to measure.

New architectures and models

The release adds three text architectures and one TTS family:

ModelNote from the release notes
Kimi-K3.1-Pro”K3 with the full-attention MLA layers replaced by DSA” (PR #31779)
Ripple-65B-Preview (ricursive)“new dense 65B architecture, no new ops” (PR #31692)
xing4_0”mHC, MLA, MoE and MTP” (PR #31589)
drama-3 TTS (fishaudio)new audio generation path via mtmd (PRs #31613, #31655)

The overview also mentions “MageVL-Cowork and Kimi-K3.1 vision”. And in the multimodal header: MTMD_GEN_AUDIO_TYPE_DRAMA3, a temp parameter in mtmd_helper_gen_audio_inp, and an mtmd_helper_resampler “for streaming raw PCM at the model sample rate” (PR #31964) — that last one is what makes streamed audio output possible rather than “download the whole file”.

How you get the binaries

One practical detail that changes what you do on release day: the v1.0.0 GitHub release page carries no prebuilt binary assets — the asset list contains only nightly-tag.txt. The official announcement lists where the artifacts actually live now:

ChannelPlatforms
DockerLinux
Debian LinuxLinux
HomebrewmacOS, Linux
conda-forgeWindows, macOS, Linux
MacPortsmacOS
WingetWindows
XCFrameworkiOS, macOS

So “download the release asset” is no longer the install path for v1.0.0; the package channels are. For container users, the Docker image is the path of least resistance. The llama.cpp built-in web UI guide covers the server’s own UI, which v1.0.0 redesigns with chat groups and a markdown editor.

Should you upgrade today?

If you…Then
Run llama-server locally and nothing elseUpgrade, but read the flag changes — --defrag-thold is gone
Expose a llama.cpp server on a networkUpgrade today; the SSRF/CORS/DNS-rebinding hardening is the reason
Build against llama.h in your own toolGrep for the removed names first, then upgrade; expect small source edits
Depend on a MoE model and are short on VRAMLook at moe_cache_size / moe_cache_cpu_share, and measure — no benchmark is published
Run a long-context workload on a tight cardTry -ctk f8_e4m3 -ctv f8_e4m3 and check the memory numbers before and after
Pin versions in productionYou can now move to v1.x and take minor updates, because minor releases promise no public API break

Honest take

v1.0.0 is a governance release wearing a feature release’s clothes. The feature list is real and useful — FP8 KV caches and a hardened server are both things you can act on — but the durable change is the versioning promise, and the durable risk is the removal of 35 public functions plus llama_set_warmup from the header that downstream projects have been compiling against for years.

Two things this article does not tell you: how much throughput the MoE CPU-share cache actually buys on your model, and whether the FP8 KV cache changes output quality for your workload. Both need a measurement on your hardware, and the upstream project did not publish either. The CPU inference review and the llama.cpp versus Ollama versus KoboldCpp comparison are still the right starting points if you are choosing a runtime rather than a version.

Sources

Last updated November 19, 2026. The release notes, the official announcement, the release dates and the comparison statistics above were all read on November 19, 2026 from the GitHub API and the Hugging Face blog. v1.0.0 was not built, run or benchmarked on this machine, so this article makes no performance or quality claim; the MoE CPU-share cache and the FP8 KV cache are described as released, not as measured.

Was this article helpful?

What self-hosting actually costs

Real cost breakdowns for self-hosted AI: hardware floors, power, maintenance hours, and the honest comparison against paying for it. No spam, unsubscribe anytime.