Local LLM server readiness: check /health and /v1/models before pointing a client at it
TL;DR: A self-hosted inference server has three states that look identical from the outside: it is still loading the model, it is ready, or you are talking to the wrong URL. The llama.cpp server documents a public GET /health endpoint that separates them — 200 with {"status": "ok" } when the model is loaded, and 503 with {"error": {"code": 503, "message": "Loading model", "type": "unavailable_error"}} while it is not. GET /v1/models then tells you which model is actually loaded. Ollama exposes the same OpenAI-compatible surface at /v1/models and /v1/chat/completions.
What you’ll be able to do:
- Tell “the server is still loading the model” apart from “the client is misconfigured” in one request.
- Read the exact HTTP status and body that each server returns before it is ready, instead of guessing from a timeout.
- Build one readiness script that works against llama.cpp’s server and Ollama, and prints a verdict instead of a stack trace.
Honest take:
/healthproves the HTTP surface is up and the model is loaded. It does not prove your client’s base URL, API key, or model name are right — a server can be perfectly healthy while your client is pointed at/v1twice.
Why check the server before you debug the client?
Self-hosted model servers are usually reached from a second piece of software — Open WebUI, Cline, Continue, a LiteLLM proxy, a script. When the pair fails, the error message is usually written by the client, not the server, and it tends to be ambiguous:
- “Connection refused” — the port is wrong, the server is not running, or the process is still binding.
- “Invalid model” — the client asked for a tag the server does not have, or the client sent a name the server does not recognise.
- “Request timed out” — the server is up but the model is still loading, and the client gave up first.
- A 401 — the server requires an API key and the client sent none, or the reverse.
Every one of those messages is compatible with “the server was never ready in the first place”. Checking the server’s own readiness endpoint first removes that whole class of confusion, and it costs one HTTP request.
The llama.cpp server documentation, checked November 15, 2026, is explicit about the endpoint it exposes for exactly this purpose, and about the two different answers it can give.
What the readiness endpoints return
The two servers that most self-hosters run expose overlapping, but not identical, surfaces:
| Endpoint | Method | Server | Ready signal | Not-ready signal |
|---|---|---|---|---|
/health | GET | llama.cpp server | 200 with {"status": "ok" } | 503 with {"error": {"code": 503, "message": "Loading model", "type": "unavailable_error"}} |
/v1/health | GET | llama.cpp server | Same as /health | Same as /health |
/v1/models | GET | llama.cpp server | 200 with one model object in data | 200 with meta set to null while the model is still loading |
/v1/models | GET | Ollama | 200 with a data list of installed models | 404 if the path or server is wrong |
/api/tags | GET | Ollama | 200 with the locally installed models | 404 if the server is not reachable |
/api/version | GET | Ollama | 200 with the version string | 404 if the server is not reachable |
Three details from the documentation are worth spelling out, because they are the ones people get wrong:
/healthis public. The llama.cpp documentation states that this endpoint is “public (no API key check)”. You can probe it even when the server was started with--api-key, so it works as a readiness gate before you have the credentials right./v1/healthalso works. If your client library insists on using the/v1prefix, the health check does not need a separate code path./v1/modelsreturns one element, andmetacan benull. The documentation says the returned list “always has one single element” and that themetafield “can benull(for example, while the model is still loading)”. A200response is therefore not on its own proof of readiness — checkmeta.
The three states, and the exact signals
A self-hosted server that a client cannot use is in one of three states. Each one has a different signal, and only one of them is a problem with your client:
State 1 — the server is not running. A TCP connection to the port is refused. There is no HTTP status at all, because there is no server to answer. This is the only state that is unambiguous, and it is the one the other two are often confused with.
State 2 — the server is running but the model is not loaded yet. The HTTP layer answers, and it answers with a structured error. llama.cpp’s server returns 503 and a body that names the reason:
{"error": {"code": 503, "message": "Loading model", "type": "unavailable_error"}}
The Ollama equivalents are different: Ollama loads a model on first use rather than at startup, so a freshly started server answers /api/tags with 200 immediately, and the first request that needs a model is the one that pays the load time. That difference alone explains a lot of “it hangs on the first message” reports.
State 3 — the server is ready, and your client is wrong. /health returns 200, /v1/models lists a model, and the client still fails. Now the problem is on the client side: the base URL, the API key, or the model name. The two servers differ here too:
- llama.cpp’s server listens on
127.0.0.1and port9931by default, per its documented--hostand--portoptions. Its OpenAI-compatible base URL ishttp://localhost:9931/v1. - Ollama’s OpenAI-compatible base URL is
http://localhost:11434/v1/, and the documentation is explicit that the client “requires an API key value, but Ollama ignores it” — soapi_key="ollama"is the conventional placeholder.
One readiness checker for both servers
The script below probes a base URL, prints the raw signals, and returns a verdict. It uses only the Python standard library, and it never sends a prompt, so it never spends tokens or blocks on generation.
#!/usr/bin/env python3
"""Check whether a local LLM server is ready before a client tries to use it.
Verdicts:
READY the server answered /health with 200 and listed a model
LOADING the server is up but still loading the model
NOT_FOUND the base URL does not look like a model server
UNREACHABLE nothing is listening on that address
"""
import json
import sys
import urllib.error
import urllib.request
# Local servers must not be reached through an HTTP proxy: HTTP_PROXY in the
# environment would otherwise send 127.0.0.1 traffic to the proxy and every
# request would fail with a connection error that looks like a dead server.
OPENER = urllib.request.build_opener(urllib.request.ProxyHandler({}))
def probe(url, method="GET"):
"""Return (status, body) or (None, error-text)."""
req = urllib.request.Request(url, method=method)
try:
with OPENER.open(req, timeout=15) as resp:
return resp.status, resp.read().decode("utf-8", "replace")
except urllib.error.HTTPError as e:
return e.code, e.read().decode("utf-8", "replace")
except Exception as e:
return None, f"{e.__class__.__name__}: {getattr(e, 'reason', e)}"
def main():
base = (sys.argv[1] if len(sys.argv) > 1 else "http://localhost:11434").rstrip("/")
print(f"base {base}")
status, body = probe(f"{base}/health")
if status is None:
print(f"health unreachable ({body})")
print("verdict UNREACHABLE - nothing is listening on that address")
return 1
print(f"health HTTP {status} {body.strip()[:160]}")
if status == 503:
print("verdict LOADING - the server is up but the model is not ready")
return 2
if status == 404:
print("verdict NOT_FOUND - /health is missing; check the base URL and port")
return 1
if status != 200:
print(f"verdict UNREACHABLE - unexpected status {status}")
return 1
status, body = probe(f"{base}/v1/models")
print(f"v1/models HTTP {status} {body.strip()[:160]}")
if status != 200:
print("verdict NOT_FOUND - /v1/models is missing; the base URL probably needs /v1")
return 1
try:
payload = json.loads(body)
except ValueError:
print("verdict NOT_FOUND - /v1/models did not return JSON")
return 1
models = payload.get("data") or []
if not models:
print("verdict LOADING - the server answered but lists no model yet")
return 2
first = models[0]
if first.get("meta", "missing") is None:
print("verdict LOADING - the model entry has meta=null while it loads")
return 2
print(f"model {first.get('id') or first.get('name')}")
print("verdict READY")
return 0
if __name__ == "__main__":
raise SystemExit(main())
Run it against both servers the same way:
python3 check_server_ready.py http://localhost:9931
python3 check_server_ready.py http://localhost:11434
The exit code carries the verdict too: 0 ready, 2 loading, 1 not usable. That makes it usable from a start-up script or a health check in your reverse proxy, where a 503 from your own checker is a better signal than a hung request.
How was the readiness checker validated?
The behaviour above was reproduced locally with a small HTTP fixture that returns the documented responses, using Python 3.13. The fixture answers /health with 200 {"status": "ok" }, with the documented 503 “Loading model” error, or with 404, and it returns a model list whose meta field is either populated or null.
| Fixture case | Condition | Observed verdict | What it proves |
|---|---|---|---|
| Server ready | /health returns 200 and /v1/models lists a model with metadata | READY | A healthy server is recognised without sending a prompt |
| Model still loading | /health returns the documented 503 error body | LOADING | A loading server is not reported as broken |
| Model entry without metadata | /health returns 200 but the model entry has meta: null | LOADING | A 200 alone is not treated as readiness |
| Wrong base URL | /health returns 404 | NOT_FOUND | A wrong path is not confused with a dead server |
| Nothing listening | The port is closed | UNREACHABLE | A refused connection is never reported as “loading” |
These are the documented HTTP contracts exercised against a local fixture. They are not measurements of how long a real model takes to load, and they say nothing about inference speed or output quality.
Two details that break the check in the wild
A proxy in the environment. The llama.cpp documentation itself warns about this for downloads: hosts listed in NO_PROXY and loopback addresses are contacted directly. The same applies to your own tooling — if HTTP_PROXY is set and NO_PROXY does not include loopback addresses, a client can fail to reach a server that is running perfectly. The script above deliberately bypasses proxies for the check.
A reverse proxy in front. If Open WebUI or LiteLLM sits in front of the model server — the LiteLLM local-only proxy setup is one documented path — then the health endpoint you should probe is the one on the proxy, not the one behind it. The Open WebUI nginx and WebSocket guide covers the layer where auth and WebSocket upgrades usually break. Probe the outermost endpoint first, then walk inwards.
Honest take: what a ready server does not prove
READY means the HTTP surface answers and a model is loaded. It does not mean:
- Your client’s configuration is right. A duplicate
/v1in the base URL, a missing API key on a server started with--api-key, or a model name the server does not know will still fail against a perfectly healthy server. - The model answers well. Readiness is a transport-level check. It says nothing about output quality, and nothing about whether the model is the one you meant to serve.
- Concurrency will hold up. A server can be ready and still queue requests behind a single slot under load. For the slots-level view, the llama.cpp server documents
/slotsfor the current processing state and/metricsfor Prometheus-compatible counters. - The right server is running. If two servers are up on the same host, a readiness check against the wrong port still returns
READY.
Check readiness first because it is cheap and it removes ambiguity. Once it passes, every remaining failure is a client-side problem — and that is a much shorter list to debug.
Sources
- LLaMA.cpp HTTP server —
/health,/v1/models,/slots,/metrics,--host,--port,--api-key— llama.cpp repository - OpenAI compatibility — Ollama Docs
- Upstream API reference (
GET /api/tags,GET /api/version,GET /api/ps) — Ollama GitHub - List local models (
GET /api/tags) — Ollama Docs - Ollama vs vLLM for local inference — AIFoss
Last updated November 15, 2026. Official llama.cpp and Ollama documentation was checked on this date. Local verification used a local HTTP fixture with Python 3.13, without running model inference, downloading weights, measuring throughput, or calling any paid API.
Was this article helpful?
Thanks for the feedback — it helps improve future articles.
Need hands-on help?
I offer 1-on-1 technical consulting for local AI setup, GPU selection, and AI coding tool configuration — same topics covered on this site.
Book a session — $49 / hour →What self-hosting actually costs
Real cost breakdowns for self-hosted AI: hardware floors, power, maintenance hours, and the honest comparison against paying for it. No spam, unsubscribe anytime.