SGLang can provide the inference API behind an Open WebUI chat interface. The useful starting point is a small model, a working authenticated request, and an explicit container network. Once those three pieces work, you can evaluate a larger model without wondering whether a blank model selector means a GPU failure or a wrong URL.

This guide targets a Linux NVIDIA Docker host and an existing Open WebUI installation. It uses the officially published SGLang v0.5.22 image and a small Qwen checkpoint for connection testing. It is a documentation-verified setup guide, not a hands-on GPU benchmark: we have not measured throughput, VRAM consumption, or response quality on this configuration. The commands below are a reproducible starting configuration that you should validate on your hardware.

Check the licenses before choosing the stack

SGLang’s v0.5.22 LICENSE is Apache-2.0. Describing this release as MIT-licensed would be incorrect. The inference runtime’s license is also separate from the license of every model you load.

The Qwen2.5-0.5B-Instruct model card identifies that checkpoint as Apache-2.0. Its roughly half-billion parameters make it a modest smoke-test download compared with a large production model. They do not establish a guaranteed minimum GPU memory requirement: the runtime, cache, context length, and other allocations also consume memory.

Open WebUI has its own license and branding terms, including a branding clause introduced for v0.6.6 and later. Keep its branding in this setup and review the current license before redistributing or rebranding a deployment. Calling the entire stack uniformly Apache-2.0 would erase an important distinction. If a strictly permissive interface license is a requirement, resolve that choice before building your workflow around the UI.

Decide whether SGLang solves your problem

An API server is useful when more than one client needs a consistent model endpoint. It also adds a service to maintain. A successful installation alone is not evidence that switching engines will improve your workload.

Situation	Practical starting point	What to establish before switching
One person wants local chat with little administration	Keep a working Ollama setup	A specific limitation you need to remove
An application already uses a stable vLLM endpoint	Keep the existing service while comparing	Equal models, prompts, concurrency, and output limits
You need to evaluate SGLang for an API workload	Run a separate SGLang test endpoint	Correct answers and latency under your actual request mix
Your only machine does not match the NVIDIA container requirements	Read the platform-specific installation path	Supported accelerator, driver, and image combination

Our Ollama versus vLLM guide covers the simpler choice between desktop convenience and serving infrastructure. This article focuses on wiring SGLang into a UI, not replacing an already reliable production engine.

Prepare the host and pin the image

The v0.5.22 release notes list lmsysorg/sglang:v0.5.22 as the NVIDIA CUDA 13 image. They list different images for other platforms. Do not interpret the NVIDIA command as an AMD, Intel, or macOS recipe. The release also requires Python 3.11 or newer for its Python installation path; the Docker path keeps that environment inside the image.

Use the official installation instructions to confirm a compatible NVIDIA driver and configured NVIDIA Container Toolkit. First establish that Docker can see the accelerator. If GPU access fails before SGLang starts, changing model flags will not repair the host configuration.

The commands in this article use Bash on the Linux host. They assume Docker is already installed and that you can run it without changing user permissions. They create a named cache volume so restarting the service does not force another model download. Initial image and model downloads require network access and disk space; calling the completed inference path local does not make installation offline.

docker pull lmsysorg/sglang:v0.5.22
docker network create local-ai
docker volume create sglang-hf-cache

If the network already exists, inspect it and reuse it rather than treating that message as a failed model installation. Keep a record of the pulled image digest if you need stronger reproducibility than a version tag provides. Before upgrading later, retain the working image reference and the exact launch command together.

Start a small authenticated inference service

The SGLang request tutorial uses Qwen2.5-0.5B-Instruct as a simple example and documents the chat-completions API. Here it serves only as a connection test, not as a recommendation for difficult coding or reasoning tasks.

Create a temporary local API secret, then launch the container. The generated value is a local authentication token, not a paid provider credential. Docker administrators can inspect container configuration, so this pattern is suitable for a trusted single-host setup; use your deployment’s secret-management process for a shared production host.

export SGLANG_API_KEY="$(python3 -c 'import secrets; print(secrets.token_urlsafe(32))')"

docker run -d --name sglang --network local-ai \
  --gpus all --shm-size 32g \
  -p 127.0.0.1:30000:30000 \
  -v sglang-hf-cache:/root/.cache/huggingface \
  lmsysorg/sglang:v0.5.22 \
  python3 -m sglang.launch_server \
  --model-path Qwen/Qwen2.5-0.5B-Instruct \
  --host 0.0.0.0 --port 30000 \
  --api-key "$SGLANG_API_KEY"

docker logs --tail 100 sglang

The two addresses have different jobs. 0.0.0.0 lets the process accept traffic through the container network. The published host port binds to 127.0.0.1, keeping that host port local instead of exposing it on every host interface. Other containers attached to local-ai can still reach the service, which is why the network should contain only trusted services.

The 32g shared-memory setting follows the installation documentation’s Docker pattern. It is a shared-memory ceiling, not a claim that this small model requires 32 GB of VRAM or immediately allocates 32 GB of host RAM. Avoid copying that number into a hardware shopping list.

Wait for the startup log to show readiness. The first start may be occupied downloading weights; repeated restarts can make diagnosis harder. If startup fails, retain the error and container logs before changing anything.

Prove the API works before adding the UI

From the same shell, request the model list and then a short completion:

curl --fail-with-body http://127.0.0.1:30000/v1/models \
  -H "Authorization: Bearer $SGLANG_API_KEY"

curl --fail-with-body http://127.0.0.1:30000/v1/chat/completions \
  -H "Authorization: Bearer $SGLANG_API_KEY" \
  -H 'Content-Type: application/json' \
  -d '{"model":"Qwen/Qwen2.5-0.5B-Instruct","messages":[{"role":"user","content":"Reply with one short greeting."}],"max_tokens":32,"stream":false}'

Check the returned model identifier rather than assuming a friendly UI label will match the API. If you later change the served name, update clients to use the identifier advertised by the server. The server argument reference documents the model, HTTP, and authentication options; check the installed version’s --help when an option differs from newer documentation.

Success means an HTTP response containing the expected response structure and generated text. This is not a throughput test. A greeting does not validate tool calling, JSON constraints, retrieval quality, long-context behavior, or all OpenAI API features. Test each capability your application actually depends on separately.

Also try the model-list request without the authorization header. It should not grant access to the protected endpoint. If it does, inspect the running command and authentication configuration before connecting additional users.

Connect an existing Open WebUI container

Suppose your existing UI container is named open-webui. Attach it to the same user-defined Docker network:

docker network connect local-ai open-webui

Use your actual container name if it differs. This operation does not replace the UI container or erase its data volume. Record this network attachment in the configuration that recreates the UI, because a later replacement container will otherwise lose the connection.

In Open WebUI, open Settings → Admin → Connections and add an OpenAI-compatible connection with these values:

Field	Value for this two-container setup
API base URL	http://sglang:30000/v1
API key	The generated SGLANG_API_KEY value
Model	The identifier returned by /v1/models

Follow the official OpenAI-compatible connection guide for the current interface. Copy the generated key privately into the configuration; do not put it in a screenshot or shared setup document. Save the connection and send the same short greeting through the selected model.

Inside the UI container, localhost refers to that UI container, not the SGLang container. This is the reason for using the Docker service name sglang. If Open WebUI instead runs directly on the Linux host, use http://127.0.0.1:30000/v1. Neither configuration needs a cloud API account.

Diagnose one layer at a time

Symptom	First check	Avoid this misleading fix
Container exits before listening	Startup logs, GPU visibility, driver compatibility	Repeatedly changing the UI URL
Direct curl request returns unauthorized	Key value and running server arguments	Disabling authentication to hide the mismatch
Host curl works but UI cannot connect	Shared Docker network and sglang hostname	Using UI-container localhost
Models appear but requests fail	Exact model ID and server error log	Assuming discovery proves inference works
Larger model runs out of memory	Weights, context, concurrent load, cache allocation	Treating active MoE parameters as total memory

For memory problems, change one variable at a time and retain a known-working small-model command. The server reference documents cache and prefill controls, but tuning them cannot turn insufficient hardware into a guaranteed fit. Our quantization format guide explains why a filename or bit width alone is an incomplete deployment plan.

Local chat also does not prove that every UI feature is local. Separately configured search, embeddings, document extraction, and other integrations can send data elsewhere. Inspect those connections before using sensitive documents, as the provider overview explains.

Costs and a useful evaluation plan

This configuration does not require a paid inference API. Hardware, electricity, storage, and administration still have costs. For an explicitly hypothetical electricity example, a machine averaging 0.2 kW for three hours daily over 30 days at $0.20/kWh costs 0.2 × 3 × 30 × 0.20 = $3.60. That is arithmetic using assumed inputs, not a measured SGLang power figure. Substitute wall-meter readings and your tariff, and include idle hours if the host stays on.

Before buying hardware or moving an existing service, prepare a small evaluation set from real tasks. Keep input lengths and output limits consistent across engines. Record failures as well as successful latency, and distinguish first-request startup effects from a warm service. For concurrent users, record how individual response times change as requests overlap; a larger aggregate token count can hide an unpleasant chat experience.

Do not turn vendor benchmark gains on datacenter accelerators into a promise for your desktop card. The release notes are useful evidence of supported features and fixes, but your model, precision, GPU, prompt reuse, and concurrency determine the result you actually need. If you cannot maintain drivers and service recovery, our guide to when not to self-host is a better next step than adding more serving flags.

Verdict

SGLang is worth evaluating when you need a configurable local inference API and can maintain the accelerator environment. Start with an authenticated small-model request, connect Open WebUI over an explicit network, and then increase model size and workload complexity. Keep a working Ollama or vLLM deployment until an equivalent comparison gives you a concrete reason to migrate. The result of this guide is a clear connection and diagnosis path, not a claim that one runtime is universally faster.

Sources

• SGLang v0.5.22 release notes
• SGLang v0.5.22 license
• SGLang installation documentation
• SGLang request tutorial
• SGLang server arguments
• Qwen2.5-0.5B-Instruct model card
• Open WebUI compatible providers
• Open WebUI license
• Open WebUI provider overview

Last updated: October 31, 2026. Release, license, and documentation links checked on this date; no GPU benchmark is claimed.

What self-hosting actually costs

Real cost breakdowns for self-hosted AI: hardware floors, power, maintenance hours, and the honest comparison against paying for it. No spam, unsubscribe anytime.

Email address Subscribe