Hermes Agent Self-Hosted Setup 2026: Ollama, Zero Cost

hermes-agentollamaai-agentselfhostedai

TL;DR: Hermes Agent is Nous Research’s MIT-licensed, self-hosted AI agent that writes reusable skills to disk after every task it completes, so it genuinely gets faster over time. Wired to a local Ollama model, it costs $0 per task. The catch: skill quality is capped by your local model, and Ollama’s 4,096-token default context breaks it silently.

Hermes AgentGooseOpenCode
Best forPersistent general-purpose agent across chat channelsDesktop + CLI task automationTerminal coding sessions
LicenseMITApache 2.0MIT
Learns between sessionsYes — skill library + memoryNoNo
InterfacesCLI, Telegram, Discord, Slack, WhatsApp, SignalCLI + desktop appTerminal TUI
Local model supportAny OpenAI-compatible endpoint (Ollama, vLLM, llama.cpp, LM Studio)Ollama + 15 providersOllama + 75 providers
The catchSkill library inherits your model’s mistakesStateless between tasksCoding only

Honest take: Hermes is the first self-hosted agent where run #50 is measurably better than run #1 — that alone justifies it over Goose or OpenCode for recurring tasks, as long as you feed it a strong tool-calling model.

Nous Research shipped Hermes Agent in June 2026 and it hit a nerve: the repo sits at over 220,000 GitHub stars as of late July 2026, with v0.19.1 released July 30, 2026. The pitch is different from every other agent framework. Goose, OpenCode, and Open Interpreter all wake up with amnesia — every task starts from zero. Hermes compiles successful task trajectories into skill files on disk, reuses them, and improves them the next time a similar task shows up.

The part the announcement posts skip is the setup that matters: running it against a local Ollama backend so nothing leaves your machine and nothing bills per token. Everything below is current as of Hermes Agent v0.19.1 and Ollama 0.30, July 2026.

License check: MIT, not Apache 2.0

Several roundup posts (including the June 2026 coverage that put Hermes on our radar) describe Hermes Agent as Apache 2.0. The LICENSE file in the NousResearch/hermes-agent repo says MIT. For self-hosters the practical difference is near zero — both allow commercial use, modification, and redistribution without copyleft obligations — but MIT drops Apache’s explicit patent grant. If your legal team cares about patent language, flag it. For a home lab or an internal tool at a small company: run it, modify it, don’t think about it again.

The agent is model-agnostic. Nous would obviously love you to point it at their hosted Hermes models, but nothing in the code requires it — any OpenAI-compatible endpoint works, which is exactly what Ollama exposes on port 11434.

Install

One command on Linux, macOS, WSL2, or Termux:

curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash

Windows has a native PowerShell path (no WSL2 required since v0.18):

iex (irm https://hermes-agent.nousresearch.com/install.ps1)

The installer bundles Python 3.11, Node.js, ripgrep, and ffmpeg, so it works on a bare VPS. Then run the wizard:

hermes setup

Setup asks which provider to use. Pick the custom/OpenAI-compatible option and point it at Ollama — or skip the wizard and edit the config file directly, which is what the next section does.

Wire it to Ollama

Hermes reads ~/.hermes/config.yaml. The minimal local-only block:

model:
  default: qwen3.6-35b-a3b
  provider: custom
  base_url: http://localhost:11434/v1
  context_length: 32768

No API key needed for a local Ollama instance. Three things matter in that block:

Model choice. The model must support tool calling, or Hermes plans eloquently and then does nothing — the same failure mode we hit in the Goose + Ollama setup. Qwen3.6-35B-A3B is the sweet spot on a 24GB card in mid-2026 (see our local setup guide for the quant math); Qwen3-Coder variants also work well. Verify tool support before blaming Hermes:

$ ollama show qwen3.6-35b-a3b
  Model
    architecture        qwen3moe
    context length      262144
    ...
  Capabilities
    completion
    tools

If tools is missing from Capabilities, pick a different model.

Context length. Set it explicitly. Hermes tries to auto-detect context by querying the endpoint’s /v1/models, but Ollama’s answer doesn’t reflect what the server will actually allocate, which brings us to the trap below.

Base URL. The /v1 suffix is required — that’s Ollama’s OpenAI-compatible surface, not its native API.

Running the backend on a different box (vLLM on a GPU server, LM Studio on a Mac)? Define named endpoints and switch mid-session:

custom_providers:
  - name: local
    base_url: http://localhost:11434/v1
  - name: gpubox
    base_url: http://192.168.1.40:8000/v1

Then /model custom:gpubox:qwen3-coder from any chat surface. For vLLM remember --enable-auto-tool-choice, and for llama.cpp’s server pass --jinja, or tool calls come back as plain text.

The problem you will actually hit: the 4k context trap

Give Hermes a real multi-step task on a stock Ollama install and there’s a good chance it loses the thread halfway through — re-asks for information already provided, then declares a half-finished task complete. No error anywhere. Hermes’ own provider docs call this out as the top local-setup failure.

The cause: Ollama does not use a model’s full context window by default. On machines under 24GB VRAM it allocates 4,096 tokens unless told otherwise. Hermes stuffs its system prompt, tool definitions, memory excerpts, and any loaded skills into context before your request even arrives — 4k is gone before the agent reads your first sentence. Ollama then silently truncates from the top, which deletes the system prompt and the tool schemas. The agent doesn’t fail; it lobotomizes.

The fix is server-side, before Hermes ever connects:

OLLAMA_CONTEXT_LENGTH=32768 ollama serve

Or per-model via a Modelfile with PARAMETER num_ctx 32768. Then keep context_length: 32768 in config.yaml so Hermes budgets correctly. 16k is the practical floor for agent work with tools; 32k is comfortable. The same trap breaks OpenCode and Goose — it’s the single most common “local agent is broken” report across all three, and the first thing to check when any tool-calling agent acts drunk.

The skill library is the actual product

Everything above gets you a competent but ordinary agent. The reason to pick Hermes over the alternatives lives in ~/.hermes/skills/.

After Hermes completes a multi-step task it judges as successful, it writes the working procedure back to disk as a skill — a structured file capturing the steps, the tools used, and the gotchas encountered. Next time a similar task arrives, the skill loads into context and the agent skips the trial-and-error. Skills also self-improve: a skill that half-works gets revised in place after the better attempt. List them with /skills in the CLI, invoke one directly with /<skill-name>.

Concretely: the first time you ask for something like “fetch this RSS feed, summarize new entries, post to Slack,” the agent burns tool calls on dead ends figuring out the feed format and the Slack payload. Once the resulting skill exists, the same request replays the known-good procedure and skips the exploration. That’s the compounding loop — and it persists across restarts, unlike anything Goose or OpenCode retain. Alongside skills, Hermes keeps agent-curated memory (MEMORY.md, USER.md) and FTS5 full-text search over past sessions with LLM summarization, so “like last month’s report” actually resolves.

Two honest caveats. First, the skill library inherits your model’s judgment: a 7B model that “succeeds” at a task badly will happily write a bad skill, and that pollution compounds the same way the good stuff does. Audit ~/.hermes/skills/ occasionally and delete garbage — they’re plain files. Run at least a 30B-class MoE locally if you want the library to stay clean. Second, skills are executable instructions your agent will follow later. Treat third-party skills from HermesHub or the agentskills.io ecosystem like you’d treat a stranger’s shell script: read before installing.

One gateway, every chat app

Hermes runs as a daemon that bridges chat surfaces:

hermes gateway

One process serves CLI, Telegram, Discord, Slack, WhatsApp, and Signal simultaneously, with the same memory and skill library behind all of them. Message your agent from your phone; it executes on your box at home. This is where self-hosting pays off: an always-on agent with persistent memory of your projects is exactly the thing you don’t want living on someone else’s server. If you expose any of this beyond localhost, read our Ollama security guide first — 175K exposed instances say most people don’t.

Hardware and the VPS split

The agent itself is lightweight — Nous advertises it running on a $5 VPS, and that’s real because the agent is orchestration, not inference. The heavy lifting happens wherever your model lives. A sensible split for 24/7 use: Hermes on a cheap VPS or a Raspberry-class box handling the gateway, pointing at a GPU machine on your LAN — a used RTX 3090 still being the 24GB value pick; see runaihome.com’s local AI GPU guides for current builds. No local GPU? A RunPod instance running vLLM gives you the same custom_providers wiring at cloud prices, and you can keep a small Ollama model configured as the fallback.

When NOT to use Hermes Agent

  • Pure coding work. For repo-scale coding sessions, OpenCode and Aider are sharper tools — purpose-built editing loops beat a generalist with a coding skill. The coding-tool comparisons at aicoderscope.com cover that lane.
  • One-off tasks. The skill loop needs repetition to pay for itself. For a single scripted job, Goose is simpler and stateless by design.
  • Small local models. Under ~14B, tool-calling reliability drops and the skill library fills with junk. At that point a hosted model via OpenRouter (Hermes supports it natively) beats a local 7B for agent work — accept the privacy tradeoff or don’t run an agent.
  • Untrusted input streams. An agent with shell access reading inbound WhatsApp messages is a prompt-injection surface. Keep channels limited to people you trust, or sandbox the terminal backend (Hermes supports Docker and remote sandboxes like Modal or Daytona as execution targets).

Verdict

Hermes Agent is the most interesting self-hosted agent release of 2026 so far, and the momentum (v0.19.0 “Quicksilver” cut first-token latency roughly 80% on July 20, then v0.19.1 stabilized it ten days later) suggests Nous is treating it as a flagship, not a demo. The skill library is a genuine architectural difference, not a feature bullet — paired with a strong local MoE on a 24GB card, you get a private agent that compounds. Start it on the CLI, let it earn skills for a week, then decide if it deserves a gateway daemon and a Slack token.

FAQ

Does Hermes Agent require Nous Research’s own models? No. Any OpenAI-compatible endpoint works — Ollama, vLLM, llama.cpp, LM Studio, or hosted providers via OpenRouter. Switch anytime with hermes model or /model mid-session.

Where are skills stored, and can I edit them? In ~/.hermes/skills/ as plain files. You can read, edit, delete, and version-control them, and install community skills compatible with the agentskills.io standard.

How much VRAM do I need to run it fully locally? The agent runs on nearly anything; the model is the constraint. Plan for a tool-calling model at 14B minimum, ideally a 30B-class MoE (~21GB at Q4) on a 24GB GPU, with 16k–32k context configured in Ollama.

Sources

Was this article helpful?