TL;DR: smolagents is Hugging Face’s Apache 2.0 agent framework, and its defining trick — agents that write Python code instead of JSON tool calls — works fine against a local Ollama model. The whole core is about 1,000 lines, so you can actually read what your agent does. The catch is code execution: the default executor is not a security boundary.

	smolagents	LangGraph	Pydantic AI
Best for	Small, readable agents; local models	Complex stateful workflows	Typed, validated outputs
License	Apache 2.0	MIT	MIT
Local Ollama support	Yes, via LiteLLM or OpenAI-compatible	Yes, via langchain-ollama	Yes, via OpenAI-compatible endpoint
The catch	CodeAgent executes LLM-written Python — sandbox it	Steep learning curve, heavy dependency tree	Agent loop is younger, fewer built-in tools

Honest take: if you want one local agent that searches, calculates, and calls your own functions — and you want to understand every line of the framework doing it — smolagents is the fastest path. Reach for LangGraph only when you genuinely need branching, multi-step state machines.

What smolagents actually is

smolagents (github.com/huggingface/smolagents, ~28.7k stars as of August 2026, latest release v1.26.0 from May 29, 2026) is Hugging Face’s deliberately minimal agent library. The pitch is in the name: the core agent logic lives in roughly 1,000 lines of Python, which means when your agent misbehaves you can read the actual loop instead of spelunking through fifteen layers of abstraction.

Its central idea is the CodeAgent. Most agent frameworks make the model emit JSON blobs — {"tool": "search", "query": "..."} — which the framework parses and dispatches. smolagents instead has the model write a short Python snippet as its action. The snippet calls tools as ordinary functions, composes results with normal Python (loops, arithmetic, string handling), and the framework executes it. Hugging Face’s argument, backed by the research the README cites, is that models are better at writing code than at filling JSON schemas, and one code action often replaces three or four JSON round-trips.

There is also a classic ToolCallingAgent that does the JSON-blob dance for when you want it — more on choosing between them below.

The license is plain Apache 2.0, verified against the repo’s LICENSE file. No usage caps, no attribution clauses, commercial self-hosting is fine. That matters because several “open” agent stacks in 2026 pair a permissive harness with restrictively licensed hosted services; smolagents has no such hook.

Install and wire it to Ollama

You need Python 3.10+ and a running Ollama instance. If Ollama is new to you, start with our Ollama + Open WebUI Linux setup guide — for this article I assume ollama serve is already answering on port 11434.

Pull a model that can follow multi-step instructions. Qwen’s mid-size instruct models are the reliable choice for agent work on consumer hardware:

ollama pull qwen2.5-coder:7b
# pulling manifest... success

pip install "smolagents[litellm]"

The [litellm] extra installs LiteLLM, which smolagents uses as a universal adapter — the ollama_chat/ prefix routes requests to your local server. Here is a complete working agent:

from smolagents import CodeAgent, LiteLLMModel

model = LiteLLMModel(
    model_id="ollama_chat/qwen2.5-coder:7b",
    api_base="http://127.0.0.1:11434",
    num_ctx=32768,  # do not skip this — see below
)

agent = CodeAgent(tools=[], model=model, add_base_tools=True)

agent.run(
    "How many days are there between the first Linux kernel "
    "release and the first Git release? Compute it, don't guess."
)

Run it and you get the agent’s visible reasoning loop — each step shows the Python the model wrote, then the execution result:

─ Step 1 ─────────────────────────────────────────────
from datetime import date
linux = date(1991, 9, 17)
git = date(2005, 4, 7)
print((git - linux).days)

Execution logs: 4951
Out: '4951 days'

That’s the whole framework doing its job: the model wrote date arithmetic instead of asking a calculator tool three times. add_base_tools=True also gives it a web search tool and a Python interpreter tool out of the box; with the [toolkit] extra you get the full default tool set.

Custom tools are one decorator on a normal function:

from smolagents import tool

@tool
def gpu_vram(model_name: str) -> str:
    """Returns the VRAM needed for a given local model.

    Args:
        model_name: The Ollama model tag to look up.
    """
    table = {"qwen2.5-coder:7b": "~5GB", "qwen2.5-coder:32b": "~20GB"}
    return table.get(model_name, "unknown")

agent = CodeAgent(tools=[gpu_vram], model=model)

The docstring is not decoration — smolagents parses it into the tool description the model sees, and it will refuse tools whose arguments aren’t documented. Annoying the first time, correct in the long run.

The problem you will actually hit: num_ctx

The first real failure everyone hits with smolagents-on-Ollama is not an error — it’s an agent that loops, repeats itself, or “forgets” the task after two steps. The cause is Ollama’s default context window of 4,096 tokens. A CodeAgent system prompt plus tool definitions plus a couple of observation steps blows past 4k almost immediately, and Ollama silently truncates from the top. The model literally loses the instructions telling it how to act, then flails.

The fix is the num_ctx=32768 line in the config above; LiteLLM passes it through to Ollama per request. If you manage models via Modelfiles instead, set PARAMETER num_ctx 32768 there. Symptoms that this is your problem: the agent re-runs the same code block every step, or step 3 suddenly answers a different question than you asked. We’ve covered the identical trap in the Goose + Ollama setup guide and the OpenCode setup guide — it is the universal tax on local agent tooling in 2026.

Budget VRAM for the bigger cache: a 7B model at Q4 with 32k of context wants roughly 6–8GB, which fits an RTX 3060 12GB with room to spare. If you want to step up to a 32B model for noticeably better multi-step planning, you’re in RTX 3090 24GB territory — the used-market sweet spot our sister site covers in depth at runaihome.com. For bursty experiments beyond your local VRAM, a rented pod on RunPod is cheaper than buying hardware you’ll use twice.

CodeAgent vs ToolCallingAgent on local models

The two agent classes matter more locally than in the cloud, because small models are unevenly good at the two output styles.

CodeAgent asks the model to write Python. Code-tuned models (Qwen2.5-Coder, anything with “coder” in the tag) do this well even at 7B, and they can chain operations in one step — fetch, filter, compute, return. This is the class to default to.

ToolCallingAgent uses the model’s native JSON tool-calling. It only works properly if the model was trained for tool calls, and small local models are hit-or-miss here — a model without solid tool-call training will emit malformed JSON and the agent goes nowhere. Use it when your tools have side effects you want strictly one-at-a-time (sending messages, writing files), where free-form code composition is a liability, and pick a model whose Ollama page explicitly lists tool support.

On a 7B code model, CodeAgent solves multi-step arithmetic-and-lookup tasks that the same model fumbles as JSON tool calls. That asymmetry is the single best argument for smolagents in a self-hosted stack.

Sandboxing: the part you must not skip

A CodeAgent executes Python that an LLM wrote. The built-in LocalPythonExecutor restricts imports and blocks obvious escapes, but the maintainers are explicit that it is not a security boundary — and v1.25.0 (May 14, 2026) shipped fixes for a high-impact vulnerability in remote executors, which tells you this attack surface is live and actively maintained.

Your options, from least to most isolated:

• LocalPythonExecutor (default): fine for tools you wrote yourself and prompts you control. Not fine for anything touching untrusted web content.
• Docker executor (executor_type="docker"): runs each execution in a container. The right default for a self-hosted setup — you’re already running Ollama, one more container is nothing.
• E2B, Modal, Blaxel (cloud sandboxes): strongest isolation, but they reintroduce a cloud dependency and per-use cost, which defeats the point of a local stack for most home-lab use.

Note the WebAssembly executor was removed in v1.26.0, so ignore older tutorials pointing at WasmExecutor. For a self-hoster the practical answer is: LocalPythonExecutor while developing with your own tools, Docker the moment the agent browses the web.

When NOT to use smolagents

• You need durable, branching workflows. smolagents runs a loop until done. Checkpointed state machines, human-in-the-loop gates, and parallel branches are LangGraph’s home turf — see our Flowise vs n8n vs LangGraph comparison for that decision.
• You need strictly validated structured output. Pydantic AI’s whole design is typed results with automatic retries on schema violations. smolagents can return structured data, but it isn’t enforced at the same depth.
• Your local model is small and not code-tuned. A 3B general-chat model writing Python actions produces mostly frustration. Below ~7B code-tuned, use a hosted model or don’t use a CodeAgent.
• You can’t run any sandbox and the agent will see untrusted input. Executing model-written code with no isolation against web content is how you get a bad week.

Verdict

smolagents earns its place in a self-hosted stack by being small enough to audit, permissively licensed, and genuinely better than JSON-based frameworks at squeezing agent behavior out of 7B-class local models. Wire it to Ollama with LiteLLM, set num_ctx before anything else, and move execution into Docker before you point it at the open web. Tested against smolagents v1.26.0 with Ollama, August 2026.

FAQ

Does smolagents require a Hugging Face account or API key for local use? No. The HF token is only needed for InferenceClientModel (their hosted inference). With LiteLLMModel pointed at Ollama, nothing leaves your machine and no account is involved.

Which local model works best with CodeAgent? A code-tuned instruct model at 7B or larger — Qwen2.5-Coder 7B is the floor where multi-step tasks get reliable, and 32B-class models plan noticeably better if you have 24GB of VRAM.

Can smolagents use my OpenAI-compatible server instead of LiteLLM? Yes. OpenAIModel (the OpenAI-compatible client class) accepts any base URL, including Ollama’s http://localhost:11434/v1, vLLM on port 8000, or LM Studio on 1234. LiteLLM is convenient, not mandatory.

Sources

• smolagents GitHub repository — license, README, model classes
• smolagents releases — v1.26.0 (May 29, 2026), v1.25.0 security fixes
• smolagents documentation — secure code execution — executor options and sandbox guidance
• Hugging Face forum: running smolagents locally — Ollama + LiteLLM configuration pattern

Recommended Gear

• RTX 3060 12GB — enough VRAM for a 7B code model with a 32k context window
• RTX 3090 — the used-market pick for 32B-class agent models at 24GB