Pydantic AI + Ollama: Type-Safe Local Agents in Python

Short answer: Pydantic AI has first-class Ollama support — pip install "pydantic-ai-slim[openai]", point OllamaProvider at http://localhost:11434/v1, and qwen3:8b (5.2GB download, fits an 8GB GPU) gives you a tool-calling agent whose output is a validated Pydantic model, not a string you regex-parse. On self-hosted Ollama v0.5.0+, NativeOutput mode is genuinely schema-enforced — Ollama constrains generation to your JSON schema — and MCP servers plug in with three lines via MCPToolset. Everything on this page runs free, offline, with no API key.

That is the whole pitch, and it is a strong one: of the big Python agent frameworks, Pydantic AI is the one built by the Pydantic team itself, so "the LLM returns typed, validated data or the run fails loudly" is the core design rather than an add-on. Below is the complete working stack — setup, model choice, structured output, tools, MCP — verified against the official docs and current package versions in August 2026, plus the failure modes the tutorials skip.

---

Why Choose Pydantic AI for Local Agents?

Answer first: pick Pydantic AI when agent output feeds real code — its whole design is typed input/output with Pydantic validation, it is MIT-licensed with ~19k GitHub stars, and version 2.25 (August 2026) has a dedicated Ollama provider, so local is a first-class citizen, not a workaround.

Every agent framework demo looks the same: model plans, calls a tool, prints a paragraph. The difference shows up the day you need the result in a program — an InvoiceData object going into a database, not markdown that mostly resembles one. Pydantic AI's bet is that this boundary is the hard part. An Agent is parameterized by an output type; the run returns an instance of that type or raises. Your IDE autocompletes result.output.total. That is the feature.

Where it sits against the frameworks we have covered before:

• LangChain + Ollama has the largest integration catalog by far, at the cost of a much larger API surface.
• CrewAI owns the multi-agent "crew of roles" pattern.
• Pydantic AI is the smallest of the three to hold in your head, and the only one where schema-validated output is the default posture rather than an output-parser bolted on.

Our broader agent framework comparison covers the field; this page is the local-first Pydantic AI track specifically. Current version: pydantic-ai 2.25.0 as of early August 2026, Python 3.10+ (PyPI). The docs are maintained by the Pydantic team and have been notably stable — but note the project moved fast through 2025, so older community tutorials show APIs (like MCPServerStdio) that current releases have replaced. Everything below matches the 2.x docs.

---

Setup in Five Minutes

Answer first: two installs and one pull — Ollama itself, qwen3:8b (5.2GB), and pydantic-ai-slim[openai]. The openai extra is not a cloud dependency; it provides the OpenAI-compatible transport that Ollama's local endpoint speaks.

# 1. Install Ollama from ollama.com/download, then pull a tool-capable model
ollama pull qwen3:8b        # 5.2GB download

# 2. Install Pydantic AI (slim install + the transport the Ollama provider uses)
pip install "pydantic-ai-slim[openai]"

The minimal agent, exactly per the official Ollama docs page:

from pydantic_ai import Agent
from pydantic_ai.models.ollama import OllamaModel
from pydantic_ai.providers.ollama import OllamaProvider

model = OllamaModel(
    'qwen3:8b',
    provider=OllamaProvider(base_url='http://localhost:11434/v1'),
)
agent = Agent(model, instructions='Answer in one sentence.')

result = agent.run_sync('Why is the sky blue?')
print(result.output)

Two things worth noticing. First, OllamaModel and OllamaProvider are dedicated classes now — the old pattern of aiming a generic OpenAI model class at localhost still floats around in 2025-era tutorials, but the dedicated provider is what the docs specify and what knows Ollama's quirks (more on that in the structured-output section). Second, base_url ends in /v1 — Ollama's OpenAI-compatible endpoint, not the bare :11434. Leaving off the /v1 is the most common "connection refused-ish" mistake in the project's issue threads.

There is nothing else. No API key, no environment variable, no billing page. If ollama list shows the model, the agent runs.

---

Which Local Model Should You Use?

Answer first: qwen3:8b is the default pick — it is the family Pydantic's own docs use, it has both tools and thinking capabilities on its Ollama library page, and 5.2GB fits an 8GB card. Move to qwen3:14b (9.3GB) on a 12-16GB GPU for visibly steadier multi-step tool use.

Agent work is the one workload where model choice is not negotiable: the model must emit well-formed tool calls, repeatedly, mid-conversation. The qwen3 family is the safest local bet right now — the Ollama library marks the whole line with tool support, and it sits at 33.4M+ downloads. Download sizes below are from the Ollama library (August 2026). The VRAM column is arithmetic, not a measurement: a quantized model needs roughly its download size in memory plus headroom for context and the KV cache, so round up a tier if you plan to run long conversations.

The docs' own example model is qwen3, which resolves to the 8b default tag. Our full qwen3 setup guide covers quantization variants and thinking mode; for a wider survey of which local models can actually call tools without face-planting, see best Ollama models for agents.

One behavioral note: qwen3 is a thinking model, so expect it to spend tokens reasoning before it acts. For agents this is usually a feature — tool-call accuracy is what you are buying — but it adds latency you will notice on a small GPU.

---

Is Structured Output Actually Enforced?

Answer first: with output_type=NativeOutput(YourModel) on self-hosted Ollama v0.5.0+, the model is constrained to emit JSON matching your schema — per the Pydantic AI docs this is real schema-valid generation via json_schema, not a polite request in the prompt. Ollama Cloud does not enforce it, and Pydantic AI knows to disable native mode there automatically.

This is the marquee feature of the pairing, so let us be precise about what is guaranteed and by whom.

Pydantic AI has three output modes (per the official output docs):

The docs describe NativeOutput as using the model's native structured-outputs feature, "where the model is forced to only output text matching the provided JSON schema." On the Ollama side, that feature is the structured outputs Ollama shipped in v0.5.0 (announced December 6, 2024): pass a JSON schema in the format parameter and Ollama constrains the model's output to it. Pydantic AI's Ollama page confirms the pairing explicitly — self-hosted Ollama v0.5.0+ supports NativeOutput with schema-valid generation via json_schema.

In code:

from pydantic import BaseModel
from pydantic_ai import Agent, NativeOutput
from pydantic_ai.models.ollama import OllamaModel
from pydantic_ai.providers.ollama import OllamaProvider


class CityInfo(BaseModel):
    city: str
    country: str
    population: int


model = OllamaModel(
    'qwen3:8b',
    provider=OllamaProvider(base_url='http://localhost:11434/v1'),
)
agent = Agent(model, output_type=NativeOutput(CityInfo))

result = agent.run_sync('Tell me about Tokyo.')
print(result.output)             # CityInfo(city='Tokyo', country='Japan', population=...)
print(result.output.population)  # an actual int — parsed and validated

No json.loads, no retry-on-parse-error loop, no "please respond ONLY with JSON" incantation in the prompt. The object either validates or the run errors.

Three honest caveats:

1. Schema-valid is not fact-valid. Enforcement guarantees the shape. The population figure is still whatever the model believes; a grammar cannot make it true.
2. Ollama Cloud is different. Per the docs, Ollama's cloud endpoint (https://ollama.com/v1) does not enforce json_schema — schemas are silently ignored there, so Pydantic AI raises a UserError if you request NativeOutput with a cloud model; use the default ToolOutput or PromptedOutput instead. This page is about local anyway, where the guarantee holds.
3. When an agent also has function tools, we default to ToolOutput. It is Pydantic AI's default for a reason — one mechanism (tool calls) for both tools and output keeps small local models on familiar ground. Reach for NativeOutput on pure extraction jobs where the hard shape guarantee is the point.

---

How Do You Give the Agent Tools?

Answer first: decorate plain Python functions with @agent.tool_plain (or @agent.tool when they need RunContext deps); type hints plus the docstring become the tool schema automatically, and the typed result comes back through the same validated pipeline.

A compact but real example — a release-notes triage agent that reads your git log and returns structured data your CI can act on:

import subprocess
from pydantic import BaseModel
from pydantic_ai import Agent
from pydantic_ai.models.ollama import OllamaModel
from pydantic_ai.providers.ollama import OllamaProvider


class ReleaseSummary(BaseModel):
    headline: str
    breaking_changes: list[str]
    needs_migration_guide: bool


model = OllamaModel(
    'qwen3:14b',
    provider=OllamaProvider(base_url='http://localhost:11434/v1'),
)
agent = Agent(
    model,
    output_type=ReleaseSummary,   # default ToolOutput mode — plays nicely with tools
    instructions='Summarize the release from the commit log. Be terse.',
)


@agent.tool_plain
def git_log(n: int = 30) -> str:
    """Return the last n commit subject lines."""
    out = subprocess.run(
        ['git', 'log', f'-{n}', '--pretty=%s'],
        capture_output=True, text=True, check=True,
    )
    return out.stdout


result = agent.run_sync('Draft the summary for the upcoming release.')
print(result.output.breaking_changes)      # list[str], guaranteed

The model decides when to call git_log, Pydantic AI executes it, feeds the result back, and keeps going until the model produces a valid ReleaseSummary. When a tool needs shared state or credentials, use @agent.tool with a RunContext first parameter and pass deps= at run time — same pattern, dependency-injected. Tools can also be passed as a plain list via Agent(tools=[...]) if you prefer functions defined elsewhere.

What to expect from local models here: reliability falls off as the tool chain lengthens, because every hop is another chance to emit a malformed call or the wrong argument, and the errors compound. An 8b model is fine for one or two hops; long autonomous chains are where stepping up to qwen3:14b earns its VRAM. Either way, design agents so a single run needs few hops — fewer, bigger tools beat many tiny ones on local models.

---

How Do You Connect an MCP Server?

Answer first: MCPToolset('http://localhost:8000/mcp') (or a path to a local server script) passed via toolsets=[...] gives your local agent every tool that MCP server exposes — three lines, lifecycle managed automatically.

The Model Context Protocol is how the ecosystem ships reusable tool servers — filesystem access, browsers, databases — instead of everyone re-writing the same functions (MCP explained here). Pydantic AI's client side is MCPToolset:

from pydantic_ai import Agent
from pydantic_ai.mcp import MCPToolset
from pydantic_ai.models.ollama import OllamaModel
from pydantic_ai.providers.ollama import OllamaProvider

model = OllamaModel(
    'qwen3:14b',
    provider=OllamaProvider(base_url='http://localhost:11434/v1'),
)

toolset = MCPToolset('http://localhost:8000/mcp')   # any MCP server URL
agent = Agent(model, toolsets=[toolset])


async def main():
    result = await agent.run('What files changed in the last commit?')
    print(result.output)

Per the docs, MCPToolset accepts a Streamable HTTP or SSE URL, a path to a local Python or Node.js server script (stdio), a FastMCP transport object, or an in-process FastMCP server. You can manage connections with async with agent:, but if you do not, the toolset is opened and closed automatically as needed — one less footgun than the earlier API generations had.

Two practical notes. First, this is the section where old tutorials will burn you: pre-2.x examples use MCPServerStdio/MCPServerSSE and agent.run_mcp_servers() — that era's API. If you are following a 2025 blog post and imports fail, this is why; the current class is MCPToolset. Second, MCP tools arrive at the model as ordinary function tools, so everything said above about local tool-calling quality applies unchanged — a server exposing 40 tools will drown an 8b model in choices. Prefer focused servers, or filter what you expose. Our Ollama MCP integration guide covers the server side of this same stack.

---

Honest Limitations

Answer first: the framework is solid; the ceiling is the local model. Budget for flaky long tool chains below 14b, remember schema enforcement stops at shape, and pin your versions — this project has a history of API renames.

• Small models fumble long chains. The single biggest gap between a GPT-class demo and your local run. One or two tool hops on qwen3:8b: fine. Long autonomous loops: use qwen3:14b+ or restructure the task. This is a model limit, not a Pydantic AI limit — the framework's retry-on-validation-failure loop actually papers over a lot of small-model sloppiness.
• NativeOutput guarantees shape, not truth — and it is enforced on self-hosted Ollama only, not Ollama Cloud. Validation passing tells you the JSON parsed, not that the answer is right.
• API churn risk. Pydantic AI went through real renames on its way to 2.x (the MCP client API most visibly). Docs are excellent and current, but pin pydantic-ai in production and read release notes before jumping majors. The flip side: the 2.x surface has been stable through 2026 and the project is run by the Pydantic team, which maintains one of the most-depended-on packages in Python — abandonment risk is about as low as it gets.
• Thinking models trade latency for accuracy. qwen3's reasoning phase means slower first tokens than a non-thinking model of the same size. For agents we accept the trade; for chat UX you might not.
• No multi-agent orchestration opinions. Pydantic AI gives you agents, tools, toolsets — composition is your job. If you want an opinionated crew topology out of the box, that is CrewAI's lane.

---

Verdict

Pydantic AI + Ollama is currently our recommended stack for local agents that feed structured data into real programs. The type-safety story is not marketing — schema-enforced generation on self-hosted Ollama plus Pydantic validation on the way out means the classic agent failure mode (garbled JSON at 2am) largely disappears. Setup is genuinely two commands, the official docs treat Ollama as a first-class target, and MCP support is three lines.

The recipe once more: ollama pull qwen3:8b to start, qwen3:14b on a 16GB card the moment your agent needs more than two tool hops, pip install "pydantic-ai-slim[openai]", default ToolOutput for tool-using agents, NativeOutput for hard-shape extraction. Total cost: $0 and some VRAM you already own.

---

Sources

• Pydantic AI docs — Ollama — OllamaModel/OllamaProvider setup, base URLs, NativeOutput support matrix (self-hosted v0.5.0+ vs Cloud)
• Pydantic AI docs — Output, Tools, MCP client — output modes, tool decorators, MCPToolset
• pydantic-ai on PyPI — version 2.25.0 as of August 2026; Python 3.10+
• pydantic/pydantic-ai on GitHub — ~19.1k stars, MIT license (August 2026)
• Ollama blog — Structured outputs — schema-constrained generation, announced December 6, 2024
• qwen3 on the Ollama library — tags, download sizes, tools/thinking capabilities, 33.4M downloads (August 2026)

---

FAQ