Ollama + MCP: Connect Local AI to Your Tools

Published April 23, 2026 · Updated August 2026 · 18 min read

Ollama does not speak MCP on its own — it is a model server, not an MCP client. You connect the two with a bridge: an MCP-aware orchestrator such as mcphost, Cline, Continue.dev or the LangChain MCP adapters acts as the client, calls Ollama for inference, and translates the model's tool calls into MCP call_tool invocations. Once that bridge is in place, every official MCP server — filesystem, GitHub, Postgres, Slack, fetch — works against a local model with nothing leaving your machine.

Model Context Protocol started as Anthropic's way to let Claude Desktop talk to your filesystem and GitHub. It has since become the de facto open standard for "this AI app needs to call external tools." Hundreds of MCP servers exist now — official ones for filesystem, GitHub, Postgres, Slack, Sentry, Puppeteer and Brave Search, plus a long tail of community servers for everything from Notion to Kubernetes. The piece most tutorials skip: you do not need Claude Desktop or a cloud LLM to use any of them.

New to the protocol itself? Start with our MCP servers explained primer, then come back here for the Ollama wiring. The configs below assume an Ollama new enough to expose tool calling in its API — that landed in Ollama 0.3.0, per the project's own release notes — and the current MCP SDKs. Check ollama -v and each server's README before copying anything, because the orchestrator layer moves faster than the protocol does.

How do you connect Ollama to an MCP server?

Install mcphost (a Go MCP client that speaks Ollama natively):

go install github.com/mark3labs/mcphost@latest
ollama pull qwen2.5:14b

Create ~/.mcp.json:

{
  "mcpServers": {
    "filesystem": {
      "command": "npx",
      "args": ["-y", "@modelcontextprotocol/server-filesystem", "/Users/you/Documents"]
    }
  }
}

Run:

mcphost -m ollama:qwen2.5:14b --config ~/.mcp.json

You now have an interactive session where the local model can read, write, and search files in ~/Documents through the official MCP filesystem server. Ask it "summarize the three most recent .md files in Documents" and watch it call list_directory, read_file three times, then generate. Zero data leaves your machine.

That is the demo. The rest of this article is the engineering: which models work, how to chain multiple servers, building your own MCP server for Ollama, and the production deployment story.

---

What is MCP, and why does it matter for Ollama?

MCP (Model Context Protocol) is JSON-RPC 2.0 over stdio or SSE, with a typed schema for three primitives:

The wire protocol is uniform. A server says "here are the tools I expose, here are their JSON schemas, here are the resources I can serve." A client says "list_tools," receives the manifest, surfaces tools to the model, executes call_tool when the model decides to use one, and feeds results back into the conversation.

The win is composability. Write one filesystem MCP server, every MCP-compatible client (Claude Desktop, Cursor, Continue.dev, mcphost, Cline, Zed, Goose) gets it. The model can be Claude, GPT-4, or your local llama3.1 — the server does not care.

For Ollama specifically, MCP solves the "tool ecosystem fragmentation" problem. Without it, every framework (LangChain, LlamaIndex, Continue, Cursor) ships its own tool definitions. With MCP, you write the tool once, every framework with an MCP client uses it. Anthropic's official MCP documentation is the authoritative spec.

---

Which MCP client should you use with Ollama?

Ollama is a model server, not an MCP client. To use MCP with Ollama, you need a client that speaks both. The maintained options:

Picks by use case:

• Trying it for the first time → mcphost (single binary, no project scaffolding)
• Coding tasks → Continue.dev or Cline in VS Code
• Building a custom agent app → LangChain MCP adapters
• Multi-user web UI for a team → Open WebUI with the MCP plugin
• No-code workflows → n8n with the MCP node (still beta but improving)

---

How do you set up mcphost with Ollama?

mcphost is the cleanest way to start. Single Go binary, native Ollama support, stdio MCP transport.

# Install
go install github.com/mark3labs/mcphost@latest

# Or download a release binary if you don't have Go
curl -L https://github.com/mark3labs/mcphost/releases/latest/download/mcphost_Linux_x86_64.tar.gz | tar xz
sudo mv mcphost /usr/local/bin/

# Verify
mcphost --version

Configure servers in ~/.mcp.json:

{
  "mcpServers": {
    "filesystem": {
      "command": "npx",
      "args": ["-y", "@modelcontextprotocol/server-filesystem", "/Users/you/Projects"]
    },
    "github": {
      "command": "npx",
      "args": ["-y", "@modelcontextprotocol/server-github"],
      "env": {
        "GITHUB_PERSONAL_ACCESS_TOKEN": "ghp_xxx"
      }
    },
    "postgres": {
      "command": "npx",
      "args": ["-y", "@modelcontextprotocol/server-postgres", "postgresql://user:pass@localhost/mydb"]
    },
    "fetch": {
      "command": "uvx",
      "args": ["mcp-server-fetch"]
    }
  }
}

Run with a specific Ollama model:

mcphost -m ollama:qwen2.5:14b --config ~/.mcp.json

You drop into an interactive REPL. Ask "What functions are defined in src/api/auth.py?" and the model:

1. Calls list_directory(/Users/you/Projects) → gets src/
2. Calls list_directory(/Users/you/Projects/src/api) → finds auth.py
3. Calls read_file(/Users/you/Projects/src/api/auth.py) → gets contents
4. Generates a summary

All four steps happen automatically. mcphost surfaces each tool call so you can see the agent's reasoning trail.

For one-shot non-interactive use:

echo "List the files in my Documents folder and tell me which one was last modified" | \
  mcphost -m ollama:qwen2.5:14b --config ~/.mcp.json --no-interactive

This is the right shape for cron jobs, CI tasks, and shell pipelines.

---

How do you use MCP servers inside a LangChain agent?

For programmatic agents in Python, the langchain-mcp-adapters package wires MCP servers into LangChain tools that any ChatModel — including ChatOllama — can call.

pip install langchain-mcp-adapters langchain-ollama langgraph

import asyncio
from langchain_mcp_adapters.client import MultiServerMCPClient
from langgraph.prebuilt import create_react_agent
from langchain_ollama import ChatOllama

async def main():
    async with MultiServerMCPClient({
        "filesystem": {
            "command": "npx",
            "args": ["-y", "@modelcontextprotocol/server-filesystem", "/Users/you/Projects"],
            "transport": "stdio",
        },
        "fetch": {
            "command": "uvx",
            "args": ["mcp-server-fetch"],
            "transport": "stdio",
        },
    }) as client:
        # Pull all tools from all servers into LangChain Tool objects
        tools = client.get_tools()
        print(f"Loaded {len(tools)} tools across MCP servers")

        llm = ChatOllama(model="qwen2.5:14b", temperature=0)
        agent = create_react_agent(llm, tools)

        result = await agent.ainvoke({
            "messages": [
                ("user", "Fetch https://localaimaster.com and summarize the homepage in 3 bullets, "
                         "then save the summary to /Users/you/Projects/summary.txt")
            ]
        })

        for m in result["messages"]:
            print(f"[{type(m).__name__}] {m.content[:200] if hasattr(m, 'content') else m}")

asyncio.run(main())

This is the same agent loop pattern from our Ollama + LangChain integration guide — only the tools come from MCP servers instead of being hand-written. You get to use the entire MCP server ecosystem from any LangChain agent.

For production, use LangGraph's StateGraph instead of create_react_agent so you have checkpointing, human-in-the-loop, and proper error recovery.

---

What breaks when you connect multiple MCP servers?

A real workflow uses several servers at once. Here is the shape of an "ops assistant" config — given a Slack message it can investigate Postgres, fetch docs, and post results back:

{
  "mcpServers": {
    "filesystem": {
      "command": "npx",
      "args": ["-y", "@modelcontextprotocol/server-filesystem", "/var/runbooks"]
    },
    "postgres-prod-readonly": {
      "command": "npx",
      "args": ["-y", "@modelcontextprotocol/server-postgres",
               "postgresql://readonly:pass@db.internal/prod"]
    },
    "github": {
      "command": "npx",
      "args": ["-y", "@modelcontextprotocol/server-github"],
      "env": { "GITHUB_PERSONAL_ACCESS_TOKEN": "ghp_xxx" }
    },
    "slack": {
      "command": "npx",
      "args": ["-y", "@modelcontextprotocol/server-slack"],
      "env": {
        "SLACK_BOT_TOKEN": "xoxb-xxx",
        "SLACK_TEAM_ID": "T01ABCDEF"
      }
    },
    "fetch": {
      "command": "uvx",
      "args": ["mcp-server-fetch"]
    },
    "sequential-thinking": {
      "command": "npx",
      "args": ["-y", "@modelcontextprotocol/server-sequential-thinking"]
    }
  }
}

When you wire 6 servers into one agent, watch out for:

1. Tool name collisions. Two servers exposing search confuses the model. mcphost prefixes with server name (filesystem.search, github.search); LangChain MCP adapters do too. If you write a custom client, namespace tools.

2. Total tool count. Every tool definition is injected into the prompt as JSON schema. Six servers can easily mean 60 tools, and that is thousands of tokens of schema before the user's question is even read — context you paid for, and a longer list for the model to disambiguate against. Cap at roughly 20 active tools per agent and split the rest into separate agents.

You can measure this cost yourself rather than trusting a number: run ollama show --modelfile to confirm the template, then send one request with tools attached and one without, and compare prompt_eval_count in the response body. That difference is your schema overhead per turn.

3. Permission scope. A model with write access to filesystem, GitHub, and Slack can do real damage. Run sensitive servers as separate processes with their own credentials, and consider a confirm-before-call wrapper for destructive operations.

4. Sequential-thinking server. This community server gives the model an explicit "let me think" tool — it externalizes chain-of-thought as a callable step instead of relying on the model to plan silently. Worth trying with smaller models, which is exactly where implicit multi-step planning is weakest.

---

How do you write a custom MCP server?

When the existing servers do not cover your tools, write one. Here is a minimal Python MCP server that exposes a "search internal wiki" tool:

pip install mcp

# wiki_mcp_server.py
from mcp.server import Server
from mcp.server.stdio import stdio_server
from mcp.types import Tool, TextContent
import asyncio

server = Server("internal-wiki")

@server.list_tools()
async def list_tools():
    return [
        Tool(
            name="search_wiki",
            description="Search the internal company wiki for a query. Returns top 5 matching pages.",
            inputSchema={
                "type": "object",
                "properties": {
                    "query": {"type": "string", "description": "Search terms"},
                    "limit": {"type": "integer", "default": 5},
                },
                "required": ["query"],
            },
        ),
        Tool(
            name="get_wiki_page",
            description="Fetch the full content of a wiki page by ID.",
            inputSchema={
                "type": "object",
                "properties": {"page_id": {"type": "string"}},
                "required": ["page_id"],
            },
        ),
    ]

@server.call_tool()
async def call_tool(name, arguments):
    if name == "search_wiki":
        # Replace with your real search backend
        results = await search_backend(arguments["query"], arguments.get("limit", 5))
        return [TextContent(type="text", text=str(results))]
    elif name == "get_wiki_page":
        page = await fetch_page(arguments["page_id"])
        return [TextContent(type="text", text=page)]

async def search_backend(query, limit):
    # Mock implementation
    return [{"id": f"page-{i}", "title": f"Result {i} for {query}"} for i in range(limit)]

async def fetch_page(page_id):
    return f"Mock content for {page_id}"

async def main():
    async with stdio_server() as (read_stream, write_stream):
        await server.run(read_stream, write_stream, server.create_initialization_options())

if __name__ == "__main__":
    asyncio.run(main())

Add to your MCP config:

{
  "mcpServers": {
    "wiki": {
      "command": "python",
      "args": ["/path/to/wiki_mcp_server.py"]
    }
  }
}

Done. mcphost or any LangChain MCP client now sees search_wiki and get_wiki_page as tools the model can call.

For TypeScript, the @modelcontextprotocol/sdk is the official package and follows the same pattern.

The biggest mistake people make writing custom MCP servers: vague tool descriptions. The model picks tools based on the description string. "Searches the wiki" is bad. "Search the internal company wiki for a query. Returns top 5 matching pages with title and ID. Use this when the user asks about company-specific knowledge, projects, processes, or onboarding documents." is good. Spend time on descriptions.

---

Which Ollama models can actually call MCP tools?

Two hard requirements, and only the first is negotiable.

1. The model must ship a tool-calling chat template. This is a published property, not an opinion: Ollama's model library tags every model that supports it with a tools capability, visible on the model page and in ollama show <model>. If the capability is absent, the model will happily reply in prose while ignoring the tool schema entirely — no error, just a useless answer. Check before you wire anything:

ollama show qwen2.5:14b | grep -i capabilities

Ollama publishes the current list itself — ollama.com/search?c=tools filters the library down to tags that carry the capability, which is the only version of this list that will not go stale. At the time of writing it includes llama3.1, llama3.2, qwen2.5 (and the qwen2.5-coder variants), mistral-nemo, mistral-small, command-r and firefunction-v2. Base gemma2 and phi3:mini tags do not appear — that alone rules them out for MCP work, regardless of how capable they are at chat.

2. The model has to fit in memory with the tool schemas loaded. Weights are the easy part to size. At Q4_K_M the quantized footprint is roughly 0.6 GB per billion parameters, and MCP adds context pressure on top — each connected server's tool definitions live in the prompt on every single turn.

Budget another 1-3 GB on top for the KV cache once you are running six servers' worth of schemas through an 8K context.

How to rank them for your own workload. Tool-selection ability is workload-specific, and generic leaderboards will mislead you if your tools are unusual. Two things worth doing instead of trusting anyone's number:

• Check the Berkeley Function-Calling Leaderboard, which scores open models on function-calling accuracy under a published, reproducible methodology. It is the closest thing to a neutral reference for this capability.
• Then run your own twenty prompts through your own server config. Ten minutes of that beats any published table, because it measures the tools you actually exposed and the descriptions you actually wrote.

Our Ollama tool calling guide covers the native function-calling API underneath all of this, and best Ollama models for tool calling tracks which tags currently carry the capability.

---

Which MCP servers are worth installing?

A curated list from the official catalog and community, with notes on what actually works well with Ollama:

For a private AI knowledge stack (the workflow most teams actually want), the magic combo is filesystem + memory + sequential-thinking + a custom RAG MCP server. Pair with our local RAG setup guide for the embedding side.

---

How do you run MCP servers in production?

When MCP graduates from "I tried it" to "the team depends on it," a few patterns matter.

1. Run MCP servers as systemd services

Long-running stdio servers are fine for desktop use but unstable for shared deployments. Wrap them as services:

# /etc/systemd/system/mcp-filesystem.service
[Unit]
Description=MCP Filesystem Server
After=network.target

[Service]
Type=simple
User=mcp
ExecStart=/usr/bin/npx -y @modelcontextprotocol/server-filesystem /var/data/shared
Restart=on-failure
RestartSec=5

[Install]
WantedBy=multi-user.target

2. Use SSE transport for remote servers

stdio works locally. For multi-host deployments, switch to Server-Sent Events:

{
  "mcpServers": {
    "remote-postgres": {
      "url": "https://mcp-postgres.internal/sse",
      "transport": "sse",
      "headers": {
        "Authorization": "Bearer $MCP_AUTH_TOKEN"
      }
    }
  }
}

3. Audit logging

Every tool call is a security-relevant event. Log them:

@server.call_tool()
async def call_tool(name, arguments):
    logger.info("mcp_call", extra={
        "tool": name,
        "args": arguments,
        "user": get_current_user(),  # if you wrap auth
        "ts": time.time(),
    })
    # ...actual implementation

This is the foundation for the local AI audit trail story — every prompt and every tool invocation captured for compliance review.

4. Sandboxing destructive tools

Filesystem and database write tools should run in restricted environments. The pattern that holds up: run the writeable filesystem server in a chroot or container with only its target directory bind-mounted, give the GitHub server a fine-grained token scoped to read, and put a human approval step in front of anything that mutates state — merging a PR, dropping a table, posting to a public channel.

5. Cost and rate accounting per tool

If your MCP servers hit paid APIs (Brave Search, OpenAI for embeddings, etc.), wrap them in a metering layer. Per-tool, per-user counters. Trip a circuit breaker if a runaway agent starts hammering search.

For the broader operational picture, our Ollama production deployment covers the model-server side, and pairing MCP with Ollama load balancing gives you horizontal scale.

---

What goes wrong with Ollama and MCP?

1. Using a very small model for multi-tool work. Tool selection gets less reliable as models shrink, and it degrades fastest on the ambiguous cases where two tools both look plausible. Start at qwen2.5:7b or llama3.1:8b and only drop to a 3B once you have confirmed your specific tool set works there.

2. Vague tool descriptions in custom servers. The model picks tools by description text. "Search docs" is wrong; describe inputs, outputs, and when to use it.

3. Loading too many tools at once. Selection accuracy falls as the tool list grows, and it falls fastest on small models — every extra tool is one more near-miss for the model to disambiguate against. Two dozen active tools is a reasonable working ceiling, but treat that as a starting point and find your own: add servers one at a time and re-run the same twenty prompts after each one. Group by use case and load only what a given agent needs.

4. Forgetting OLLAMA_KEEP_ALIVE. The first MCP call after an idle period cold-loads the model from disk, and the pause scales with the model's size and your storage speed. Set OLLAMA_KEEP_ALIVE long enough that an interactive session never pays it twice.

5. Mixing tool-capable and non-tool-capable models. Models without a tool-calling template silently ignore tool schemas — Ollama returns prose instead of tool_calls and nothing errors. Run ollama show <model> and confirm the tools capability before you debug anything else.

6. stdio MCP servers in containers. They depend on stdin/stdout pipes. Many container setups close stdin. Use SSE transport for containerized deployments.

7. Unbounded filesystem scope. Granting MCP filesystem access to / is a foot-gun. Always restrict to specific paths, ideally with read-only mounts where possible.

8. No timeout on tool calls. A slow Postgres query hangs the agent forever. Wrap MCP tool calls with timeouts at the orchestrator layer.

9. Trusting model-generated SQL. The Postgres MCP server runs whatever query the model generates. Always use a read-only DB role or a query allowlist for production.

10. Skipping the official MCP playground. Anthropic's MCP inspector is the fastest way to debug what tools a server exposes. Use it before wiring anything to Ollama.

---

FAQs

Does Ollama natively speak MCP?

No. MCP is a client-server protocol and Ollama is a model server, not an MCP client. The working pattern is an MCP-aware orchestrator — mcphost, mcp-cli, Cline, Continue.dev, or your own LangChain setup — acting as the client: it holds the MCP connections, calls Ollama for inference, and translates between Ollama's tool_calls format and MCP call_tool invocations.

Can I use the same MCP servers Claude Desktop uses?

Yes, with one caveat. The wire protocol is identical, so filesystem, github, postgres, sequential-thinking, time, fetch, sqlite, memory and puppeteer all work against an Ollama-backed client. The caveat is prompt tuning: some servers ship tool descriptions written with a frontier model in mind. Local models often need those descriptions tightened at the orchestrator layer. The tools themselves work unchanged.

Which Ollama models work with MCP tools?

Any model whose Ollama tag carries the tools capability — llama3.1, llama3.2, the qwen2.5 family, mistral-nemo, mistral-small, command-r and firefunction-v2 among others. Confirm with ollama show <model> rather than guessing; models without the capability ignore tool schemas silently. Size matters separately: sub-7B models handle single-tool prompts far better than ambiguous multi-tool ones.

Can I run MCP servers locally for full data privacy?

Yes, and it is the main reason to pair MCP with Ollama rather than a hosted model. The official filesystem, github, postgres and sqlite servers all run as local Node or Python processes. With Ollama as the backend, no prompt and no tool result leaves your machine — which is what makes agentic workflows possible on data you are contractually not allowed to send anywhere.

What is the latency overhead of MCP versus native tool calling?

The model-side cost is identical — MCP does not change how the model generates a tool call. The extra cost is transport: a stdio server is a local process talking over pipes, which is fast; SSE adds a network round trip per invocation, so co-locate servers with the client where you can. If you want the real figure for your own setup, time a call_tool round trip directly against the server with the MCP inspector — no LLM in the loop — and compare that to the same call through your agent.

What is mcphost and should I use it?

mcphost is a CLI MCP client written in Go with native Ollama support. It is the shortest path from nothing to a working agent: one binary, one JSON config, no project scaffolding. For production services or a web UI you will outgrow it and move to LangChain/LangGraph or a custom client — but it is the right thing to try first.

How do I write a custom MCP server?

Use the official @modelcontextprotocol/sdk (TypeScript) or the mcp Python package. Define resources (read-only data), tools (callable functions) and prompts (templates), then implement the stdio or SSE transport. Because the protocol is client-agnostic, a server that works in Claude Desktop works in any Ollama-MCP bridge unchanged.

---

Conclusion

MCP is the closest thing the local AI world has to USB for tools. Write a server once, use it from any client, swap LLMs without rewriting your tool layer. Pair it with Ollama and you have a fully private agentic AI stack — your model, your tools, your data, your hardware.

The honest state of the integration: smaller models still struggle with multi-tool reasoning, and the orchestrator layer moves fast enough that some configs break between releases. For greenfield projects that is fine. For anything the team depends on, pin versions and expect to spend an afternoon debugging when you upgrade.

Start with mcphost and the filesystem server. Wire in fetch and sequential-thinking. Once you trust the workflow, write a custom MCP server for whatever your team actually does — internal API, knowledge base, ticketing system. If you would rather do the same thing without Ollama in the middle, llama.cpp has its own MCP server story.

---

Want the next deep dives — production-grade MCP server templates, agent evaluation harnesses, multi-tenant MCP gateways? Subscribe to the Local AI Master newsletter. Weekly playbooks for builders.