Ollama Tool Calling: The Practical Function Calling Guide

Published April 23, 2026 • Updated August 23, 2026 • 20 min read

Short answer: Ollama tool calling — what the OpenAI world calls function calling — lets a local model invoke your code. You send a tools array of JSON schemas with the chat request; a tool-capable model replies with a structured tool_calls object naming a function and its arguments instead of prose; your code runs it and feeds the result back for the final answer. Pull qwen3:8b if you want one model that works, keep the tool list short, and spend your effort on the tool descriptions — that is where accuracy actually comes from.

That loop is the whole difference between a chatbot and an agent. Ollama added native tool support in version 0.3.0 (July 2024) and it has matured across dozens of releases since.

The catch: tool calling is the area where local LLMs are most uneven. Some models handle it cleanly. Some carry the capability flag but produce arguments that fail schema validation. Some degrade sharply once the tool list grows. The official docs do not warn you about any of this.

This guide covers which models expose a real tools API and how to check, how to write schemas a model will follow, the multi-tool agent loop with error handling, and the LangChain, CrewAI and MCP wiring on top of it.

On numbers in this guide: model download sizes are the figures published in the Ollama library, and where a page cites an accuracy score it names the third-party benchmark it came from. Anything that would need a specific machine to establish — latency, tokens per second — this guide gives you the harness to measure yourself instead of quoting someone else's laptop.

---

Quick Start: First Tool Call in 90 Seconds

# Install Ollama and pull a model that handles tools well
ollama pull llama3.1:8b

# tools_minimal.py
import ollama
import json

def get_weather(city: str) -> str:
    # Stub: in real life, hit a weather API
    return json.dumps({"city": city, "temp_c": 22, "condition": "sunny"})

tools = [{
    "type": "function",
    "function": {
        "name": "get_weather",
        "description": "Get current weather for a city",
        "parameters": {
            "type": "object",
            "properties": {
                "city": {"type": "string", "description": "City name"},
            },
            "required": ["city"],
        },
    },
}]

messages = [{"role": "user", "content": "What is the weather in Paris?"}]
res = ollama.chat(model="llama3.1:8b", messages=messages, tools=tools)

if res["message"].get("tool_calls"):
    for call in res["message"]["tool_calls"]:
        result = get_weather(**call["function"]["arguments"])
        messages.append(res["message"])
        messages.append({"role": "tool", "content": result, "name": call["function"]["name"]})
    final = ollama.chat(model="llama3.1:8b", messages=messages, tools=tools)
    print(final["message"]["content"])
else:
    print(res["message"]["content"])

Run it:

pip install ollama
python tools_minimal.py
# > "It is currently 22°C and sunny in Paris."

That is the entire shape of tool calling: model decides to invoke a tool, you execute it, you append the result, the model uses the result to write the final answer.

---

Which models actually support tools?

Start with the part that is a fact rather than an opinion: Ollama only exposes a real tools API for models whose chat template defines one. Every model page in the Ollama library carries a capability badge, and the library is filterable by tools. If the badge is absent, passing a tools array does nothing useful — the model will answer in prose and you will spend an afternoon debugging your JSON for no reason. Check the badge first, always.

That check is binary and verifiable. Everything past it — how well a model uses tools — is where you need a named benchmark rather than a blog's say-so.

The reference table

Download sizes below are the figures published on each model's Ollama library page. You can sanity-check any of them without downloading anything, because Q4_K_M quantization lands at roughly 0.6 GB per billion parameters: an 8B model is about 4.8 GB, a 14B about 8.4 GB, a 70B about 42 GB. If a number you see anywhere is wildly off that line, it is quoting a different quant.

Where to get real accuracy numbers

The benchmark the field actually uses for this is the Berkeley Function Calling Leaderboard (BFCL) from UC Berkeley's Gorilla project. It scores models on simple calls, parallel calls, multiple-function selection, and — importantly — relevance detection, meaning whether the model correctly declines to call a tool when none applies. That last category is the one that separates models that look fine in a demo from models that work in production, and it is exactly what a hand-rolled ten-question test would miss.

BFCL is re-run as models ship, so read the current board rather than any snapshot. Filter it to the open-weight models in the size class you can actually run.

For a fuller comparison of these model families, see our best Ollama models guide. For coding-specific tool work, best local AI models for programming goes deeper.

Updated picks: Qwen 3, Hermes 4, and the current agent models

If you are starting a new agent today, the default to pull is qwen3:8b — Apache 2.0, native tool support, and about 5 GB at Q4_K_M (8B x 0.6 GB/B), which fits an 8 GB card with room for context. The table above still describes the Llama 3.1 / Qwen 2.5 generation accurately, but the current-generation options are these. Sizes are computed from the 0.6 GB-per-billion rule above, so treat them as the ballpark you need free, not exact download bytes:

• qwen3:8b — the all-round pick for 8 GB cards and 16 GB Macs. Hybrid thinking mode (/think / /no_think) lets you trade latency for deeper planning per step. ~5 GB at Q4.
• qwen3:30b-a3b — a 30B Mixture-of-Experts with only ~3B active params: reasons like a big model, runs close to a small one. ~18 GB at Q4, which is why it suits a 24 GB card. Note the MoE catch: you need VRAM for all 30B of weights, but you only pay compute for the 3B active — memory-heavy, compute-light.
• llama3-groq-tool-use:8b — the function-calling specialist (~5 GB, 8K context). Groq fine-tuned it purely for tool use and reported 89.06% on the Berkeley Function Calling Leaderboard for the 8B model at launch. Narrow — it is a tool router, not a conversationalist — but that is exactly the job here.
• Hermes 4 14B — NousResearch's reasoning-plus-tools model, built on Qwen3-14B. It emits tool calls inside <tool_call> tags after a visible reasoning step, which makes calls easy to parse and debug. ~8-9 GB at Q4. No first-party entry in the official Ollama library at the time of writing — import the GGUF from the NousResearch repo or use a vetted community upload. Our Hermes agent setup guide covers it end to end.
• gemma4:31b — Gemma 4's 31B dense variant ships with native function-calling and structured JSON output. ~19 GB at Q4. Pull the size explicitly: the bare gemma4 tag defaults to the small E4B edge model.
• mistral-small3.2:24b — tuned for low-latency function calls (~14 GB at Q4), but its tool parser has had known teething issues in some Ollama builds. Confirm tool calls actually parse on your version before shipping it.

For the full ranked breakdown by VRAM tier — including which model to pair with CrewAI, LangGraph, or Continue — see best Ollama models for AI agents and the reliability-focused best local LLMs for tool calling.

---

How Ollama Function Calling Actually Works

Ollama implements an OpenAI-compatible tool calling API. The flow is:

1. You send: messages + tools (JSON schemas)
2. Model returns either:
   a. A normal text message (no tool needed), or
   b. A "tool_calls" list with name + arguments
3. You execute each tool call locally
4. You append the tool result as a "tool" role message
5. You call the model again with the updated messages
6. Model returns the final natural-language answer

Ollama parses the model's structured output into the OpenAI tool-calls format under the hood. This works because Llama 3.1+, Qwen 2.5+, Qwen 3, Mistral, and similar models were post-trained on tool-calling data with consistent special tokens or JSON schemas.

Worth being explicit about, because it trips people up: the model never executes code or accesses the internet directly. It only decides which tool to call and what arguments to pass. Your code handles all execution — which is also why tool calling is safe to experiment with.

The other important consequence: the model decides whether to call a tool. If it thinks the question is conversational ("hello, who are you"), it will not invoke a tool even if one is available. This is correct behavior — but if your application requires structured output 100% of the time, enforce it at the application layer (see the patterns section below).

---

Step 1: Define Tools With Good Schemas

Tool schemas use JSON Schema. The quality of your schema directly determines the model's accuracy. Two principles:

1. Description is everything. The model picks tools and arguments based on the descriptions, not the names.
2. Be strict. Specify required fields, enum values, and exact types. Loose schemas → loose calls.

A well-defined tool:

search_tool = {
    "type": "function",
    "function": {
        "name": "search_internal_docs",
        "description": (
            "Search the company's internal documentation for relevant content. "
            "Use this when the user asks about company policies, procedures, "
            "engineering wikis, or internal codebases. Do not use for public knowledge."
        ),
        "parameters": {
            "type": "object",
            "properties": {
                "query": {
                    "type": "string",
                    "description": "Search keywords (3-8 words). Use specific technical terms.",
                },
                "department": {
                    "type": "string",
                    "enum": ["engineering", "hr", "security", "finance", "all"],
                    "description": "Filter by department; use 'all' if unknown.",
                },
                "max_results": {
                    "type": "integer",
                    "description": "Number of results to return (1-10).",
                    "default": 5,
                },
            },
            "required": ["query", "department"],
        },
    },
}

A bad version of the same tool:

{
    "type": "function",
    "function": {
        "name": "search",
        "description": "Search docs",
        "parameters": {
            "type": "object",
            "properties": {
                "q": {"type": "string"},
            },
        },
    },
}

The bad version will fire on every question, ignore the department filter entirely because nothing tells the model it exists, and pass whatever the user typed as the query. Description quality is the single biggest lever you control — bigger than model size, and free.

Two schema guardrails worth copying

The description field is also where your safety policy gets stated. Two patterns worth copying into every agent — a database tool that declares itself read-only, and a side-effect tool that requires explicit user intent:

{
    "type": "function",
    "function": {
        "name": "query_database",
        "description": "Run a read-only SQL query against the application database. Only SELECT queries are allowed.",
        "parameters": {
            "type": "object",
            "properties": {
                "sql": {"type": "string", "description": "SQL SELECT query"},
                "limit": {"type": "integer", "description": "Max rows to return (default 10)"}
            },
            "required": ["sql"]
        }
    }
}

{
    "type": "function",
    "function": {
        "name": "send_email",
        "description": "Send an email. Use only when the user explicitly asks to send an email.",
        "parameters": {
            "type": "object",
            "properties": {
                "to": {"type": "string", "description": "Recipient email address"},
                "subject": {"type": "string", "description": "Email subject line"},
                "body": {"type": "string", "description": "Email body text"}
            },
            "required": ["to", "subject", "body"]
        }
    }
}

The description steers the model, but never rely on it alone: enforce the SELECT-only rule and the "explicitly asks" rule in your dispatcher code too. The model proposes; your code disposes.

---

Step 2: Multi-Tool Agent Pattern

Real applications expose multiple tools. The agent loop must handle: zero tools called, one tool, multiple tools in one turn, and chained tools across turns.

# agent.py
import ollama
import json

# --- Tool implementations ---
def get_weather(city: str) -> str:
    return json.dumps({"city": city, "temp_c": 22, "condition": "sunny"})

def search_news(query: str, limit: int = 3) -> str:
    return json.dumps([
        {"title": f"Result about {query}", "url": "https://example.com/1"}
    ])

def calculate(expression: str) -> str:
    try:
        return json.dumps({"result": eval(expression, {"__builtins__": {}}, {})})
    except Exception as e:
        return json.dumps({"error": str(e)})

TOOL_REGISTRY = {
    "get_weather": get_weather,
    "search_news": search_news,
    "calculate": calculate,
}

TOOLS_SCHEMA = [
    {"type": "function", "function": {
        "name": "get_weather",
        "description": "Get current weather for a city.",
        "parameters": {
            "type": "object",
            "properties": {"city": {"type": "string", "description": "City name."}},
            "required": ["city"],
        },
    }},
    {"type": "function", "function": {
        "name": "search_news",
        "description": "Search recent news headlines.",
        "parameters": {
            "type": "object",
            "properties": {
                "query": {"type": "string", "description": "Search keywords."},
                "limit": {"type": "integer", "description": "Number of results.", "default": 3},
            },
            "required": ["query"],
        },
    }},
    {"type": "function", "function": {
        "name": "calculate",
        "description": "Evaluate a math expression. No variables.",
        "parameters": {
            "type": "object",
            "properties": {"expression": {"type": "string"}},
            "required": ["expression"],
        },
    }},
]

# --- Agent loop ---
def run_agent(user_question: str, model="qwen2.5:7b", max_turns=6):
    messages = [
        {"role": "system", "content": (
            "You are a careful assistant. Use the provided tools when needed. "
            "Do not invent tool results. If a tool fails, explain what happened "
            "and try a different approach."
        )},
        {"role": "user", "content": user_question},
    ]

    for turn in range(max_turns):
        res = ollama.chat(model=model, messages=messages, tools=TOOLS_SCHEMA)
        msg = res["message"]
        messages.append(msg)

        tool_calls = msg.get("tool_calls") or []
        if not tool_calls:
            return msg["content"]

        for call in tool_calls:
            name = call["function"]["name"]
            args = call["function"]["arguments"]
            if name not in TOOL_REGISTRY:
                result = json.dumps({"error": f"unknown tool: {name}"})
            else:
                try:
                    result = TOOL_REGISTRY[name](**args)
                except TypeError as e:
                    result = json.dumps({"error": f"bad arguments: {e}"})
                except Exception as e:
                    result = json.dumps({"error": str(e)})
            messages.append({"role": "tool", "name": name, "content": result})

    return "Reached max turns without a final answer."

if __name__ == "__main__":
    print(run_agent("What is the weather in Tokyo, and what is 17 * 23?"))

Key patterns to copy:

1. TOOL_REGISTRY dispatch: maps tool names to Python callables.
2. Bounded loop: max_turns prevents runaway loops if the model keeps calling tools.
3. Error wrapping: every tool call is wrapped in try/except and returns JSON, so the model can recover gracefully.
4. System prompt: enforces grounded behavior without inventing tool results.

This is the whole agent. Frameworks add memory, delegation and parallelism on top, but the loop underneath is always this shape — which is why it is worth writing once by hand before you reach for one.

---

Step 3: Tool Use From JavaScript / TypeScript

For Node and browser apps, the official ollama JS package exposes the same API.

// agent.ts
import ollama from "ollama";

const tools = [
  {
    type: "function",
    function: {
      name: "get_weather",
      description: "Get current weather for a city.",
      parameters: {
        type: "object",
        properties: { city: { type: "string", description: "City name." } },
        required: ["city"],
      },
    },
  },
];

const TOOLS: Record<string, (args: any) => Promise<string>> = {
  get_weather: async ({ city }) =>
    JSON.stringify({ city, temp_c: 22, condition: "sunny" }),
};

async function runAgent(question: string) {
  const messages: any[] = [{ role: "user", content: question }];

  for (let turn = 0; turn < 6; turn++) {
    const res = await ollama.chat({
      model: "qwen2.5:7b",
      messages,
      tools,
    });
    messages.push(res.message);

    const calls = res.message.tool_calls ?? [];
    if (calls.length === 0) return res.message.content;

    for (const c of calls) {
      const fn = TOOLS[c.function.name];
      const result = fn ? await fn(c.function.arguments) : "{}";
      messages.push({ role: "tool", name: c.function.name, content: result });
    }
  }
  return "Hit max turns.";
}

runAgent("Weather in Paris?").then(console.log);

For full Node/Next.js patterns including streaming and the Vercel AI SDK, see our companion guide on Ollama with JavaScript and TypeScript.

---

Step 4: Common Patterns

Pattern 1: Forcing a tool call. Ollama's native API leaves the call/no-call decision to the model — there is no reliable server-side switch to force one. When a turn absolutely must produce a tool call, enforce it at the application layer: tell the model in the system prompt ("For this request you MUST call one of the provided tools — do not answer in prose"), then validate the response and re-prompt once if it answered in text anyway. If you need a hard guarantee of structured output and do not need tool execution, skip tools and use JSON mode instead:

ollama.chat(
    model="llama3.1:8b",
    messages=[{"role": "user", "content": "Extract entities from: ..."}],
    format="json",
)

Pattern 2: Structured output without tools. Same format="json" mode — use it when you want JSON you parse yourself rather than functions the model invokes. Extraction, classification, and form-filling all belong here.

Pattern 3: Tool result chaining. When tool A's output feeds tool B, structure tool descriptions to encourage the chain:

"Use search_news first to find article URLs, then summarize_article on each URL."

The model handles the orchestration if your descriptions are explicit.

Pattern 4: Cost-effective routing. Use a small model (qwen2.5:7b or qwen3:8b) for tool selection, hand off to a larger model for the final synthesis. Saves significant time on multi-step agents.

---

Step 5: Error Handling and Retry

Tool calls fail. Networks drop. APIs return weird JSON. The agent must survive.

def safe_call_tool(tool_fn, args, retries=2):
    last_error = None
    for attempt in range(retries + 1):
        try:
            return tool_fn(**args)
        except Exception as e:
            last_error = str(e)
            if attempt < retries:
                continue
            return json.dumps({"error": f"tool failed after {retries+1} attempts: {last_error}"})

Three failures to plan for:

1. Bad arguments from the model. The model passes a string where you wanted an int. Wrap in try/except and return a structured error so the model can retry.
2. Tool downtime. External APIs return 500s. Always set a timeout and return an error JSON.
3. Hallucinated tool names. The model sometimes invents tool names that do not exist. Catch this in dispatch and return a list of valid tool names to help the model recover.

A robust dispatcher:

def dispatch(name: str, args: dict) -> str:
    if name not in TOOL_REGISTRY:
        return json.dumps({
            "error": f"unknown tool: {name}",
            "available_tools": list(TOOL_REGISTRY.keys()),
        })
    return safe_call_tool(TOOL_REGISTRY[name], args)

The agent recovers gracefully because it sees the available tools and corrects on the next turn.

---

Step 6: Streaming With Tool Calls

Tool calls and streaming have a tricky interaction. In older Ollama builds the tool-call payload arrived only at the end of the stream; newer versions can stream tool calls incrementally as well (see Ollama's own streaming tool calls announcement). The robust pattern handles both — accumulate text as it arrives, and check for tool calls on every chunk as well as at the end:

stream = ollama.chat(
    model="qwen2.5:7b",
    messages=messages,
    tools=tools,
    stream=True,
)

text_parts = []
final_message = None
for chunk in stream:
    msg = chunk.get("message", {})
    if msg.get("content"):
        text_parts.append(msg["content"])
        print(msg["content"], end="", flush=True)
    if chunk.get("done"):
        final_message = msg

if final_message and final_message.get("tool_calls"):
    # process tool calls as usual
    ...

For text-only responses, streaming gives you token-by-token UI updates. For tool-driven responses, the user sees nothing until the tools resolve. To improve UX, render a "Calling search_internal_docs..." indicator the moment you see a tool call.

---

How fast will tool calls be, and how do I compare models?

Any latency figure published in a guide is a statement about somebody else's CPU, GPU and thermal headroom. What transfers is the arithmetic, and the harness to run it yourself.

Where the wall-clock time goes

An agent turn costs more than one model call. Break it down before you blame the model:

turn latency = prompt processing + generation + tool execution

prompt processing  = (system + history + every tool schema) / prefill speed
generation         = tokens emitted / decode speed
tool execution     = your code — network calls usually dominate here

Two consequences fall straight out of that:

1. Tool schemas are prompt tokens. Ten verbose schemas can add more to prefill than the user's question does, and you pay that cost on every turn of the loop, because the whole tool array is re-sent each time. This is the real reason to keep the tool list tight — not a mystical "model gets confused" effect.
2. A chained N-tool workflow is roughly N+1 model calls, not one. Each tool result appends to the history, so turn 4 processes a longer prompt than turn 1. Latency grows super-linearly with chain depth. If an agent feels slow, count the round trips before you swap the model.

A decode-speed ceiling you can compute for any local model:

max tokens/sec  ~=  memory bandwidth (GB/s) / model size in memory (GB)

That is an arithmetic upper bound — it assumes every weight is read exactly once per token and nothing else costs anything, so real output lands well below it. It is still useful as a sanity check: a 5 GB Q4 model on a machine with 100 GB/s of bandwidth cannot exceed ~20 tok/s no matter what else you tune, and if you are getting 3 tok/s the problem is that the model spilled out of VRAM, not your sampler settings.

Measure your own stack

#!/usr/bin/env python3
"""toolbench.py - time a tool-calling turn on YOUR machine."""
import time, statistics, ollama

MODEL = "qwen3:8b"
QUESTIONS = [
    "What is the weather in Paris?",
    "What is 17 * 23?",
    "Weather in Tokyo and what is 91 / 7?",
]

latencies, called = [], 0
for q in QUESTIONS:
    start = time.perf_counter()
    res = ollama.chat(
        model=MODEL,
        messages=[{"role": "user", "content": q}],
        tools=TOOLS_SCHEMA,          # reuse the schema from the agent above
    )
    latencies.append(time.perf_counter() - start)
    if res["message"].get("tool_calls"):
        called += 1

print(f"model={MODEL}")
print(f"median first-call latency: {statistics.median(latencies):.2f}s")
print(f"tool invoked on {called}/{len(QUESTIONS)} prompts")

Run it against two candidate models with your real schemas loaded. That tells you what a published table never can: how your tool descriptions behave, on your hardware.

For accuracy, use a real benchmark

Latency you measure locally; correctness you should not eyeball. The Berkeley Function Calling Leaderboard evaluates exactly the failure modes that matter — wrong function chosen, malformed arguments, parallel calls, and calling a tool when none was appropriate — across hundreds of cases per model. Read the current board, filter to models you can run, and use your local harness only to confirm the winner behaves on your schemas.

For the ranked, VRAM-tier-by-VRAM-tier version of this decision, see best local LLMs for tool calling.

---

Using Tool Calling With LangChain and CrewAI

You do not have to write the agent loop yourself: LangChain, CrewAI, and LangGraph all speak Ollama's tools API and run the call-execute-respond cycle for you. LangChain binds tools onto a ChatOllama instance; CrewAI takes functions decorated as tools and hands them to role-based agents. Use a framework when you need memory, delegation, or parallel tool execution — use the raw loop from this guide when you want zero dependencies and full control.

LangChain

from langchain_ollama import ChatOllama
from langchain_core.tools import tool

@tool
def get_weather(city: str) -> str:
    """Get the current weather for a city."""
    # Your implementation
    return f"Weather in {city}: 22°C, sunny"

llm = ChatOllama(model="llama3.1")
llm_with_tools = llm.bind_tools([get_weather])

result = llm_with_tools.invoke("What's the weather in Paris?")

bind_tools() converts your decorated functions into the same JSON schemas you saw earlier — everything about description quality still applies. Our Ollama + LangChain integration guide covers chains, memory, and streaming on top of this.

CrewAI

from crewai import Agent
from crewai.tools import tool

@tool("Search Tool")
def search(query: str) -> str:
    """Search the web for information."""
    # Your implementation
    return "search results..."

researcher = Agent(
    role="Researcher",
    goal="Find accurate information",
    tools=[search],
    llm="ollama/llama3.1"
)

CrewAI handles the multi-turn tool loop automatically — including sending results back and getting the final answer. Setup details are in our CrewAI local setup guide, and if you are choosing between frameworks, the AI agent frameworks comparison weighs CrewAI against LangGraph and AutoGen.

---

Ollama and MCP: Where Tool Calling Meets the Ecosystem

Ollama is a model server, not an MCP client — so "Ollama MCP" means putting a bridge in the middle. The bridge (mcphost is the simplest) connects to one or more MCP servers, asks each for its tool manifest, converts those tools into the same tools array you have used throughout this guide, and executes whatever calls the model makes. Every pattern on this page applies unchanged; MCP just standardizes where the tools come from, so one filesystem or GitHub server works with every MCP-capable client instead of being rewritten per framework.

The fastest way to see it working — the official filesystem MCP server driven by a local model:

go install github.com/mark3labs/mcphost@latest
ollama pull qwen2.5:14b

Create ~/.mcp.json:

{
  "mcpServers": {
    "filesystem": {
      "command": "npx",
      "args": ["-y", "@modelcontextprotocol/server-filesystem", "/Users/you/Documents"]
    }
  }
}

mcphost -m ollama:qwen2.5:14b --config ~/.mcp.json

Ask it to "summarize the three most recent .md files in Documents" and watch it call list_directory and read_file through the MCP server — zero data leaves your machine.

That is the demo; the engineering lives in our dedicated Ollama + MCP integration guide — chaining multiple servers, writing your own MCP server, and which models pick the right tool reliably. Running llama.cpp instead of Ollama? The llama.cpp MCP server guide covers that route.

---

Pitfalls and Gotchas

1. The model sometimes ignores tools and answers from training data. Solution: explicit system prompt — "If you do not have current information, you MUST call a tool. Do not answer from memory."

2. Argument types are inconsistent. A model may return "limit": "5" (string) when you specified integer. Coerce types in the dispatcher: int(args.get("limit", 5)).

3. Long tool descriptions cost you twice. They are re-sent on every turn of the loop, and they dilute the distinguishing detail the model actually routes on. Write them like a good function docstring — one sentence on what it does, one on when not to use it — and move standing context into the system prompt, where it is sent once.

4. Too many tools degrades routing. The model has to discriminate between an increasing number of similar-sounding descriptions, and every schema competes for context. A sound default is to keep a single agent under about eight tools; past that, group related capabilities behind one tool with an enum action parameter, or split into sub-agents with a router. Only ever send the tools relevant to the current task.

5. Models call tools redundantly. They sometimes call get_weather twice in a row for the same city. Add deduplication at the dispatcher: cache results per turn.

6. Local models lag cloud models on chained reasoning. A single tool call is the easy case. Long chains — where the model has to hold a plan across five or more dependent calls — are where open models still trail the frontier ones. Use a larger model, or decompose the workflow into short chains you orchestrate yourself.

7. Context grows every turn. Each turn appends the assistant message and every tool result to the history, and the full tool schema array is re-sent alongside it. A ten-turn loop with verbose JSON results can outgrow a small model's context window entirely, at which point the model silently loses the beginning of the conversation. Trim or summarize old tool results, and cap results at the size the model actually needs.

8. JSON mode is not tool calling. format="json" returns JSON in the content field but does not invoke tools. Different feature, different use case.

---

Production Hardening

For a production agent:

• Per-tool timeout (30s default, lower for fast tools)
• Bounded max_turns (4-8 for most agents)
• Structured error responses with retry hints
• Logging of every tool call and result (auditability)
• Rate limiting on expensive tools (web fetches, paid APIs)
• Schema validation of tool arguments before execution
• Dedup of identical consecutive tool calls
• Concurrent tool execution when safe (asyncio gather)
• Graceful fallback to text-only mode if tools repeatedly fail
• Unit tests for each tool and an integration test for the full loop

For broader production patterns including auth, monitoring, and multi-user concurrency, our Ollama production deployment guide covers the hosting layer. For knowledge-augmented agents, pair this with the Ollama + ChromaDB RAG pipeline.

---

Three agent shapes worth copying

Most working local agents fall into one of three patterns. Each one is a different answer to "how much reasoning does the model actually have to do?", which is what should drive your model choice.

1. Retrieval-and-lookup bot. Tools: search_docs, lookup_user, create_ticket. The model's whole job is picking the right lookup and formatting the answer — almost no chained reasoning. This is the pattern local models handle best, and a 7B-14B model is genuinely sufficient. Pair it with retrieval using the Ollama + ChromaDB RAG pipeline so the answers are grounded in your documents rather than the model's training data.

2. Scheduled batch agent. Tools: get_transactions, categorize, summarize. Runs on a cron, no human waiting, output goes to a file or an email. Latency is irrelevant here, which means you can afford a bigger model than you would tolerate interactively — this is the one place where a 70B on slow hardware is a perfectly sensible choice.

3. Research / multi-step agent. Tools: search, fetch, summarize, save. The hardest of the three, because it requires holding a plan across many dependent calls. This is where local models most visibly trail the frontier ones, and where you should either use the largest model you can fit or break the workflow into short, individually-orchestrated chains.

The common thread: the value is not raw model intelligence, it is the model acting as a careful router across a small set of well-defined tools. Routing is exactly what local models are good at, which is why the retrieval pattern works on a 7B and the research pattern struggles on a 70B.

For the framework-level version of these patterns, see best Ollama models for AI agents and the AI agent frameworks comparison.

---

FAQ

What is the difference between tool calling and function calling?

Nothing — they are two names for the same capability. "Function calling" was coined by OpenAI when the feature launched in GPT-3.5/4; "tool calling" is the broader term used by Anthropic, Meta, and Ollama. In both cases the model reads your function definitions, decides when to use one, and returns structured JSON with the function name and arguments. Ollama's API uses the tools parameter.

Which Ollama model is best for tool calling?

Start with qwen3:8b on 8-16 GB machines, qwen3:30b-a3b on a 24 GB card, and llama3-groq-tool-use:8b when the agent does nothing but route function calls — Groq reported 89.06% on the Berkeley Function Calling Leaderboard for that model at launch. From the previous generation, qwen2.5:7b and llama3.1:8b remain solid choices, with qwen2.5:14b the obvious step-up. Do not build on gemma2 at all: it has no Tools badge in the Ollama library. For current, independently-scored rankings check the BFCL board; for the VRAM-tier view, best Ollama models for AI agents.

Does Ollama function calling work the same as the OpenAI API?

Largely yes. Ollama implements an OpenAI-compatible tools schema using the same JSON Schema format and the same message types (assistant tool_calls, tool role responses). You can usually port OpenAI tool-calling code by changing the base URL and model. The differences: chained reasoning over 5+ tools is weaker on local models, and JSON adherence varies more by model.

How many tools can I expose without degrading performance?

There is no hard number, but about eight is a sound working limit for a 7B-14B model, and larger models tolerate more. Two things degrade as the list grows: the model has more similar-sounding descriptions to discriminate between, and the full schema array is re-sent as prompt tokens on every turn of the loop, so prefill cost climbs with it. When you outgrow the limit, group related capabilities into one tool with an enum action parameter, or split into sub-agents behind a router. Above all, only send the tools relevant to the current task rather than everything you have defined.

How do I debug tool calling issues?

Work through four checks: (1) Model ignores tools → confirm the model actually has the Tools badge and the tools array is correctly formatted. (2) Invalid JSON in arguments → drop temperature to 0.1-0.3. (3) Wrong tool selected → make the descriptions more specific and more distinct from each other. (4) Still stuck → run the server with OLLAMA_DEBUG=1 to see the raw model output before tool parsing.

How do I prevent the model from making up tool results instead of calling tools?

Three things: (1) a system prompt with an explicit "If you do not have current information, you MUST call a tool. Do not answer from memory." (2) validate every turn — if a required tool call did not happen, re-prompt once. (3) a lower temperature (0.1-0.3) to reduce hallucination. Letting the model decide is right for chatbots but wrong for structured workflows.

Can I stream the response while using tools?

Yes. Set stream=True. Text streams token-by-token; tool calls arrive as a final chunk on older Ollama builds and can stream incrementally on newer ones — handle both. Render text as it streams, show a "Calling tool_name..." indicator the moment a tool call appears, then run the tool and continue. The user only waits during tool execution.

What is the difference between JSON mode and function calling?

JSON mode (format="json") forces well-formed JSON in the content field — no tool execution, you parse it yourself. Function calling exposes a tool registry the model decides when to invoke, runs actual code, and feeds results back for synthesis. Use JSON mode for structured extraction; use function calling for agents.

Can the model call multiple tools in a single turn?

Yes. Ollama returns a tool_calls array, not a single call — a model can invoke 2-4 tools in one turn for parallelizable queries like "get weather in Paris, London, and Tokyo." Iterate the whole array, append all results, then call the model once for synthesis. This is the single biggest perf win for multi-tool agents.

Does Ollama tool calling work with LangChain or CrewAI?

Yes — see the frameworks section above. LangChain via ChatOllama + bind_tools(), CrewAI via its @tool decorator and llm="ollama/...", and LangGraph with explicit tool nodes for the most control. All three run the multi-turn call-execute-respond loop for you.

---

Closing Take

Function calling is what makes local LLMs genuinely useful for real workflows. Anyone can run a local chatbot. Building a local agent that searches your docs, runs a query and summarizes the result — that is the unlock, and Ollama is good enough for it on the right model with the right schemas.

The recipe worth starting from: qwen3:8b for development, the agent loop from this guide, three to five tightly-described tools, a system prompt that forbids answering from memory, and an evaluation harness of roughly twenty prompts that exercises every tool including the cases where no tool should fire. That last category is the one people skip and the one that breaks in production. Ship it, then measure it against your own prompts rather than anyone's published table.

Sources and further reference: Ollama API documentation · Ollama tool support announcement · Ollama streaming tool calls · Hugging Face Llama 3.1 tool-calling deep dive · LangChain Ollama integration