Best Ollama Models for Tool Calling: BFCL Ranked (2026)

The highest published tool-calling score for a self-hostable model belongs to Llama-3-Groq-70B-Tool-Use, at 90.76% overall on the Berkeley Function Calling Leaderboard (BFCL) as reported by Groq, with the matching 8B at 89.06% for single-GPU boxes. For a current general-purpose pick, Qwen3 has native tool calling built into its chat template and runs cleanly through Ollama's tools API, while Cohere's Command-R (35B) is the specialist for multi-step tool use plus RAG. Below is the ranking, the VRAM arithmetic, and a tools-array script you can run yourself to see the failure modes rather than the marketing.

If you are wiring a local model into an agent, the metric that actually matters is not raw chat quality — it is whether the model emits valid, parseable JSON that matches your tool schema, every time, without narrating around it. A model that writes beautiful prose but wraps its function call in a markdown fence or invents an argument name will break your agent loop. This guide ranks models on that reliability.

What makes a model good at tool calling?

Tool calling (also called function calling) is when you hand the model a list of available functions — each with a name, description, and a JSON schema for its arguments — and the model decides which to call and fills in the arguments. Ollama exposes this through the tools array on its /api/chat endpoint, and the model is expected to return a structured tool_calls object rather than free text.

Four things separate a good tool-calling model from a frustrating one:

1. Valid-JSON rate — does it return arguments that actually parse, with no trailing commas, no markdown fences, no commentary mixed in?
2. Argument accuracy — does it map your prompt to the right tool and fill the right fields (correct types, no hallucinated params)?
3. Parallel tool calls — when a request needs two lookups at once, can it emit multiple calls in a single turn instead of one-at-a-time?
4. Knowing when NOT to call — does it answer directly when no tool is needed, instead of forcing an irrelevant call?

You can check any of these yourself in about a minute. Define two tools, pass them to Ollama's tools API, and read what comes back. Here is the minimal harness — swap the model name to compare candidates on your own hardware:

import ollama

tools = [
  {
    "type": "function",
    "function": {
      "name": "get_weather",
      "description": "Get the current weather for a city",
      "parameters": {
        "type": "object",
        "properties": {
          "city": {"type": "string", "description": "City name"},
          "unit": {"type": "string", "enum": ["celsius", "fahrenheit"]}
        },
        "required": ["city"]
      }
    }
  },
  {
    "type": "function",
    "function": {
      "name": "convert_currency",
      "description": "Convert an amount between two currencies",
      "parameters": {
        "type": "object",
        "properties": {
          "amount": {"type": "number"},
          "from": {"type": "string"},
          "to": {"type": "string"}
        },
        "required": ["amount", "from", "to"]
      }
    }
  }
]

resp = ollama.chat(
  model="llama3-groq-tool-use:8b",
  messages=[{"role": "user",
             "content": "Weather in Tokyo in celsius, and convert 50 USD to JPY"}],
  tools=tools,
)
print(resp["message"]["tool_calls"])

A strong model returns two tool_calls from that one prompt — get_weather(city="Tokyo", unit="celsius") and convert_currency(amount=50, from="USD", to="JPY") — with clean JSON and no extra text. A weak model returns one call, forgets the unit, or wraps the JSON in a code fence that breaks the parser. For the full setup walkthrough, see our Ollama tool calling guide.

What is the best Ollama model for tool calling in 2026? (Ranked)

The ranking below blends published Berkeley Function Calling Leaderboard (BFCL) results — the standard academic benchmark for function calling, maintained at the Gorilla project at UC Berkeley — with what each model's chat template and documentation actually support in Ollama's native tools API. BFCL scores are quoted from each model's own published source; where a model has no single published BFCL figure, the cell says so rather than inventing one.

A few honest notes on this table. The two Llama-3-Groq-Tool-Use models still hold the cleanest published BFCL numbers of any easily self-hosted option — 90.76% and 89.06% — because Groq fine-tuned them specifically for this one job with Glaive. They are based on Llama 3 (mid-2024), so they are not the newest brains, but for pure schema-following they are hard to beat. Qwen3 is the model most people should reach for today: tool calling is baked into its chat template, it runs through Ollama's tools API without hacks, and it is two model generations newer than the 2024-era Groq finetunes, with a correspondingly more recent knowledge cutoff. Command-R earns its spot because Cohere trained it for genuine multi-step tool use and grounded RAG, with publicly available 35B weights and a 128K context.

Why Llama-3-Groq-Tool-Use still tops the published benchmark

When Groq released these two models (in collaboration with Glaive), the 70B variant took the #1 spot on the Berkeley Function Calling Leaderboard at 90.76% overall accuracy, and the 8B variant landed #3 at 89.06% — beating, at the time, several proprietary models on that specific benchmark. They got there by doing one narrow thing extremely well: full fine-tuning plus Direct Preference Optimization (DPO) aimed purely at emitting correct tool calls.

You can run the 8B locally through Ollama:

ollama pull llama3-groq-tool-use:8b
ollama run llama3-groq-tool-use:8b

The trade-off is age. These are Llama-3 finetunes from mid-2024, so their general reasoning and world knowledge trail newer models. If your agent only needs to translate a request into the right function call against a fixed toolset, that does not matter and the Groq 8B is an excellent, lean choice. If your agent also has to reason hard between steps, a newer model like Qwen3 will feel smarter even if its raw BFCL number is lower. You can read Groq's own write-up of the scores on the official Groq Tool Use announcement, and we cover the 8B in depth on our Llama 3 Groq 8B model page.

Best small model for tool calling (single GPU / 8GB-16GB)

Not everyone has a 70B-capable box. If you are on a single consumer GPU, three small models stand out:

• Llama-3-Groq-8B-Tool-Use (8B) — the highest published BFCL score in the small class at 89.06%. It fits comfortably in ~6-8 GB of VRAM at a 4-bit quant and is purpose-built for clean tool calls. This is the safe default if tool reliability is the only thing you care about.
• Qwen3 8B — a better all-rounder. Native tool template, strong reasoning, and a long context, so it both calls tools correctly and reasons well between calls. Pick this if your agent does more than dumb function dispatch. See our companion guide on the build-a-local-AI-agent walkthrough for how it slots into an agent loop.
• Mistral 7B (v0.3 or later) — native function calling and the lightest footprint here, ideal for resource-constrained or edge setups. It is less specialized than the Groq finetune, so expect to validate its JSON a bit more carefully.

There are also community finetunes built solely for this — Watt-Tool-8B and ToolACE-8B, both based on Llama-3.1-8B-Instruct — that report state-of-the-art small-model results on BFCL. They are worth trying if the mainstream 8B options leave gaps, though they live on Hugging Face rather than the core Ollama library, so you may need to import a GGUF. For sizing any of these against your card, run the numbers through our VRAM calculator first.

A note on Qwen2.5, Gemma, and "Gemma 4"

Two quick clarifications, because the naming trips people up:

• Qwen2.5-7B still works fine for tool calling and remains a solid, lighter alternative if you have not moved to Qwen3 — its instruct models support the same tools-array flow. Details are on our Qwen2.5 7B model page.
• There is no "Gemma 4." As of mid-2026 the current Google open model is Gemma 3 (the 27B is the largest). Gemma 3 can do function calling, but it has no dedicated tool-call tokens — you steer it into structured JSON purely through prompting, which makes it less reliable for agents than the natively trained models above. Google also shipped a tiny specialized variant, FunctionGemma (based on Gemma 3 270M), aimed only at function calling. If you want plug-and-play tool calling, prefer Qwen3 or the Groq Tool-Use models over base Gemma 3.

How fast are tool-calling models, and what VRAM do they need?

You can work both numbers out before downloading anything. Token generation is memory-bandwidth bound — the GPU reads every weight once per token — so two bits of arithmetic cover it:

VRAM for weights (GB) at Q4_K_M  ~=  0.6 x (parameters in billions)
throughput ceiling (tok/s)        =  memory bandwidth (GB/s) / weights (GB)

The second line is an arithmetic upper bound, not a prediction. Attention, sampling, prompt processing and framework overhead all cost time, and the gap is widest on small models where fixed per-token costs dominate. What the bound is good for is sizing and sanity-checking: any tokens-per-second claim above the ceiling is wrong, whoever published it.

Worked below against 936 GB/s — the RTX 3090's published memory bandwidth, per the TechPowerUp GPU database. Substitute your own card's figure and the whole column rescales linearly.

Two practical consequences. First, tool calls are short outputs — a few dozen JSON tokens — so anything in this table returns a call quickly once the model is resident; latency is dominated by prompt processing and your own tool execution, not by generation. Second, the moment any layer spills to system RAM the ceiling drops by roughly an order of magnitude, because DDR bandwidth is a fraction of GDDR6X. Keep the whole model on the GPU. The 70B needs 48 GB, which in practice means a pair of cards — see the Ollama multi-GPU setup guide for how that split actually behaves.

Common tool-calling failure modes (and how to fix them)

Even the best model misbehaves eventually. These are the failure modes that break an agent loop, and what to do about each:

• JSON wrapped in a markdown fence. The model returns a fenced JSON code block (a triple-backtick json block) instead of a clean tool_calls object, and your parser chokes. Fix: use a model with native tool support (Qwen3, Groq Tool-Use) so Ollama parses it into tool_calls for you, rather than asking a non-tool model to "respond in JSON."
• Hallucinated or renamed arguments. The model invents a field that is not in your schema, or renames city to location. Fix: write tight, unambiguous parameter descriptions, mark fields required, and validate the returned args against your schema before executing — never trust them blind.
• One call when you needed two (no parallel calls). Weaker models answer only the first half of a compound request. Fix: prefer models that explicitly support parallel tool calls (Firefunction-v2, Qwen3, the Groq finetunes), or split the request into separate turns.
• Calling a tool when none was needed. The model forces an irrelevant function call on a plain question. Fix: add a clear "answer directly if no tool applies" instruction and choose a model trained to abstain — the Groq Tool-Use finetunes were specifically tuned to decide whether to call.
• Wrong types (string where a number belongs). "amount": "50" instead of 50. Fix: coerce/validate types on your side; do not assume the model respects "type": "number".

For end-to-end agent patterns that handle these gracefully, see our local AI agents guide.

Key Takeaways

1. Llama-3-Groq-70B-Tool-Use leads the published benchmark at 90.76% BFCL (8B at 89.06%) — the cleanest function-calling finetunes you can self-host, if older-generation reasoning is acceptable.
2. Qwen3 is the best modern general-purpose pick — native tool template, works directly through Ollama's tools API, and two generations newer than the 2024-era finetunes above it.
3. Command-R (35B) is the multi-step + RAG specialist; Hermes 4 and Mistral round out open agent stacks; Firefunction-v2 is parallel-call capable but hosted-first.
4. For a single GPU, pick Llama-3-Groq-8B for pure tool reliability or Qwen3 8B for reasoning-plus-tools — both fit ~8-12 GB at 4-bit.
5. There is no "Gemma 4." The current model is Gemma 3, which lacks native tool tokens — prefer natively trained tool models for agents.
6. The real metric is valid-JSON rate and argument accuracy, not chat quality. Always validate returned arguments against your schema before executing them.

Next Steps

• New to the mechanics? Start with the hands-on Ollama tool calling guide.
• Building an agent? Follow the build a local AI agent walkthrough and the broader local AI agents guide.
• Choosing the small-GPU model? Compare the Llama 3 Groq 8B and Qwen2.5 7B model pages.
• Not sure a model fits your card? Size any quant with the VRAM calculator before downloading the weights.