Best Ollama Models for AI Agents 2026: Ranked by Tool Use

For most people in 2026 the best Ollama model for AI agents is Qwen3 8B (Apache 2.0, released April 29, 2025) — it needs roughly 5 GB of weights at Q4_K_M, carries Ollama's native Tools capability, and comes from a family with published function-calling scores. If you have ~16-24 GB to spare, Qwen3 30B-A3B (a 30B Mixture-of-Experts that activates only ~3B params) is the sweet spot, and Llama 3 Groq Tool Use 8B is the specialist pick when your agent does nothing but call tools — Groq reported 89.06% on the Berkeley Function Calling Leaderboard at its launch. The trap to avoid: a model that writes great prose but emits malformed JSON tool calls is useless in an agent loop, so rank by tool support first and fluency second.

An agent is only as good as its weakest tool call. Below, nine Ollama-runnable models are ranked by VRAM tier, using evidence you can check yourself: the Tools capability badge on each model's Ollama library page, published leaderboard scores where the vendor has posted them, and VRAM computed from parameter count.

What makes a good Ollama model for AI agents?

A chatbot can ramble and still be useful. An agent cannot. When a model is wrapped in a loop — CrewAI, LangGraph, AutoGen, or your own ReAct harness — three things decide whether it works:

1. Tool-call reliability. Does it emit valid, schema-correct JSON (or the framework's expected tool-call format) every time, or does it sometimes describe the call in prose instead of emitting it? One malformed call breaks the whole chain.
2. Multi-step coherence. Can it hold a plan across several tool calls without forgetting earlier results or looping?
3. Footprint vs. speed. Bigger usually means more reliable, but if it spills out of VRAM it crawls. The right pick is the most reliable model that still fully fits your card.

Ollama only exposes a real tools API for models whose template supports it — you can check the "Tools" capability badge on each model's Ollama library page. Models without it can still be coaxed into JSON with prompting, but native tool support is far more reliable, so nearly every model in our ranking carries the Tools badge (the one exception is Qwen2.5-Coder-32B, included specifically for code-agent loops — more on that below). For the mechanics of wiring this up, see our Ollama tool-calling guide.

How is this ranking put together?

Three inputs, all of them things you can verify without taking anyone's word for it.

1. Native tool support. Ollama only exposes a real tools API for models whose chat template implements it, and it publishes that as a Tools capability badge on the model's library page. This is binary and checkable: either the badge is there or you are prompting your way to JSON.
2. Published function-calling scores. Where a vendor has posted an audited number — Groq did for Llama 3 Groq Tool Use on the Berkeley Function Calling Leaderboard — that number is cited and attributed. Where no such number exists, the table says so instead of inventing one.
3. VRAM computed from parameter count. Q4_K_M averages roughly 4.8 bits per weight, which works out to about 0.6 GB per billion parameters. So a 14B model is 14 × 0.6 ≈ 8.4 GB of weights, plus KV cache and context on top. Every VRAM figure below is that arithmetic, not a reading off a monitor — you can redo it for any model in seconds.

What this deliberately does not include is a subjective "reliability score". Tool-call behaviour drifts with Ollama version and template patches — Mistral Small 3.2's tool parser is the standing example, having shipped with parser errors that were fixed across releases — so any single snapshot ages badly. The durable signal is whether the model was trained and templated for tool use at all.

Best Ollama models for AI agents in 2026 (ranked by tier)

Here is the ranking, grouped by the VRAM you need so you can jump straight to your hardware tier. VRAM is weights only, computed at 0.6 GB per billion parameters; budget a few GB more for context. For MoE models, note the gap between total params (what you must hold in VRAM) and active params (what gets read per token, which is what drives speed).

A quick orientation before we go model-by-model: the entry tier (8 GB and under) is dominated by Qwen3 8B and the two 8B tool specialists; the mid tier (16-24 GB) is where Qwen3 30B-A3B, Gemma 4 and the 24-32B dense models live; and the top tier (Llama 4) is realistically a multi-GPU or server conversation, not a laptop one.

Entry tier (≤8 GB VRAM): Qwen3 8B and the tool specialists

Qwen3 8B — the default pick. Released April 29, 2025 under Apache 2.0, Qwen3 8B is the sensible first model for a local agent. It pulls as a ~5.2 GB download at Q4_K_M (ollama pull qwen3:8b) and lands around 6-8 GB in use once you add a working context, so it fits on an 8 GB card or a 16 GB Mac with room for the framework. It carries Ollama's Tools badge, so agent frameworks talk to it through the real tools API rather than prompt-coaxed JSON. Bonus: Qwen3 has a hybrid "thinking" mode (toggle with /think and /no_think) so you can trade latency for deeper planning per step. The family posts competitive function-calling results on the Berkeley Function Calling Leaderboard — check the current standings there rather than trusting any figure frozen into an article — and the 8B inherits the same tool-trained lineage. See the official Qwen3 announcement for the full family.

Llama 3 Groq Tool Use 8B — the specialist. If your agent does little besides call functions, this is the sharpest small tool. Built by Groq with Glaive on Meta-Llama-3-8B and fine-tuned with full SFT + DPO purely for tool use, it scored 89.06% overall accuracy on the Berkeley Function Calling Leaderboard (#3 among all models, best open 8B at the July 2024 launch); the 70B sibling hit 90.76% (#1 at launch). Pull it with ollama pull llama3-groq-tool-use:8b (~4.7 GB, 8K context). The trade-off is that it is older (Llama 3 era) and narrow — it is a function-calling scalpel, not a general reasoner, and its 8K context is tight for long agent transcripts. We keep a deeper spec sheet on it in our Llama 3 Groq 8B model page.

Both fit the entry tier, but they solve different problems: Qwen3 8B is the generalist that also calls tools well; Groq Tool Use 8B is the purpose-built caller you bolt onto a deterministic pipeline.

Mid tier (16-24 GB VRAM): the reliability sweet spot

Qwen3 30B-A3B — big-model behaviour at small-model speed. This is a 30B-total Mixture-of-Experts that activates only ~3B parameters per token. The arithmetic is the whole story: you must hold all 30B in memory (30 × 0.6 ≈ 18 GB of weights, so a 24 GB card such as an RTX 3090 or 4090 with context headroom), but only ~3B params get read per token (~1.8 GB), and per-token read volume is what sets the speed ceiling. That is why it behaves like a 30B while moving closer to the pace of a 3B — the combination most people in this tier actually want. Pull with ollama pull qwen3:30b-a3b.

Hermes 4 14B — steerable reasoning + tools. NousResearch's Hermes 4 (technical report on arXiv, 2508.18255, August 2025) is fine-tuned on top of Qwen3-14B and emits tool calls inside <tool_call> tags after an explicit reasoning step, which makes its calls easy to parse and its planning transparent. It lands around 9-10 GB at Q4, so it actually straddles the entry/mid line and runs on a 12 GB card. Hermes 4 also ships in larger 70B and 405B variants if you have the hardware. One practical wrinkle: at the time of writing there is no first-party Hermes 4 14B entry in Ollama's official library, so you import the GGUF (from the NousResearch Hermes-4-14B repo) or pull a community upload — confirm the source before trusting it in production. Pick it when you want a model that "shows its work" before each tool call.

Gemma 4 (31B dense) — native function-calling. Google released Gemma 4 on April 2, 2026 in four flavors: E2B, E4B, a 26B MoE (the "26B A4B", ~4B active), and a 31B dense model. The whole family was built with "native support for function-calling, structured JSON output, and native system instructions," explicitly to build agents that interact with tools and APIs. The 31B dense variant is the one to run for serious agent work — pull it specifically with ollama pull gemma4:31b (the bare gemma4 tag defaults to the small E4B edge model, not the 31B). Its weights work out to 31 × 0.6 ≈ 18.6 GB at Q4_K_M, so budget roughly 21 GB with a working context — a 24 GB card. (Watch the naming: Gemma 4 is 31B dense / 26B MoE — the old 27B size belonged to Gemma 3, not Gemma 4.) Details are on the official Gemma 4 announcement.

Mistral Small 3.2 24B — low-latency calls, with a caveat. Mistral Small 3.2 (24B) is an official Ollama model (mistral-small3.2:24b) tuned for low-latency function calling and JSON output, sitting around 14-15 GB at Q4. The honest caveat: tool calling for this model has had parser issues in Ollama (a "failed to create tool parser" error was reported around its release) that get patched across versions — so confirm tool calls work on your Ollama build before committing it to a production agent.

Qwen2.5-Coder-32B — the coding-agent pick. If your agent's job is writing and running code (a SWE-style loop), Qwen2.5-Coder-32B is the strongest local choice that fits a single 24 GB card. It needs roughly 18-20 GB at Q4_K_M (RTX 3090 is the practical minimum) and Qwen positions it explicitly for "Code Agents." The one caveat versus the rest of this list: it does not carry Ollama's native Tools badge, so it shines inside a coding harness that drives tool use through prompting (Continue, Aider, an editor agent) rather than the framework-native tools API. Pair it with a coding harness rather than a generic tool-calling one.

Top tier (multi-GPU / workstation): Llama 4 Scout and Maverick

Meta's Llama 4 models are MoE and large. Llama 4 Scout is 109B total with 17B active across 16 experts and a very long context window; by the same 0.6 GB-per-billion arithmetic its Q4_K_M weights come to about 65 GB (109 × 0.6), so it realistically wants a single 80 GB H100 (or a small multi-GPU rig). It can be squeezed onto a 24 GB card only with extreme sub-2-bit dynamic quants, at a real quality and speed cost — not the way most people should run it. Llama 4 Maverick is 400B total with 17B active — capable, but realistically a multi-GPU or server deployment, not a desktop one. Both are pullable (ollama run llama4:scout / ollama run llama4:maverick) and support tool calling, but for the vast majority of local-agent builders they are aspirational rather than practical. If you are weighing the whole Llama 4 line for local use, that is its own decision — most readers will get more done with Qwen3 30B-A3B at a fraction of the hardware.

Which model for CrewAI, LangGraph, or Continue?

The framework changes what "best" means, because each one stresses a different capability:

• CrewAI spins up multiple role-playing agents that delegate and call tools constantly, so tool-call reliability is everything. Start with Qwen3 8B if you are on modest hardware, and step up to Qwen3 30B-A3B if you have 24 GB — the extra reliability pays off across a multi-agent crew where one bad call cascades. Our CrewAI local setup guide walks through pointing CrewAI at an Ollama endpoint.
• LangGraph builds explicit state-machine graphs where each node may call a tool; it rewards models that emit clean, deterministic calls and follow a plan. Qwen3 30B-A3B or Hermes 4 14B (for its visible reasoning step) are the picks here. The strictly-tool Llama 3 Groq Tool Use 8B also shines as a dedicated "tool node" model.
• Continue (the IDE assistant) is really a coding agent, so reach for Qwen2.5-Coder-32B if you have the VRAM, or a smaller Qwen coder if you do not — general agent models underperform on in-editor code tasks.

For the bigger picture on architecting local agents end-to-end — memory, planning, and tool wiring — read our local AI agents guide. And if you just want the best general-purpose Ollama models regardless of agent use, our best Ollama models roundup ranks the wider field.

Will it be fast enough? Work out the ceiling yourself

You do not need anyone's benchmark to know roughly how fast a model will run in an agent loop, because token generation is memory-bandwidth bound. Every token requires reading the active weights once, so:

tokens/sec ceiling = memory bandwidth (GB/s) ÷ bytes read per token (GB)

This is an arithmetic upper bound. Real output lands well below it — sampling overhead, KV-cache reads that grow with context, and framework round-trips all eat into it — but it ranks options correctly and it exposes the two things that actually matter.

Using published memory bandwidth figures and the 0.6 GB-per-billion rule from earlier:

Two conclusions fall straight out of the table. First, MoE is the cheat code for agents: Qwen3 30B-A3B reads just over a third of the bytes per token that Qwen3 8B does, because only the active experts are touched — a 30B that is arithmetically faster than an 8B, as long as you can hold all 18 GB.

Second, and more important than any model choice: keep the whole model in VRAM. A PCIe 4.0 x16 link moves about 32 GB/s. If even part of the model has to be fetched across it every token, your divisor swaps a ~936 GB/s number for a ~32 GB/s one — roughly a 29× haircut on the ceiling. That single cliff explains almost every "why is my local agent unusably slow" report. If a model does not fit, drop to a smaller one or a lighter quant rather than letting it spill. Our VRAM calculator will check a given model against your card before you pull 20 GB.

Key takeaways

1. Qwen3 8B is the default best Ollama model for AI agents in 2026 — Apache 2.0, ~4.8 GB of weights at Q4_K_M (6-8 GB in use with context), and Ollama's native Tools capability so frameworks use the real API.
2. Check the Tools badge before anything else. A model that writes beautifully but emits malformed JSON breaks the agent loop; nearly every pick here carries Ollama's native Tools badge (Qwen2.5-Coder-32B is the code-agent exception, driven through a coding harness instead).
3. Qwen3 30B-A3B is the 16-24 GB sweet spot — a 30B MoE that stores 18 GB but reads only ~1.8 GB per token, so its speed ceiling beats a dense 8B while its behaviour is that of a 30B.
4. Llama 3 Groq Tool Use 8B is the specialist — Groq reported 89.06% on the Berkeley Function Calling Leaderboard at launch (#3 overall; the 70B sibling led at 90.76%); narrow but excellent as a dedicated tool node.
5. Match the model to the framework: Qwen3 (8B or 30B-A3B) for CrewAI, Qwen3 30B-A3B / Hermes 4 14B for LangGraph, Qwen2.5-Coder-32B for Continue. Llama 4 Scout/Maverick are multi-GPU territory.
6. Fit beats size. Spilling any layer to system RAM swaps a ~936 GB/s divisor for a ~32 GB/s one — about a 29× cut to your ceiling. Choose the largest model that fully fits, never the largest model.

Next steps

• Wiring tools into Ollama for the first time? Start with the Ollama tool-calling guide — it covers the tools API and the JSON formats agents expect.
• Building a multi-agent crew? Follow our CrewAI local setup guide to point CrewAI at an Ollama model.
• Want the architecture-level view of local agents? Read the local AI agents guide.
• Just shopping for the best Ollama models overall? See our best Ollama models roundup.
• Curious about the function-calling specialist? Our Llama 3 Groq 8B page has the full spec sheet.