Build AI Agents Locally with Ollama: No API Costs (2026 Guide)

What Are AI Agents?

AI agents are autonomous systems that can plan, reason, and execute complex tasks by breaking them into steps and using tools. Unlike simple chatbots that respond to single queries, agents:

• Plan: Break complex goals into subtasks
• Execute: Perform actions using tools (search, code, APIs)
• Iterate: Refine results based on feedback
• Remember: Maintain context across interactions

Agent Architecture

User Goal → Planning → Tool Selection → Execution → Observation → Reasoning → Output
              ↑                                                        ↓
              ←────────────────── Iteration Loop ─────────────────────←

Why Build Agents Locally?

Running agents locally with Ollama costs nothing after initial hardware—and you keep complete control of your data.

Framework Comparison

Status and star counts come from each project's GitHub repository on November 2, 2026: CrewAI, LangGraph, AutoGen and Swarm. The AutoGen README now says the project is archived and asks new users to start with Microsoft Agent Framework; the Swarm README says it has been replaced by the OpenAI Agents SDK.

When to Choose Each Framework

Choose CrewAI when:

• You want the fastest path to a working prototype
• Your workflow is sequential or hierarchical (research → write → review)
• You're new to AI agents and want an intuitive API
• You need a small, focused team of 2-5 agents

Choose LangGraph when:

• You're building a production system that needs reliability
• You need complex control flow (loops, branching, conditional paths)
• You want human-in-the-loop approval at specific steps
• You need checkpointing for long-running or resumable workflows
• You're already in the LangChain ecosystem

Choose AutoGen when:

• You already have AutoGen code that works and are planning its migration
• You want agents that write code and run it in a loop
• You're doing research or experimentation with multi-agent conversations

For a new Microsoft-stack project, start with Microsoft Agent Framework instead of AutoGen. A common path is to prototype in CrewAI and move to LangGraph when you hit a control-flow wall; the CrewAI local setup guide and LangGraph local setup guide cover each one end to end.

CrewAI: Build Your First Local Agent Team

CrewAI makes it easy to create agent teams with defined roles.

Installation

pip install crewai 'crewai[tools]'
ollama pull llama3.1:70b

Basic Crew Example

from crewai import Agent, Task, Crew, LLM

# Configure local LLM (CrewAI routes "ollama/..." models to your Ollama server)
llm = LLM(
    model="ollama/llama3.1:70b",
    temperature=0.7,
    base_url="http://localhost:11434"
)

# Define agents with roles
researcher = Agent(
    role="Research Analyst",
    goal="Find accurate, comprehensive information on topics",
    backstory="Expert researcher with attention to detail",
    llm=llm,
    verbose=True
)

writer = Agent(
    role="Content Writer",
    goal="Create clear, engaging content from research",
    backstory="Skilled writer who makes complex topics accessible",
    llm=llm,
    verbose=True
)

# Define tasks
research_task = Task(
    description="Research the latest developments in local AI agents",
    agent=researcher,
    expected_output="Detailed research summary with key findings"
)

writing_task = Task(
    description="Write a blog post based on the research",
    agent=writer,
    expected_output="Polished 500-word blog post",
    context=[research_task]  # Uses research output
)

# Create and run crew
crew = Crew(
    agents=[researcher, writer],
    tasks=[research_task, writing_task],
    verbose=True
)

result = crew.kickoff()
print(result)

Adding Tools to Agents

from crewai_tools import (
    FileReadTool,
    DirectoryReadTool,
    WebsiteSearchTool
)

# Create tools
file_tool = FileReadTool()
dir_tool = DirectoryReadTool()
search_tool = WebsiteSearchTool()

# Agent with tools
researcher = Agent(
    role="Research Analyst",
    goal="Research topics using web search and local files",
    llm=llm,
    tools=[file_tool, dir_tool, search_tool],
    verbose=True
)

LangGraph: Custom Agent Workflows

LangGraph provides fine-grained control over agent state and flow. For a step-by-step walkthrough wiring LangGraph to a fully local model, see our LangGraph local setup guide.

Installation

pip install langgraph langchain-ollama

ReAct Agent Pattern

from langgraph.graph import StateGraph, END
from langchain_ollama import ChatOllama
from langchain_core.messages import HumanMessage, AIMessage
from typing import TypedDict, Annotated
import operator

# Define state
class AgentState(TypedDict):
    messages: Annotated[list, operator.add]
    next_action: str

# Initialize LLM
llm = ChatOllama(model="llama3.1:70b")

# Define nodes
def reasoning_node(state: AgentState):
    """Agent reasoning step"""
    messages = state["messages"]
    response = llm.invoke(messages)
    return {"messages": [response], "next_action": "decide"}

def tool_node(state: AgentState):
    """Execute tools based on agent decision"""
    last_message = state["messages"][-1]
    # Parse and execute tool calls
    # ... tool execution logic
    return {"messages": [tool_result], "next_action": "reason"}

def should_continue(state: AgentState):
    """Decide whether to continue or finish"""
    last_message = state["messages"][-1]
    if "FINAL ANSWER" in last_message.content:
        return "end"
    return "continue"

# Build graph
workflow = StateGraph(AgentState)

workflow.add_node("reason", reasoning_node)
workflow.add_node("act", tool_node)

workflow.set_entry_point("reason")

workflow.add_conditional_edges(
    "reason",
    should_continue,
    {"continue": "act", "end": END}
)

workflow.add_edge("act", "reason")

# Compile and run
app = workflow.compile()
result = app.invoke({
    "messages": [HumanMessage(content="Research AI agents")],
    "next_action": "reason"
})

AutoGen: Conversational Coding Agents

AutoGen excels at agents that write and execute code. One status note before you start: Microsoft's AutoGen repository now describes the project as archived and no longer actively maintained, warns against production use, and recommends Microsoft Agent Framework for new projects. If you already run AutoGen, the code below uses its current AgentChat API.

The old from autogen import AssistantAgent, UserProxyAgent examples you will find in older tutorials no longer work after pip install pyautogen: on PyPI that name is now a proxy for autogen-agentchat, which uses the imports shown here.

Installation

pip install -U "autogen-agentchat" "autogen-ext[ollama]"
ollama pull qwen2.5-coder:32b

Code-Writing Agent Team

import asyncio
from autogen_agentchat.agents import AssistantAgent, CodeExecutorAgent
from autogen_agentchat.conditions import MaxMessageTermination, TextMentionTermination
from autogen_agentchat.teams import RoundRobinGroupChat
from autogen_agentchat.ui import Console
from autogen_ext.code_executors.local import LocalCommandLineCodeExecutor
from autogen_ext.models.ollama import OllamaChatCompletionClient

async def main() -> None:
    # Configure local LLM (talks to Ollama on localhost:11434)
    model_client = OllamaChatCompletionClient(model="qwen2.5-coder:32b")

    # The AI that writes code
    coder = AssistantAgent(
        "coding_assistant",
        model_client=model_client,
        system_message="Write Python in one fenced block. Reply TERMINATE once the script has run without errors.",
    )

    # Runs the code blocks it receives (use a Docker executor for untrusted tasks)
    executor = CodeExecutorAgent(
        "executor",
        code_executor=LocalCommandLineCodeExecutor(work_dir="workspace"),
    )

    stop = TextMentionTermination("TERMINATE") | MaxMessageTermination(12)
    team = RoundRobinGroupChat([coder, executor], termination_condition=stop)
    await Console(team.run_stream(task="Write a Python script that scrapes HackerNews top stories"))
    await model_client.close()

asyncio.run(main())

Tool Integration Patterns

Web Search Tool

from langchain_community.tools import DuckDuckGoSearchRun

search = DuckDuckGoSearchRun()

# Use in agent
agent_with_search = Agent(
    role="Researcher",
    tools=[search],
    llm=llm
)

Code Execution Tool

from langchain_experimental.tools import PythonREPLTool

python_repl = PythonREPLTool()

# Agent can write and execute code
coder = Agent(
    role="Python Developer",
    tools=[python_repl],
    llm=llm
)

File System Tools

from langchain_community.tools import (
    ReadFileTool,
    WriteFileTool,
    ListDirectoryTool
)

file_tools = [
    ReadFileTool(),
    WriteFileTool(),
    ListDirectoryTool()
]

# Agent with file access
file_agent = Agent(
    role="File Manager",
    tools=file_tools,
    llm=llm
)

Memory Systems for Agents

Conversation Memory

from langchain.memory import ConversationBufferMemory

memory = ConversationBufferMemory(
    memory_key="chat_history",
    return_messages=True
)

Vector Store Memory (Long-term)

from langchain_community.vectorstores import Chroma
from langchain_community.embeddings import OllamaEmbeddings

# Create vector store for memories
embeddings = OllamaEmbeddings(model="nomic-embed-text")
vectorstore = Chroma(
    collection_name="agent_memory",
    embedding_function=embeddings,
    persist_directory="./memory_db"
)

# Store and retrieve memories
vectorstore.add_texts(["Important fact from previous session"])
relevant_memories = vectorstore.similarity_search("query", k=5)

Recommended Local Models for Agents

Download sizes are the file sizes Ollama's library lists for each default tag (checked November 2, 2026). The model has to fit in VRAM or unified memory with room left for the context window, so plan for a few GB above the file size. DeepSeek V3 is a mixture-of-experts model: only about 37B parameters are active per token, but all 671B still have to be loaded, which puts it out of reach of any single consumer GPU. For a 24GB card, deepseek-r1:32b (19.9GB) is the practical reasoning pick.

Picking the right model matters more for agents than for chat, because weak tool-calling breaks the whole loop. We rank the top picks in the best Ollama models for agents, and you can filter every model by VRAM and capability in the local AI model database.

Production Considerations

Error Handling

from tenacity import retry, stop_after_attempt, wait_exponential

@retry(stop=stop_after_attempt(3), wait=wait_exponential(min=1, max=10))
def run_agent_with_retry(crew, inputs):
    try:
        return crew.kickoff(inputs=inputs)
    except Exception as e:
        print(f"Agent error: {e}")
        raise

Iteration Limits

crew = Crew(
    agents=[researcher, writer],
    tasks=[task],
    max_iter=10,  # Prevent infinite loops
    verbose=True
)

Logging and Monitoring

import logging

logging.basicConfig(level=logging.INFO)
logger = logging.getLogger("agent")

# Log agent actions
logger.info(f"Agent {agent.role} starting task: {task.description}")

Once agents stop being scripts you run and start being processes that stay up, the plumbing around them—scheduling, persistent memory, tool permissions, approval steps—starts to look less like an app and more like an operating layer. That is the argument behind AI operating systems, which arranges the same components you have wired here into a five-layer stack with a policy tier sitting between the model and anything destructive.

Real-World Agent Examples

1. Research Assistant

Agent that searches the web, reads documents, and creates summaries.

2. Code Review Bot

Agent that analyzes code, finds bugs, and suggests improvements.

3. Data Analysis Pipeline

Agent that queries databases, creates visualizations, and writes reports.

4. Customer Support

Agent that answers questions using a knowledge base and escalates when needed.

5. Content Creation

Multi-agent team that researches, writes, and edits content.

Each of these is a back-end loop that still needs a surface people can talk to. The fastest one to stand up is a chat front end — build a Telegram bot with local AI walks through the Ollama + Python wiring. If you'd rather ship a browser UI, Ollama with the Vercel AI SDK covers streaming agent output into a web app.

Key Takeaways

1. AI agents can run 100% locally using Ollama and open-source frameworks
2. CrewAI is best for beginners with its role-based team approach
3. LangGraph offers maximum flexibility for custom agent architectures
4. 16GB+ VRAM recommended for smooth agent operation with capable models
5. Tools enable real-world actions—web search, code execution, file access
6. Memory systems allow agents to learn and persist knowledge

Next Steps

1. Set up DeepSeek R1 for reasoning-heavy agent tasks
2. Connect Ollama to MCP servers for advanced tool integration
3. Build RAG pipelines for document-aware agents
4. Optimize your GPU for faster agent execution

---

AI agents represent the next evolution of AI applications—from simple Q&A to autonomous task completion. With local models and open-source frameworks, you can build powerful agents without cloud dependencies or ongoing costs.