Build a Local AI Slack & Discord Bot with Ollama (Full Tutorial)

Published on April 23, 2026 • 19 min read

The pitch deck for cloud chatbots is always the same: connect your team chat, get an AI assistant. The fine print is also always the same: the assistant reads your messages, your DMs, your customer data, your internal docs, and the vendor stores it long enough to "improve the service." For most engineering teams that is a non-starter.

The good news is that a bot which mentions-and-replies is genuinely an afternoon's work. The gap between that and something a team can use unattended is everything that only shows up once real people are in the channel: threaded replies so it does not shout into the main feed, per-user rate limiting so one enthusiast cannot starve the queue, RAG over a private docs folder so the answers are about your company rather than the internet's average company, and a deployment that survives a reboot.

This guide covers both halves, for Slack and for Discord, with Python you can drop into a repo and run today. If your team lives on Telegram instead, the companion guide on how to build a Telegram bot with local AI uses the same Ollama backend.

---

Quick Start: 8 Minutes to a Working Bot

If you just want to see a bot reply in your channel:

# Prerequisites: Python 3.11+, Ollama already installed
ollama pull llama3.1:8b

# Slack version
pip install slack-bolt ollama python-dotenv
export SLACK_BOT_TOKEN=xoxb-...
export SLACK_APP_TOKEN=xapp-...
python slack_bot.py

# Or Discord version
pip install discord.py ollama python-dotenv
export DISCORD_TOKEN=...
python discord_bot.py

The minimal Slack bot is a few dozen lines of Python (the full version is below). By the end of this guide you will have something a real team can use without you babysitting it.

---

Table of Contents

1. Why Self-Host Your Team AI Bot
2. Architecture Overview
3. Hardware & Hosting
4. Ollama Setup for Bot Workloads
5. Slack Bot — Full Implementation
6. Discord Bot — Full Implementation
7. Adding RAG over Team Docs
8. Slash Commands & Tools
9. Rate Limiting & Cost Control
10. Production Deployment
11. Pitfalls Worth Knowing Before You Ship
12. FAQs

---

Why Self-Host Your Team AI Bot

Three reasons that hold up under scrutiny:

1. Channel content is sensitive by default. Engineering, security, finance and exec channels routinely contain credentials, customer names and unannounced product details. A bot that reads mentions reads whatever was pasted three messages earlier. Routing that to a hosted API creates a data-flow your security team has to reason about, however good the vendor's privacy terms are.

2. Per-token pricing punishes adoption, and adoption is the goal. Do the arithmetic for your own team rather than trusting a headline figure. A single bot reply carries a system prompt, a few turns of thread history and — once RAG is on — several retrieved document chunks; a couple of thousand input tokens per message is unremarkable. A team generating 50,000 bot messages a month is therefore on the order of 100 million input tokens a month. Multiply by whatever your provider charges per million and compare it to a fixed monthly box. The uncomfortable part of the cloud version is that the bill rises precisely when the bot becomes useful.

3. RAG over private docs needs to stay private. The most useful team bots answer "what does our pricing tier contain?" or "where is the runbook for this service?" That requires indexing internal docs. Cloud RAG means uploading your wiki to a third party; local RAG keeps it on your hardware, which is often the difference between a project being approved and being shelved.

Latency is a genuine trade rather than a clean win, and it is worth being precise about it. A cloud call pays a network round trip and whatever queueing a shared multi-tenant service imposes. A local call pays neither — but it pays model load time if the model has been evicted from VRAM (see OLLAMA_KEEP_ALIVE below) and it pays queueing behind other people using your one GPU. Which of those dominates depends on your team size and your hardware, so measure it rather than assuming.

---

Architecture Overview

┌──────────┐   websocket    ┌────────────┐   HTTP    ┌────────┐
│  Slack   │◄──────────────►│  Bot Proc  │──────────►│ Ollama │
│ Discord  │   socket mode  │  (Python)  │   :11434  │  LLM   │
└──────────┘                └────────────┘           └────────┘
                                  │
                                  ▼
                            ┌─────────────┐
                            │  ChromaDB   │  ← team docs RAG
                            │  (local)    │
                            └─────────────┘

Three components:

1. Bot process — Python event loop that listens to Slack/Discord events and orchestrates responses.
2. Ollama — Local LLM server. Same machine or a different one on your network.
3. ChromaDB (optional) — Vector store for RAG. Docker container on the same host.

No public ingress required. Slack uses Socket Mode and Discord uses websockets, so your bot connects out — no inbound port exposure.

---

Hardware & Hosting

Two things decide what you can run: whether the model fits in VRAM, and how fast the card can read it. Both are arithmetic, so you can size the box before you buy it.

Does it fit?

At Q4_K_M a model's weights come to roughly 0.6 GB per billion parameters. You need headroom on top for the KV cache, which grows with context length and with OLLAMA_NUM_PARALLEL — each concurrent slot gets its own cache, which is the part people forget when a bot that worked in testing starts falling over at ten users.

How fast will it answer?

Decoding reads every weight once per token, so throughput is memory-bandwidth bound and the ceiling is bandwidth divided by weight bytes:

Arithmetic upper bounds, not measurements. Real decode always lands below them — but nothing you do in software gets above them, which is what makes them the right numbers for capacity planning.

Two consequences worth internalising:

• A 70B is slower than it looks. At Q4 it reads about 42 GB per token. Split across two RTX 3090s the ceiling is 936 ÷ 42 ≈ 22 tok/s — the layers are read sequentially, so the two cards' bandwidths do not add. For a bot where somebody is watching a "typing…" indicator, a 14B that answers immediately usually beats a 70B that is marginally more often right.
• Concurrency divides the ceiling; it does not create bandwidth. OLLAMA_NUM_PARALLEL=4 lets four requests share one card, so four people decoding at once each see a fraction of the figures above. Size for your peak concurrency, not your average message rate.

Where to run it

• A fixed-price GPU dedicated server (Hetzner's GEX line, for example) — predictable monthly cost, no per-token meter. Check the current spec and price page, since the models on offer rotate.
• Hourly GPU marketplaces (Vast.ai, RunPod) — rates float, so check the live board. Good for trialling a model size before committing to hardware.
• Your own box — a one-time hardware spend and no recurring bill at all, which is the whole argument for self-hosting taken to its conclusion.

For team deployments behind your firewall, see Ollama production deployment.

---

Ollama Setup for Bot Workloads

Default Ollama settings are tuned for single-user laptops. For a bot serving multiple concurrent users, change three things:

# /etc/systemd/system/ollama.service.d/override.conf
[Service]
Environment="OLLAMA_NUM_PARALLEL=4"            # 4 concurrent requests
Environment="OLLAMA_MAX_LOADED_MODELS=2"       # keep 2 models hot
Environment="OLLAMA_KEEP_ALIVE=24h"            # don't unload between messages
Environment="OLLAMA_HOST=0.0.0.0:11434"        # if bot is on different host

Then:

sudo systemctl daemon-reload
sudo systemctl restart ollama
ollama pull llama3.1:8b
ollama pull nomic-embed-text   # for RAG

Verify it's serving:

curl http://localhost:11434/api/generate -d '{
  "model": "llama3.1:8b",
  "prompt": "Say hello in 5 words"
}'

---

Slack Bot — Full Implementation

Step 1: Create the Slack App

1. Go to api.slack.com/apps → Create New App → From scratch
2. Socket Mode: enable it (you don't need a public URL)
3. OAuth Scopes (Bot Token Scopes): app_mentions:read, chat:write, channels:history, im:history, im:write, commands
4. Event Subscriptions → enable → subscribe to app_mention, message.im
5. Slash Commands → create /ai with description "Ask the local AI"
6. Install to Workspace — copy the Bot Token (starts with xoxb-)
7. Basic Information → App-Level Tokens → generate one with connections:write scope (starts with xapp-)

Step 2: The Bot Code

# slack_bot.py
import os
import re
import logging
from slack_bolt import App
from slack_bolt.adapter.socket_mode import SocketModeHandler
import ollama

logging.basicConfig(level=logging.INFO)
log = logging.getLogger("slack_bot")

OLLAMA_HOST = os.environ.get("OLLAMA_HOST", "http://localhost:11434")
MODEL = os.environ.get("OLLAMA_MODEL", "llama3.1:8b")

app = App(token=os.environ["SLACK_BOT_TOKEN"])
client = ollama.Client(host=OLLAMA_HOST)

SYSTEM_PROMPT = (
    "You are a helpful assistant for an engineering team in Slack. "
    "Be concise. Format code in fenced blocks. "
    "If you do not know, say so. Do not invent facts about the team."
)

def strip_mention(text: str) -> str:
    return re.sub(r"<@[A-Z0-9]+>", "", text).strip()

def ask_ollama(prompt: str, history: list[dict]) -> str:
    messages = [{"role": "system", "content": SYSTEM_PROMPT}]
    messages.extend(history[-10:])  # keep last 10 turns
    messages.append({"role": "user", "content": prompt})
    resp = client.chat(model=MODEL, messages=messages,
                       options={"temperature": 0.4, "num_predict": 800})
    return resp["message"]["content"].strip()

# Per-thread conversation memory (production: use Redis)
THREAD_HISTORY: dict[str, list[dict]] = {}

@app.event("app_mention")
def on_mention(event, say, client_slack):
    text = strip_mention(event["text"])
    thread_ts = event.get("thread_ts") or event["ts"]
    history = THREAD_HISTORY.setdefault(thread_ts, [])

    # Show "thinking" reaction
    client_slack.reactions_add(channel=event["channel"], name="hourglass_flowing_sand", timestamp=event["ts"])

    try:
        answer = ask_ollama(text, history)
        history.append({"role": "user", "content": text})
        history.append({"role": "assistant", "content": answer})
        say(text=answer, thread_ts=thread_ts)
    except Exception as e:
        log.exception("ollama error")
        say(text=f"Sorry — backend error: {e}", thread_ts=thread_ts)
    finally:
        client_slack.reactions_remove(channel=event["channel"], name="hourglass_flowing_sand", timestamp=event["ts"])

@app.command("/ai")
def slash_ai(ack, respond, command):
    ack()
    prompt = command["text"]
    if not prompt:
        respond("Usage: /ai <your question>")
        return
    answer = ask_ollama(prompt, [])
    respond(text=answer, response_type="in_channel")

@app.event("message")
def on_dm(event, say):
    # Only respond to DMs, ignore channel messages (handled by app_mention)
    if event.get("channel_type") != "im":
        return
    if event.get("subtype") == "bot_message":
        return
    history = THREAD_HISTORY.setdefault(event["channel"], [])
    answer = ask_ollama(event["text"], history)
    history.append({"role": "user", "content": event["text"]})
    history.append({"role": "assistant", "content": answer})
    say(answer)

if __name__ == "__main__":
    handler = SocketModeHandler(app, os.environ["SLACK_APP_TOKEN"])
    log.info("Slack bot starting...")
    handler.start()

That's a fully functional Slack bot. @bot what does our deploy script do? works in any channel where the bot is added, /ai slash command works anywhere, and DMs work too.

---

Discord Bot — Full Implementation

Step 1: Create the Discord App

1. Discord Developer Portal → New Application
2. Bot tab → Reset Token → copy it (this is your DISCORD_TOKEN)
3. Bot tab → enable Message Content Intent (required to read message text)
4. OAuth2 → URL Generator → scopes: bot, applications.commands → permissions: Send Messages, Read Messages, Add Reactions, Use Slash Commands → invite the bot to your server

Step 2: The Bot Code

# discord_bot.py
import os
import logging
import asyncio
import discord
from discord import app_commands
import ollama

logging.basicConfig(level=logging.INFO)
log = logging.getLogger("discord_bot")

OLLAMA_HOST = os.environ.get("OLLAMA_HOST", "http://localhost:11434")
MODEL = os.environ.get("OLLAMA_MODEL", "llama3.1:8b")

intents = discord.Intents.default()
intents.message_content = True
client_d = discord.Client(intents=intents)
tree = app_commands.CommandTree(client_d)
ollama_client = ollama.Client(host=OLLAMA_HOST)

SYSTEM_PROMPT = "You are a helpful assistant in Discord. Be concise. Use markdown."

CHANNEL_HISTORY: dict[int, list[dict]] = {}

async def ask_ollama_async(prompt: str, history: list[dict]) -> str:
    def _call():
        messages = [{"role": "system", "content": SYSTEM_PROMPT}]
        messages.extend(history[-10:])
        messages.append({"role": "user", "content": prompt})
        return ollama_client.chat(model=MODEL, messages=messages,
                                  options={"temperature": 0.4, "num_predict": 800})
    resp = await asyncio.to_thread(_call)
    return resp["message"]["content"].strip()

@client_d.event
async def on_ready():
    await tree.sync()
    log.info(f"Logged in as {client_d.user}")

@client_d.event
async def on_message(message: discord.Message):
    if message.author == client_d.user or message.author.bot:
        return
    # Respond on mention or DM
    is_dm = isinstance(message.channel, discord.DMChannel)
    is_mention = client_d.user in message.mentions
    if not (is_dm or is_mention):
        return

    prompt = message.content.replace(f"<@{client_d.user.id}>", "").strip()
    if not prompt:
        return

    history = CHANNEL_HISTORY.setdefault(message.channel.id, [])
    async with message.channel.typing():
        try:
            answer = await ask_ollama_async(prompt, history)
            history.append({"role": "user", "content": prompt})
            history.append({"role": "assistant", "content": answer})
            # Discord max message length is 2000 chars
            for i in range(0, len(answer), 1900):
                await message.reply(answer[i:i+1900], mention_author=False)
        except Exception as e:
            log.exception("ollama error")
            await message.reply(f"Backend error: {e}")

@tree.command(name="ai", description="Ask the local AI a question")
async def slash_ai(interaction: discord.Interaction, prompt: str):
    await interaction.response.defer()
    answer = await ask_ollama_async(prompt, [])
    for i in range(0, len(answer), 1900):
        if i == 0:
            await interaction.followup.send(answer[i:i+1900])
        else:
            await interaction.followup.send(answer[i:i+1900])

if __name__ == "__main__":
    client_d.run(os.environ["DISCORD_TOKEN"])

Mention the bot or DM it — it replies. /ai prompt slash command works server-wide. The 1900-char chunking handles long responses (Discord limits messages to 2000 chars).

---

Adding RAG over Team Docs

This is the killer feature. The bot answers questions from your internal docs, runbooks, and wikis instead of generic training data.

Step 1: Run ChromaDB

docker run -d -p 8000:8000 -v chroma-data:/chroma/chroma --name chroma chromadb/chroma:latest

Step 2: Index Your Docs

# index_docs.py
import os, glob, chromadb
import ollama

ollama_client = ollama.Client(host="http://localhost:11434")
chroma = chromadb.HttpClient(host="localhost", port=8000)
coll = chroma.get_or_create_collection(name="team_docs")

def embed(text: str):
    return ollama_client.embeddings(model="nomic-embed-text", prompt=text)["embedding"]

def chunk(text: str, size: int = 800, overlap: int = 100):
    chunks = []
    for i in range(0, len(text), size - overlap):
        chunks.append(text[i:i+size])
    return chunks

for path in glob.glob("./docs/**/*.md", recursive=True):
    with open(path) as f:
        content = f.read()
    for i, ch in enumerate(chunk(content)):
        coll.upsert(
            ids=[f"{path}:{i}"],
            documents=[ch],
            embeddings=[embed(ch)],
            metadatas=[{"source": path, "chunk": i}],
        )
print("Indexed all docs.")

Run python index_docs.py whenever your docs change. For automatic re-indexing on file changes, wrap it in watchdog.

Step 3: RAG-Enabled Chat Function

Replace the ask_ollama function in either bot:

def ask_with_rag(prompt: str, history: list[dict]) -> str:
    q_embedding = ollama_client.embeddings(model="nomic-embed-text", prompt=prompt)["embedding"]
    results = coll.query(query_embeddings=[q_embedding], n_results=5)
    context = "\n\n---\n\n".join(results["documents"][0])

    augmented_system = (
        SYSTEM_PROMPT + "\n\n"
        "Use the following context from internal team docs to answer. "
        "If the answer is not in the context, say so plainly.\n\n"
        f"CONTEXT:\n{context}"
    )
    messages = [{"role": "system", "content": augmented_system}]
    messages.extend(history[-6:])  # shorter history when context is large
    messages.append({"role": "user", "content": prompt})
    resp = ollama_client.chat(model=MODEL, messages=messages,
                              options={"temperature": 0.2, "num_predict": 800})
    return resp["message"]["content"].strip()

Now @bot what is our deploy procedure? returns answers grounded in your actual runbook. For deeper RAG tuning, see RAG local setup guide.

---

Slash Commands & Tools

Add structured commands beyond /ai:

# Slack
@app.command("/summarize")
def summarize_thread(ack, respond, command, client_slack):
    ack()
    channel = command["channel_id"]
    # Fetch last 50 messages
    history = client_slack.conversations_history(channel=channel, limit=50)
    text = "\n".join(m["text"] for m in history["messages"] if "text" in m)
    summary = ask_ollama(f"Summarize this Slack channel in 5 bullets:\n{text}", [])
    respond(summary, response_type="in_channel")

@app.command("/translate")
def translate(ack, respond, command):
    ack()
    args = command["text"].split(" ", 1)
    if len(args) != 2:
        respond("Usage: /translate <lang_code> <text>")
        return
    lang, text = args
    answer = ask_ollama(f"Translate to {lang}: {text}", [])
    respond(answer)

Discord equivalent uses @tree.command decorator with the same logic.

Useful slash commands seen in the wild:

• /summarize — collapse a long channel into bullets
• /translate — quick translation
• /sql — natural language to SQL
• /onboard — generate onboarding steps for a new team member
• /runbook — pull a runbook by name from RAG

---

Rate Limiting & Cost Control

Without limits, one curious user will lock up the bot for everyone. The pattern:

import time
from collections import defaultdict, deque

USER_REQUESTS: dict[str, deque] = defaultdict(deque)
USER_RATE_LIMIT = 10      # max requests
USER_RATE_WINDOW = 60     # per 60 seconds
GLOBAL_QUEUE_SIZE = 4     # match OLLAMA_NUM_PARALLEL

def check_rate_limit(user_id: str) -> bool:
    now = time.time()
    q = USER_REQUESTS[user_id]
    while q and q[0] < now - USER_RATE_WINDOW:
        q.popleft()
    if len(q) >= USER_RATE_LIMIT:
        return False
    q.append(now)
    return True

In the message handler:

if not check_rate_limit(event["user"]):
    say("You're sending too fast — try again in a minute.", thread_ts=thread_ts)
    return

For team-wide cost monitoring, track tokens per user in Redis or Postgres and alert on a daily budget you pick to match your hardware. The point is not to police usage — it is to notice a runaway retry loop, a script hammering the bot, or someone pasting a 200-page document into a thread before any of those starve everyone else off the GPU.

For multi-user rate-limiting at the Ollama layer itself, see Ollama rate limiting for multi-user setups.

---

Production Deployment

systemd unit (Linux)

# /etc/systemd/system/ai-bot.service
[Unit]
Description=Local AI Slack/Discord bot
After=network.target ollama.service

[Service]
Type=simple
User=botuser
WorkingDirectory=/opt/ai-bot
EnvironmentFile=/opt/ai-bot/.env
ExecStart=/opt/ai-bot/venv/bin/python slack_bot.py
Restart=always
RestartSec=10

[Install]
WantedBy=multi-user.target

sudo systemctl daemon-reload
sudo systemctl enable --now ai-bot
sudo journalctl -u ai-bot -f

Docker

FROM python:3.12-slim
WORKDIR /app
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
COPY . .
CMD ["python", "slack_bot.py"]

docker build -t ai-bot .
docker run -d --restart always --env-file .env --name ai-bot --network host ai-bot

Monitoring

Bare minimum: log every request, response length, and latency. Plug into Prometheus for real metrics:

from prometheus_client import Counter, Histogram, start_http_server

REQ = Counter("bot_requests_total", "Total requests", ["channel_type"])
LAT = Histogram("bot_latency_seconds", "End-to-end latency")

start_http_server(9100)

For full observability, see Ollama Prometheus + Grafana.

---

Pitfalls Worth Knowing Before You Ship

1. Slack rate limits are per-method, not global. If you call reactions_add on every message, you will hit 429s under load. Cache per-channel reaction state.
2. Discord intents must be enabled in the developer portal AND in code. Forgetting one or the other causes silent message-ignore behavior with no error.
3. Streaming responses break Slack. Slack does not support edit-as-you-stream. Buffer the full response then post once.
4. OLLAMA_KEEP_ALIVE matters more than any other setting. The default unloads the model after a few minutes idle, so the next message pays a cold start: re-reading the whole quantized model from disk into VRAM. That is a disk-bandwidth problem, so do the division — an 8.4 GB model over a SATA SSD's ~550 MB/s is on the order of fifteen seconds, over a 3.5 GB/s NVMe drive a few. Either way, the first person to ask a question after lunch is the one who notices. Set it to 24h.
5. Thread history grows unbounded in memory. The in-memory dict in the example code above never forgets a thread. Move it to Redis with a 24-hour TTL before real traffic arrives; otherwise memory grows with every conversation the bot has ever seen and you are restarting it on a schedule.
6. The model will roleplay as your CEO if asked. Add a system prompt rule: "Never impersonate specific people. Never claim to be a human."
7. Bot owners get DM'd weird stuff. Add an admin command /audit that lets you see anonymized log samples to spot abuse patterns.

---

Wrap-Up

A self-hosted team chat bot is one of the higher-leverage things you can build for a company. The marginal cost per message is electricity, non-technical staff get an AI helper without proprietary conversation leaving the building, and — through RAG — internal documentation stops being a thing nobody reads and becomes a thing people query by accident.

Start with the Quick Start and get a plain mention working. Then point RAG at your handbook. Then add the two slash commands your team asks for most; you will find out which two within a week of it going live, and it will not be the two you guessed. The part your security team cares about is architectural rather than a promise on a vendor's website: with Socket Mode and outbound websockets there is no inbound port, and with a local model there is no third party in the message path at all.

---

Want a deeper integration story? Read Ollama function calling and tool use for action-taking bots, or private OpenAI-compatible API to expose your bot's brain to other apps.