Local AI + Obsidian Canvas: Private Visual Thinking Maps

Obsidian Canvas ships with no AI features at all. You add them with three pieces: Ollama running a local model, the Copilot for Obsidian plugin pointed at http://127.0.0.1:11434, and Smart Connections generating vault embeddings through the same endpoint. With those in place, selecting a card and running "Ask Copilot on selection" puts the model's answer into a new card, and a short Python script against the open JSONCanvas format turns one card into three connected children on a hotkey. Nothing leaves the machine and nothing costs money.

Canvas is a freeform whiteboard inside your vault: every card is a note, a webpage, an image or a block of text, and every line you draw between two cards is structure you can use later. Without AI it stays a static brainstorming surface. Wired to a local model it becomes something else — you drop in a question, the model branches it into three cards positioned around the original; you select a cluster and ask for the thesis they collectively point at; you drag in a note and the embeddings surface the four most related notes in your library.

This guide is the exact setup: two Obsidian plugins, a small JSONCanvas script, Ollama with a 7B model, and Smart Connections for embeddings.

How do you get AI into Canvas in 20 minutes?

If you only want the working setup:

1. Install Ollama: brew install ollama (Mac) or follow the Linux/Windows installer.
2. Pull the model: ollama pull qwen2.5:7b (4.4 GB).
3. In Obsidian, install Copilot for Obsidian (community plugin) and Smart Connections.
4. Configure Copilot: model provider Ollama, base URL http://127.0.0.1:11434, model qwen2.5:7b.
5. Open a Canvas, select a card, run "Copilot: Ask AI on selection." The response opens in a new card.
6. Drop the JSONCanvas script (below) into ~/scripts/canvas-expand.py and bind it to a hotkey.

That's the working AI Canvas. The rest of this guide makes it actually useful for thinking.

Table of Contents

1. Why Canvas plus local AI is different
2. The stack
3. What hardware do you need?
4. Step 1 — Install Ollama and pull models
5. Step 2 — Configure Copilot for Obsidian
6. Step 3 — Smart Connections for vault embeddings
7. Step 4 — Programmatic Canvas expansion via JSONCanvas
8. Five workflows that work
9. Prompt patterns per node type
10. Pitfalls and performance
11. Cloud plugins vs the local stack
12. FAQ

---

Why is Canvas plus local AI different from a chat window?

Three things make this combination meaningful instead of gimmicky:

1. Spatial reasoning is a different skill from prose writing. When you can see five ideas on a canvas with lines drawn between them, you notice gaps and contradictions that linear writing hides. The model becomes a collaborator that respects spatial structure: it sees what's near what, what's connected to what, and answers in a way that fits.

2. Your vault is the corpus, not the model's training data. A model with access to your last six years of notes via Smart Connections produces dramatically more relevant suggestions than a generic LLM. When the AI proposes adding a card linking the current canvas to an essay you wrote in 2022, that's only possible because the embeddings know your library.

3. The output stays in the same file system as the input. A Canvas file is just JSON. Cards generated by the AI become real cards. Notes the AI proposes become real notes. Nothing lives behind a SaaS API. You can rsync the whole thing to an external drive and it still works.

Cloud plugins like Smart Composer and Text Generator can do versions of this — but every keystroke goes to OpenAI or Anthropic. With a local stack, the equivalent loop is: keystroke, your laptop, response. Period.

---

The Stack

Total cost: $0. All open source. Disk footprint: 4.4 GB + 2.0 GB + 274 MB from the Ollama library listings below, so about 6.7 GB of model weights, plus whatever Smart Connections' vector index adds for your vault.

The JSONCanvas spec is an open file format documented at jsoncanvas.org and is what makes programmatic canvas manipulation possible without reverse-engineering Obsidian internals.

---

What hardware do you actually need?

16 GB of RAM is the floor, and the thing that decides whether this feels snappy is memory bandwidth, not core count. You can work out roughly what your machine will do before installing anything, with two pieces of arithmetic.

How much memory the model occupies. A rule that holds well across GGUF quantizations:

model size (GB) ≈ 0.6 × parameters in billions      (at Q4_K_M)

Qwen 2.5 7B works out to about 4.2 GB, which lines up with the 4.4 GB Ollama lists for the default tag. Llama 3.2 3B works out to about 1.8 GB against a listed 2.0 GB. On a 16 GB machine, 4.4 GB of weights plus Obsidian plus a browser is comfortable; on 8 GB it is not, and you will watch the model get paged out between expansions.

How fast a card expansion can possibly be. Every generated token requires reading the whole model out of memory once, so:

tokens/sec ≤ memory bandwidth (GB/s) ÷ model size (GB)

Plug in each vendor's published bandwidth figure and you get a hard ceiling for Qwen 2.5 7B at 4.4 GB:

These are arithmetic upper bounds, not measurements. Real output lands well below them — attention, sampling and framework overhead all cost time the formula ignores. Use them for the comparison they support: a three-card expansion is a few hundred tokens, so a machine near the top of that table finishes it in a couple of seconds and a machine near the bottom takes tens of seconds. That difference is what makes the iterative Canvas loop pleasant or annoying, and it is why Apple Silicon does well here despite modest core counts — unified memory gives a laptop bandwidth that x86 laptops reach only with a discrete GPU.

If you land in the bottom half of that table, drop to llama3.2:3b for expansions (1.8 GB, so roughly 2.4× the ceiling) and keep the 7B for cluster summaries where you are willing to wait.

---

Step 1 — Install Ollama and pull models

Same commands as every local AI guide, but the model selection matters here:

# Mac
brew install ollama
brew services start ollama

# Linux
curl -fsSL https://ollama.com/install.sh | sh
sudo systemctl enable --now ollama

# Pull the canvas models
ollama pull qwen2.5:7b           # Primary reasoning (4.4 GB)
ollama pull llama3.2:3b          # Fast lightweight tasks (2.0 GB)
ollama pull nomic-embed-text     # Embeddings for Smart Connections (274 MB)

Verify Ollama is reachable from Obsidian:

curl http://127.0.0.1:11434/api/tags

If you get JSON listing the models, you are ready for the plugin layer. For broader Ollama setup, see the Mac local AI setup guide or the Linux local AI setup.

---

Step 2 — Configure Copilot for Obsidian

Open Obsidian. Settings > Community plugins > Browse > search "Copilot for Obsidian" by Logan Yang. Install and enable.

Configuration:

1. Default chat model: Add a custom model with these settings:

   • Provider: OllamaServer
   • Model: qwen2.5:7b
   • Base URL: http://127.0.0.1:11434
   • Display name: Local Qwen 7B

2. Embedding model: Add another custom entry:

   • Provider: OllamaServer
   • Model: nomic-embed-text
   • Base URL: http://127.0.0.1:11434

3. Default mode: Set "Default chat mode" to "QA" so questions automatically use vault context.

4. Hotkey: In Obsidian's hotkey settings, bind "Copilot: Open Copilot Chat" to something fast — Cmd+Shift+L and Cmd+Shift+K for "Copilot: Quick action — Ask Copilot" are both free by default on macOS.

The critical test: open any note, select a sentence, hit your "Ask Copilot" hotkey. If a response streams in within a few seconds, the wiring is correct. If you get a connection error, Ollama is not running — brew services restart ollama or ollama serve in a terminal.

---

Step 3 — Smart Connections for vault embeddings

Smart Connections is the plugin that makes the AI aware of your whole vault, not just the current canvas.

Install: Community plugins > Browse > search "Smart Connections" by Brian Petro. Install, enable.

Configuration:

1. Embedding model: Ollama: nomic-embed-text with base URL http://127.0.0.1:11434.
2. Indexing: the first run embeds every note in your vault, and it is the slow one — later runs only re-embed changed files. How long it takes scales with note count and with the same bandwidth ceiling as above, so plan to start it and go and do something else. It runs in the background; Obsidian stays usable.
3. Folder filters: Exclude folders that should never surface in AI context — templates/, daily-journal/, anything personal.

Once indexed, the Smart Connections side panel shows the most semantically similar notes for whatever you have open. On a canvas, this is gold: select any card, the panel shows your five most-related notes from anywhere in the vault. Drag any of them onto the canvas to attach.

For broader vault-AI patterns, see the local AI + Obsidian integration guide.

---

Step 4 — Programmatic Canvas expansion via JSONCanvas

Copilot handles ad-hoc card-level AI well, but for repeatable workflows you want a script. Obsidian Canvas files are valid JSON conforming to the JSONCanvas spec. We can read them, generate new nodes, and write them back.

Save this as ~/scripts/canvas-expand.py:

#!/usr/bin/env python3
"""Expand the most recently edited card in a canvas into 3 child cards."""
import json
import sys
import uuid
import subprocess
from pathlib import Path

CANVAS_PATH = Path(sys.argv[1])
OLLAMA_MODEL = "qwen2.5:7b"

data = json.loads(CANVAS_PATH.read_text())

# Find the most recent text card by ID alphabetical (canvas IDs sort by time)
text_nodes = [n for n in data["nodes"] if n["type"] == "text"]
if not text_nodes:
    sys.exit("No text nodes in canvas")

source = sorted(text_nodes, key=lambda n: n["id"])[-1]
prompt = f"""Read the following idea card. Generate exactly THREE short follow-up
ideas, each one a single tight paragraph. Output them as a JSON array of strings.
Output ONLY the JSON array, no preamble.

Card content:
{source['text']}"""

result = subprocess.run(
    ["ollama", "run", OLLAMA_MODEL, prompt],
    capture_output=True, text=True, timeout=120
)
new_texts = json.loads(result.stdout.strip())

# Position three children to the right of the source
sx, sy = source["x"], source["y"]
sw, sh = source.get("width", 250), source.get("height", 60)

for i, txt in enumerate(new_texts):
    new_id = uuid.uuid4().hex[:16]
    data["nodes"].append({
        "id": new_id,
        "type": "text",
        "text": txt,
        "x": sx + sw + 80,
        "y": sy + (i - 1) * (sh + 40),
        "width": sw,
        "height": sh,
        "color": "4",
    })
    data.setdefault("edges", []).append({
        "id": uuid.uuid4().hex[:16],
        "fromNode": source["id"],
        "fromSide": "right",
        "toNode": new_id,
        "toSide": "left",
    })

CANVAS_PATH.write_text(json.dumps(data, indent=2))
print(f"Added 3 cards to {CANVAS_PATH.name}")

Make it executable: chmod +x ~/scripts/canvas-expand.py. Bind it to a system-level hotkey using Hammerspoon (Mac), AutoHotkey (Windows), or sxhkd (Linux). The hotkey runs the script against the active canvas file.

The result: select a card, hit the hotkey, three connected child cards appear. This is the piece of the stack that changes how Canvas feels, because it removes typing from the loop.

For other Canvas automation patterns, see the JSONCanvas spec on GitHub.

---

Which workflows are worth building?

Five patterns that earn their setup cost over plain Canvas.

Workflow 1 — Topic exploration

Drop a single card with a topic. Run the canvas-expand script three times and you have nine child cards. Read them, delete the bad ones, expand the good ones again. A few rounds of that produces a map of the territory in the time it would take to type out one branch of it — your hands stop being the bottleneck.

Workflow 2 — Cluster summarization

Select multiple cards on a canvas. Use Copilot's "Ask Copilot on selection" with the prompt template:

"Read the following cards. Identify the underlying thesis they collectively point toward. Output a single paragraph that could serve as the summary card for this cluster."

The new card opens. Drag it to the visual center of the cluster, draw lines from the cluster cards to it. You now have an emergent thesis you didn't articulate yourself.

Workflow 3 — Note-suggestion drag

Select a cluster of cards on the canvas. Open Smart Connections side panel. The five most-related vault notes appear. Drag the relevant ones onto the canvas. The model just connected your current thinking to your past thinking.

Workflow 4 — Counter-argument generation

For any card containing a claim, run the prompt:

"Generate the strongest possible counter-argument to this card, in two sentences. Then generate the steelman version of that counter-argument in two more sentences."

Two cards appear. Connect them to the original. Your canvas now has built-in dialectical structure.

Workflow 5 — Outline export

Select an entire canvas (Cmd+A) and run:

"These cards form a knowledge map. Convert them into a hierarchical outline in Markdown, ordered for a long-form essay. Use the spatial layout (cards close to each other belong in the same section) and the edges (arrows indicate logical flow) as input."

The output is a usable outline you can drop into a new note. The cards on the canvas remain as the source of truth; the outline is just an alternate view.

---

What should you prompt for each card type?

Different card types call for different prompts. Start from these templates:

Question card → Three-answer expansion

"Treat the following as a question. Generate three substantively different answers, each in two sentences. Output as JSON array of strings."

Claim card → Evidence and challenge

"Treat this as a claim. Generate two cards: one with the strongest piece of supporting evidence, one with the strongest challenge. Output as JSON: {"evidence": "...", "challenge": "..."}."

Quote card → Five-why expansion

"Treat this as a quote. Apply five-whys to surface the underlying assumption. Output as a JSON array of five strings, each one tightening the question."

List card → Categorization

"Cluster the items in this list into 3-5 named groups. Output as JSON: [{"group": "name", "items": [...]}, ...]."

Definition card → Examples and counterexamples

"Generate two cards: one card with three concrete examples that fit this definition, one card with two near-misses that do not. Output as JSON."

Save these in your vault at templates/canvas-prompts.md and paste them into Copilot as needed. The specific instruction that matters in all of them is "output as JSON" — it is what lets the expansion script parse the result instead of guessing where one idea ends and the next begins.

---

What goes wrong, and how fast is it really?

Pitfall 1: Long contexts kill performance. A canvas with 60+ text cards passed wholesale to a 7B model produces slow, mediocre output. Constrain context: only the selection plus its first-degree neighbors.

Pitfall 2: Cards that contain entire essays. A 2,000-word card pasted into a prompt eats the model's working memory. Either summarize the card first, or explicitly tell the model "summarize this card before reasoning about it."

Pitfall 3: JSONCanvas position math. When the script generates child cards, hard-coded x/y offsets work for the first three cards but pile up if you run the script repeatedly. Add a layout pass that scans existing cards and finds free space before placing.

Pitfall 4: Smart Connections re-indexing on every model swap. If you change the embedding model, you must re-index the entire vault. nomic-embed-text is fine; do not switch lightly.

Pitfall 5: Network restrictions block plugins. Some plugins phone home for telemetry on first run. If you're trying to run fully offline, audit each plugin's network calls. The two plugins recommended here (Copilot for Obsidian, Smart Connections) work entirely against your local Ollama once configured.

---

Cloud plugins vs the local stack

The honest trade: a hosted frontier model is bigger than anything you will run on a laptop, and on long synthesis prompts it will produce better prose. What it cannot do is stay on your machine or run on a plane. If your canvases contain client work, research in progress, or anything you would not paste into a web form, that is the whole argument.

---

Where this goes next

Once the AI Canvas is working, the natural extensions:

• Local AI + Obsidian: full vault Q&A with AnythingLLM — chat with your entire library, not just one canvas.
• Best Ollama clients — alternatives if Copilot for Obsidian doesn't fit.
• Local AI privacy guide — the broader case for self-hosted thinking tools.

A canvas respects spatial structure. A local model respects privacy. Together they give you a thinking environment no commercial product ships, because the value comes from the combination of your specific vault and your specific way of arranging things on a plane — which is exactly the thing nobody can package and sell you.

---

FAQ

Does Obsidian Canvas work with local AI out of the box?

No. Canvas itself has no AI features. You add them by installing Copilot for Obsidian and pointing it at a local Ollama instance; the plugin's commands, including "Ask Copilot on selection", then work on Canvas cards exactly as they do on notes. Smart Connections adds vault-wide semantic search on top. Neither capability exists in Obsidian by default.

Which local model works best for Canvas brainstorming?

Qwen 2.5 7B is a good default: it is small enough for the iterative card-by-card loop, produces tight short paragraphs, and follows "output as a JSON array" reliably enough for the expansion script to parse. Llama 3.1 8B is a close substitute. Below about 7B, structured-output instructions start getting ignored, which breaks the script; above about 13B the memory-bandwidth ceiling makes each expansion slow enough that you stop using it.

How does the JSONCanvas script decide where to put new cards?

It reads the canvas JSON, picks the most recently created text card (Canvas IDs sort roughly by creation time), and places children to the right of it stacked vertically. That works for one round of expansion. For repeated rounds you need a layout pass that checks for collisions with existing nodes before placing, otherwise stacked expansions overlap. The JSONCanvas spec documents the full coordinate system.

Can the AI use my whole vault as context, not just the open canvas?

Yes, through Smart Connections. It embeds every note using nomic-embed-text running locally via Ollama, then surfaces semantically similar notes for whatever you have open. On a canvas that means selecting a card and seeing your most-related vault notes in the side panel, ready to drag in. The vectors are stored in a file inside your vault's plugin folder, so they travel with the vault and never touch a server.

How long does Smart Connections take to index a vault?

Longer than you expect on the first run and negligible after that — only changed notes get re-embedded. The duration scales with note count and with your machine's memory bandwidth, the same ceiling described in the hardware section. Start it, leave it, come back. If you want it shorter, exclude folders you would never want surfaced anyway before you start.

Is this safer than using a hosted model through Smart Composer?

Architecturally, yes. In the local stack the prompt and every retrieved note stay on your machine: Ollama runs the model, Smart Connections holds the embeddings, Copilot is only the UI. Cloud plugins transmit the prompt plus any retrieved context to a provider's servers. For a journal, client work or research in progress, local is the only arrangement where the content stays under your control.

Can I use Canvas AI features offline?

Yes. Once the models are pulled and Smart Connections has indexed the vault, the whole stack works with the network off — expansions, questions and vault-aware suggestions all keep running. That is the one thing cloud plugins cannot do at any price.

How do I bind the canvas-expand script to a hotkey?

Use Hammerspoon on macOS, AutoHotkey on Windows, or sxhkd on Linux — each lets you map a global shortcut to a shell command. The command needs the path of the active canvas file; the simplest reliable approach is a small helper that reads Obsidian's last-active-file metadata and passes it through. If that feels like too much plumbing, run the script from a terminal with the filename as an argument until the workflow proves itself.