Chat With Your PDFs Locally: Private ChatPDF Alternatives That Actually Work

Short answer: install AnythingLLM Desktop (free; macOS, Windows, Linux) plus Ollama with one small model — qwen3:4b is a 2.5GB download — and you can chat with any PDF fully offline in about ten minutes. AnythingLLM ships its own CPU embedder (all-MiniLM-L6-v2, a ~25MB download with a ~2GB RAM floor, per its docs) and its own vector database (LanceDB), so nothing about your document ever leaves your machine. Want more control or multi-user access? Ollama + Open WebUI is the power stack. Want the least setup for a single file? LM Studio lets you drag a PDF straight into a chat.

That is the whole recommendation. The rest of this page is the part the SaaS tools don't tell you: the exact setup for each app, with commands checked against each project's current docs (August 2026), the retrieval defaults that quietly decide whether your answers are any good, and the honest list of PDFs this approach handles badly.

---

Why Local Beats ChatPDF Uploads {#why-local}

A ChatPDF-style web tool works by uploading your document to someone else's server. A local stack does the same job on your hardware, free, with no page caps, no daily question limits, and no third party holding your contracts.

The web tools are genuinely convenient, and for a public PDF nobody cares about, they're fine. The case for local is specific:

• Privacy that doesn't depend on a policy page. Contracts, medical records, financial statements, client work, unpublished research — the whole point is that these shouldn't sit on a vendor's server. Locally, the document is parsed, indexed, and queried on your disk. There is no retention policy because there is no upload.
• No metering. Hosted PDF-chat free tiers cap pages, file sizes, or questions per day, and the caps change at the vendor's convenience. A local stack processes a 900-page PDF with the same enthusiasm as a 9-page one — it just takes longer to index.
• It's actually free. Every tool on this page costs $0 for this use: AnythingLLM and Open WebUI are open source, and LM Studio has been free for both home and work use since July 2025, per its own announcement.
• It works on a plane. After the first-run model downloads, none of this needs a connection.

The trade: you do ten minutes of setup once, and you supply the computer. Any reasonably modern 8GB+ machine qualifies — details in the hardware section below.

---

The Three Picks at a Glance {#picks-at-a-glance}

One line each: AnythingLLM is the easiest real document workspace, Open WebUI is the most powerful and tunable, LM Studio is the fastest way to ask one PDF one question.

All three are actively maintained — versions checked on their GitHub releases and official sites, August 2026:

Sources: AnythingLLM docs and GitHub releases; Open WebUI docs, GitHub releases, and backend config defaults; LM Studio docs (all checked August 2026).

Notice the first two default to the same embedding model — the practical difference between them is not retrieval quality, it's how they fit your setup. For a deeper head-to-head, see our AnythingLLM vs Open WebUI comparison.

---

Pick 1: AnythingLLM — the Easiest Full Setup {#anythingllm}

AnythingLLM Desktop is a free app for macOS (Apple Silicon + Intel), Windows (x64 + ARM), and Linux that bundles the entire PDF-chat pipeline: embedder, vector database, document parser, and chat UI. You add Ollama for the language model and you're done.

What makes it the default recommendation is what you don't have to set up. Per the official docs:

• Embedder: built in. It ships with all-MiniLM-L6-v2 — a ~25MB model downloaded on first use that runs on CPU with a documented minimum of about 2GB RAM. You never think about it.
• Vector database: built in. LanceDB runs inside the app. No Docker, no separate service — your vectors sit in the app's data folder like any other file.
• Privacy: the design goal. The project's own line is that your models, documents, and chat history stay on your machine — "nothing phones home."

Setup, start to finish

1. Install Ollama from ollama.com, then pull a chat model:

ollama pull qwen3:4b

That's a 2.5GB download (per the Ollama library) that runs on most 8GB machines. Got 16GB+ of RAM or a GPU? Take qwen3:8b (5.2GB) — noticeably better answers.

1. Install AnythingLLM Desktop from anythingllm.com and open it. In onboarding, pick Ollama as your LLM provider — it detects the local server — and accept the defaults for embedder and vector database (that's the built-in pair above).

2. Create a workspace and drag your PDF into it, then confirm the embed step. First indexing of a big document takes a minute or two on CPU; it happens once, not per question.

3. Ask. Answers cite the source chunks they drew from, so you can check the model's homework — the single most underrated feature in this category.

The workspace model is the quiet win: drop in the whole project folder — the contract and the amendment and the meeting notes — and every chat in that workspace retrieves across all of it. That's a real document library, not a one-off trick. When one PDF becomes forty and you want the same idea at team scale, our private AI knowledge base guide covers the graduation path, and the full AnythingLLM setup guide goes deeper on configuration than this overview.

One honest caveat from the docs: the built-in embedder is "primarily trained on English documents." For German contracts or Japanese papers, retrieval quality drops — swap the embedder to Ollama's nomic-embed-text (a 274MB pull with 81M+ downloads, per its Ollama model card) in settings. Do it before you index, because changing embedders means re-embedding the workspace.

---

Pick 2: Ollama + Open WebUI — the Power Stack {#open-webui}

Open WebUI gives you the same offline PDF chat with far more control: persistent knowledge bases you reference mid-chat with #, folder syncing, swappable vector databases, OCR-capable extraction engines, and hybrid search. Cost of admission: it runs via Docker or pip, not a desktop installer.

With Ollama already installed from Pick 1, one command starts it:

docker run -d -p 3000:8080 -v open-webui:/app/backend/data --name open-webui ghcr.io/open-webui/open-webui:main

Then open http://localhost:3000. (No Docker? pip install open-webui then open-webui serve works on Python 3.11/3.12 — the docs note 3.13 is not yet supported. Full walkthrough, including the image that bundles Ollama in the same container: our Ollama + Open WebUI Docker guide.)

The PDF workflow

Per the current Open WebUI docs:

1. Workspace → Knowledge → Create. Name the collection ("Lease documents", "Thesis sources").
2. Upload your PDFs into it.
3. In any chat, type # and pick the collection — the model retrieves from it for that conversation. Or attach the collection permanently to a custom model under Workspace → Models, which gives you a purpose-built "ask my documents" assistant.

Two features push this past the desktop apps. Sync Directory mirrors a local folder into a knowledge base incrementally — new, changed, and deleted files only — which is how a documents folder stays queryable without babysitting. And the docs list support for 13 vector databases and 8 extraction engines (Tika, Docling, Mistral OCR, and others), which matters the moment you hit scanned PDFs.

The defaults that decide your answer quality

These are the actual out-of-the-box values from the project's backend configuration on GitHub (checked August 2026):

The practical tuning advice, in order of payoff: raise Top-K from 3 to 5-10 for dense documents (three paragraphs of context is thin for a 200-page PDF); switch the embedding model to nomic-embed-text served by Ollama if your documents aren't English (re-index afterward — same rule as AnythingLLM); and enable hybrid search when you query exact strings — invoice numbers, clause references, proper nouns — because keyword matching catches what semantic similarity misses. All of it lives in the admin settings under Documents. For what these knobs actually do, our local RAG setup guide walks the same pipeline by hand.

---

Pick 3: LM Studio — Fastest to a First Answer {#lm-studio}

If the job is "one PDF, three questions, right now," LM Studio is the shortest path: install the app, download a model in its built-in browser, drag the file into the chat. It accepts .pdf, .docx, and .txt attachments directly.

The interesting part is how it uses the file, because LM Studio's docs describe a dual mode the other apps don't lead with:

• Document fits in the model's context window? The entire text goes into the conversation. No retrieval, no chunks, no misses — for a 10-page PDF with a modern long-context model, this is strictly better than RAG, because the model genuinely sees every word.
• Too long to fit? It falls back to retrieval, attempting — in the docs' own words — to "fish out relevant bits of a very long document" for the model. The docs are refreshingly honest that this "sometimes works really well, but sometimes it requires some tuning and experimentation."

That candor matches our reading of the whole category, so treat LM Studio as the single-document tool: attachments live in the chat where you dropped them, and there are no persistent collections to build a library from — that's AnythingLLM's and Open WebUI's job. Since July 2025, LM Studio's terms make it free at home and at work, which removed the old licensing asterisk for using it on office documents.

---

What About GPT4All and Msty? {#gpt4all-msty}

GPT4All — for years the default answer to this exact question — has gone quiet: its last release, v3.10.0, shipped February 25, 2025, with nothing since (per its GitHub releases, checked August 2026). It still works; we just can't recommend starting there in 2026.

Credit where due: GPT4All's LocalDocs got this category right early — point it at a folder, it indexes with Nomic's on-device embedding models, and a Sources button under each answer shows which files it drew from. If you already run it happily, nothing breaks tomorrow. But recommending a dormant app over three actively maintained ones would be doing you a disservice, when PDF extraction and retrieval quality are exactly the parts still improving release over release elsewhere.

Msty is the other name you'll meet — the polished desktop app from the Ollama ecosystem, which we covered in Msty vs Ollama vs LM Studio. In 2026 it repositioned as a multi-product platform (Studio, Go, Nexus, Stack), and the piece that matters for this page — "Knowledge Stacks," its governed document-retrieval component — is listed as coming soon on msty.ai as of August 2026. Promising, unfinished. Re-evaluate when Stack ships; don't plan tonight's PDF session around it.

---

How PDF Chat Actually Works — and Where It Fails {#how-it-works}

Every app on this page does the same five things: extract the PDF's text, split it into chunks, turn each chunk into a vector with an embedding model, store those in a vector database, and — per question — retrieve the few most similar chunks and hand them to the model as context. Knowing this pipeline is how you debug bad answers.

The failure modes all fall out of that design. The honest list:

• Scanned PDFs are invisible. A scan is a photo; if there's no text layer, extraction yields nothing and the app indexes an empty document — then confidently answers from nothing. Fix: OCR first. Open WebUI's selectable extraction engines (Docling, Tika, Mistral OCR, per its docs) are the most integrated route; for the desktop apps, run OCR before importing.
• "Summarize this document" is the wrong tool. Retrieval fetches the top few chunks — a summary needs all of them. Ask targeted questions instead ("what are the termination conditions?"), or use a proper map-reduce workflow: our local document summarizer guide covers that pattern.
• Tables come out scrambled. PDF table extraction flattens rows and columns into word soup, and financial tables are the worst case. Verify any retrieved number against the page itself — this is precisely what citations are for.
• Cross-page reasoning is thin at default settings. With 3 retrieved chunks of ~1,000 characters (Open WebUI's defaults), the model sees roughly one page of context per question. Answers that require synthesizing many sections need a higher Top-K, better questions, or both.
• Non-English documents underperform silently. The default embedder in both AnythingLLM and Open WebUI is the same English-leaning all-MiniLM-L6-v2, so retrieval — not the chat model — becomes the bottleneck. Swap the embedder (the nomic-embed-text route from the sections above) and re-index.
• The model can still make things up. Retrieval narrows hallucination, it doesn't eliminate it. The rule that makes local PDF chat safe for real work: never quote an answer you haven't clicked the citation on.

None of this is a reason to go back to the cloud — hosted PDF tools sit on the same RAG pipeline and inherit the same failures. Local just lets you see, and fix, the knobs.

---

Hardware You Actually Need {#hardware}

Almost any machine from the last five years works: the embedding side needs ~2GB of RAM on CPU (per AnythingLLM's docs), and a capable 4B chat model is a 2.5GB download. A GPU makes answers faster and lets you run smarter models — it is not required.

Rough tiers, with model sizes from the Ollama library (August 2026):

Two notes. Indexing is a one-time cost per document — a few minutes for a large PDF on CPU — so a slow machine hurts once, not on every question. And for this workload, context beats cleverness: a mid-size model that can hold more retrieved chunks often does better than a bigger model fed less. If you have a GPU and want the best model it can hold, our best LLMs for 8GB VRAM picks are kept current.

---

Verdict {#verdict}

AnythingLLM for most people. Open WebUI when you want control or multi-user. LM Studio for one-off questions. All three: free, offline after setup, no upload caps, and your documents never leave the machine.

The decision tree in full:

1. You want PDF chat working today with the least reading: AnythingLLM Desktop + Ollama + qwen3:4b. Built-in embedder and vector database mean two installs and zero configuration.
2. You'll build a real document library, tune retrieval, or share with others: Ollama + Open WebUI. Folder sync, #-referenced knowledge bases, and every knob exposed — start by raising Top-K from its default of 3.
3. You have one PDF and one question: LM Studio. Drag, drop, ask — and for short documents its full-context mode literally reads every word, which no retrieval setup can promise.
4. Skip GPT4All for new setups (dormant since early 2025), and check back on Msty when its Knowledge Stacks component actually ships.

The pattern behind all of it: "chatting with a PDF" stopped being a SaaS product the moment a 2.5GB local model got good enough to answer from retrieved context. What's left of the hosted tools' pitch is convenience — and ten minutes of setup buys that back permanently.

---

Sources {#sources}

• AnythingLLM docs — built-in embedder — all-MiniLM-L6-v2, ~25MB, CPU, ~2GB RAM minimum, English-primary note
• AnythingLLM docs — vector databases — LanceDB as the built-in default; desktop page — free, macOS/Windows/Linux, local-first claims
• AnythingLLM GitHub releases — v1.15.0 (June 2026)
• Open WebUI docs — Knowledge — collections, # referencing, Sync Directory, vector database and extraction-engine support
• Open WebUI docs — Quick Start — Docker and pip install commands, Python version support
• Open WebUI GitHub — v0.11.0 release (July 2026) and backend config defaults (embedding model, ChromaDB, chunk size 1,000 / overlap 100, Top-K 3, hybrid search off)
• LM Studio docs — Chat with Documents — .pdf/.docx/.txt attachments, full-context vs RAG behavior; free-for-work announcement (July 8, 2025)
• GPT4All GitHub releases — v3.10.0 (Feb 25, 2025) as the latest release, checked August 2026; LocalDocs docs — Nomic on-device embeddings, Sources button
• nomic-embed-text and qwen3 Ollama model cards — download sizes and pull counts
• msty.ai — Studio/Go/Nexus/Stack product split; Knowledge Stacks listed "coming soon" (checked August 2026)

---

FAQ {#faq}