Best Ollama Embedding Models Compared for Local RAG

Pull nomic-embed-text unless you have a specific reason not to: it is the most-pulled embedding model in the Ollama library, one of the smallest, and it accepts chunks up to 8,192 tokens where its closest rival truncates at 512. Choose bge-m3 when you need 100+ languages, qwen3-embedding when you want a 32K context and adjustable output dimensions. And whichever you pick, read the prefix section before you index anything — Ollama's /api/embed applies no prompt template, and several of these models were trained expecting one.

Embedding models are quietly one of the biggest things people use Ollama for: nomic-embed-text sits near the top of the entire library by pull count, ahead of most chat models. Yet nearly every "best embedding model" list rehashes MTEB leaderboard scores measured outside Ollama, on hardware you do not have, against corpora nothing like yours. This page does something narrower and more useful — it lays out the specifications that actually constrain a local RAG pipeline (context limit, dimensions, licence, prompt format), then shows you how to run the only benchmark that matters, which is one over your own documents. New to embeddings entirely? Our local embeddings guide covers the concepts first.

---

Which one should I pull?

Default to nomic-embed-text; deviate only for a specific reason. Here is the whole article in one table:

The uncomfortable truth about this category: on clean English prose, bi-encoder retrieval quality is not where these models meaningfully differ. Their published MTEB averages sit within a few points of one another, and a few points of MTEB rarely survives contact with a real corpus. What does differ — hard, predictably, and in ways no leaderboard captures — is everything in the table below.

---

What actually separates them?

Every figure here comes from the model's Ollama library page or its official model card, both linked in the sources. These are the constraints that decide whether a pipeline works, and each one is checkable in about thirty seconds.

Sizes, parameter counts and dimensions as listed on each model's ollama.com library page and official card. "Max input" is the num_ctx value in the Modelfile Ollama ships, which for two of these models is lower than the upstream model supports — run ollama show <model> after pulling to confirm what you actually got.

Read the "max input" column carefully, because it is the most common way a local RAG pipeline goes quietly wrong. A typical splitter emits 300-500-token chunks. Ollama defaults to "truncate": true, which means an over-length chunk is silently cut and embedded anyway — no error, no warning, just a vector representing the first part of your text. On all-minilm that ceiling is 256 tokens, so most real RAG chunks lose their tail before they are ever indexed. On mxbai-embed-large it is 512, which a long chunk will clip.

---

Do I need prompt prefixes?

Several of these models were trained with task-specific prompt templates, and Ollama's /api/embed does not add them for you — it embeds exactly the string you send. If you have come from sentence-transformers, where the template is often applied for you, this is the trap.

Each model card documents its own format:

The exact strings, straight from the model cards:

# embeddinggemma (Google's card) — prefix BOTH sides
query:    task: search result | query: {your question}
document: title: none | text: {your chunk}

# nomic-embed-text (Nomic's card) — different prefix each side
query:    search_query: {your question}
document: search_document: {your chunk}

# qwen3-embedding (Qwen's card) — instruction before the query only
Instruct: Given a web search query, retrieve relevant passages that answer the query
Query: {your question}

Two rules fall out of this, and they matter more than which model you picked:

• Whatever scheme you index with, query with the same scheme. Mixing prefixed documents with bare queries is the silent killer — nothing errors, similarities just come out subtly wrong, and you conclude the model is bad.
• If retrieval quality seems inexplicably poor, check prefixes before swapping models. A "bad" embedding model is very often a good model being fed a format it was never trained on. It costs five minutes to rule out and it is free to fix.

---

How much VRAM and disk will this cost?

Almost nothing — and that is the point. The embedding model is a rounding error next to the chat model it feeds. Two numbers are worth computing before you commit.

Memory. These models load at roughly their download size, so the whole lineup sits between 46MB and 1.2GB. Every one of them coexists with a 7B-class chat model on an 8GB card without a fight, so spend your VRAM planning on the generation side instead (our 8GB VRAM model picks cover that half of the stack).

Vector storage is the one that surprises people at scale, and it is pure arithmetic:

bytes = number of chunks × dimensions × 4      (float32)

Dimension count is therefore a storage decision as much as a quality one — and on Matryoshka-trained models like nomic you can shrink it deliberately, which is what the dimensions parameter in the setup section does.

Throughput is the number this page will not invent for you. What you can reason about safely is the ratio: embedding is a single forward pass per chunk — not one per output token, as generation is — and its cost scales roughly with parameter count. all-minilm at 23M parameters against bge-m3 at 567M is a ~25x spread, and on a CPU-only indexer that spread is the difference between minutes and hours for the same corpus. Query-time cost is negligible for all six; it vanishes next to the seconds your chat model spends generating. If you want real figures for your own box, the DIY eval below produces them in about fifteen lines of Python.

---

The six models, one by one

nomic-embed-text — the default, and deservedly so. Smallest serious download, and long-chunk capable: the library page lists a 2K context, but the shipped Modelfile raises num_ctx to 8192, matching the 8,192-token max sequence length on Nomic's v1.5 card (run ollama show nomic-embed-text to see it). That card reports 62.28 MTEB at 768 dims, degrading gracefully via Matryoshka training to 61.04 at 256 dims — relevant because Ollama's dimensions parameter lets you store the smaller vectors and halve your storage bill. A multilingual sibling exists (nomic-embed-text-v2-moe, ~100 languages) but caps at 512 tokens per input, so for multilingual work bge-m3 is the easier reach.

mxbai-embed-large — mixedbread.ai's BERT-large-class model, and the strongest published MTEB average in this lineup: 64.68 across the 56 English MTEB datasets, per its card, which at release made it the top open model of its size. The catch is the 512-token context. Standard RAG splitters emitting 300-500-token chunks sit uncomfortably close to that ceiling, and Ollama's default truncate: true will silently cut anything over it. Short-chunk pipelines only — and if you go this route, assert your chunk lengths in code rather than trusting the splitter.

bge-m3 — BAAI's multilingual workhorse: 100+ languages, an 8,192-token window, MIT licence, and the "M3" trio of dense, sparse and multi-vector retrieval — though through Ollama's /api/embed you get the dense 1024-dimension vectors only. Years after release it remains the safe multilingual default, and it pairs naturally with bge-reranker-v2-m3 from the same family (see the verdict section). If your corpus is not all-English, start here.

qwen3-embedding:0.6b — the newest serious entry. A 32K context window (by far the largest here), instruction-aware prompting, output dimensions adjustable from 32 to 1024, and 100+ languages. Its card reports 70.70 on MTEB English v2 and 64.33 on multilingual v2 for this 0.6b size, and Qwen reports the same family's 8B variant at 70.58 on the MTEB multilingual leaderboard. Note that Ollama ships the 0.6b tag quantized to Q8_0 while the card's scores were measured at BF16, so do not expect the published number to transfer exactly; 4b and 8b tags exist when you want the bigger versions of the same recipe.

all-minilm — the 46MB veteran (all-MiniLM-L6-v2 lineage, ~23M parameters, 384 dims). Untouchable on CPU cost per chunk, but the shipped Modelfile sets num_ctx to 256, so typical RAG chunks are silently truncated to roughly their first 256 tokens. Fine for prototypes, short strings, and semantic-deduplication jobs. Not for production document retrieval, and the reason is that truncation rather than anything about the model's quality.

embeddinggemma — Google's 300M model, which its card positions as state-of-the-art for its size on multilingual MTEB (61.15 at 768 dims). Under Ollama it comes with two asterisks: the prompt prefixes above are documented as part of how the model is meant to be used, and it ships under Gemma licence terms rather than Apache or MIT — usually fine, always worth reading before a commercial deployment. Use it when you are standardising on the Gemma ecosystem, and always send the prefixes.

Also in the library: snowflake-arctic-embed2 (568M params, 8K context, Apache 2.0, with Matryoshka compression to 256 dims per Snowflake) and IBM's granite-embedding (English and multilingual variants) are credible options that sit well behind the six above in adoption. They get spec coverage here and nothing more.

---

Setup in two commands

Pull a model, then POST text to /api/embed — that is the entire embedding API, as documented in Ollama's API reference:

ollama pull nomic-embed-text

curl http://localhost:11434/api/embed -d '{
  "model": "nomic-embed-text",
  "input": ["first chunk of text", "second chunk of text"]
}'

input takes a single string or an array — always batch your corpus, the per-request overhead is what kills naive loops. The response's embeddings field is an array of vectors in input order. In Python:

pip install ollama

import ollama
r = ollama.embed(model="nomic-embed-text",
                 input=["search_document: first chunk",
                        "search_document: second chunk"])
vectors = r["embeddings"]

Note the search_document: prefixes going in by hand — that is the prefix lesson applied. Queries get search_query: instead. Anything speaking the OpenAI API works too, pointed at Ollama's compatibility endpoint:

from openai import OpenAI
client = OpenAI(base_url="http://localhost:11434/v1", api_key="ollama")
resp = client.embeddings.create(model="nomic-embed-text",
                                input="search_query: how do I resize a volume?")

Two flags worth knowing: "truncate": true is the default (inputs over the context limit are cut silently — decide whether you would rather get an error), and "dimensions": 256 shrinks output vectors on Matryoshka-trained models like nomic (only use it on models whose cards advertise MRL training; on others the parameter is meaningless or harmful). From here, the full pipeline — chunking, ChromaDB, retrieval, generation — is exactly what our Ollama + ChromaDB RAG guide walks through, our local RAG setup guide covers the LangChain and LlamaIndex variants, and our vector database comparison helps you pick the store. New to Ollama itself? Start with the complete Ollama guide.

---

How do I test these on my own data?

This is the section that replaces a benchmark you would have to take on faith. Twenty of your own queries beat anyone's published MTEB score, and building the harness takes an afternoon at most.

The pattern is always the same three steps:

1. Take 30-50 real chunks from the corpus you actually intend to search — not sample text. Include several chunks on the same topic, so the retriever has to separate genuinely similar candidates rather than keyword-match.
2. Write 20-25 queries in your users' words, each with exactly one chunk you consider correct. Deliberately word them differently from the passage; if the query repeats the passage's phrasing you are testing string matching, not embeddings.
3. Embed both sides, rank by cosine similarity, and count how often the correct chunk lands first (hit@1) and in the top three (hit@3). Time the corpus pass while you are at it — that is your indexing throughput, on your hardware, which no article can give you.

import time, numpy as np, ollama

MODEL = "nomic-embed-text"
docs = [...]      # your chunks
queries = [...]   # (question, index_of_correct_chunk) pairs

t0 = time.time()
D = np.array(ollama.embed(model=MODEL,
        input=[f"search_document: {d}" for d in docs])["embeddings"])
print(f"{len(docs)/(time.time()-t0):.1f} chunks/sec")

D /= np.linalg.norm(D, axis=1, keepdims=True)
hits = 0
for q, correct in queries:
    v = np.array(ollama.embed(model=MODEL, input=f"search_query: {q}")["embeddings"][0])
    v /= np.linalg.norm(v)
    if int(np.argmax(D @ v)) == correct:
        hits += 1
print(f"hit@1: {hits/len(queries):.1%}")

Swap MODEL and the prefix strings, rerun, compare. Two cautions on reading your own results, both of which apply to every published embedding benchmark as well: with 25 queries one query is four percentage points, so a single-query gap between two models is noise and not a result; and force CPU with options={"num_gpu": 0} if you plan to index on a machine without a GPU, because the ordering there can differ sharply from the GPU ordering.

---

Which one should you use?

A sensible default stack: nomic-embed-text for embeddings, ChromaDB for storage, and bge-reranker-v2-m3 as a reranking stage once quality matters.

• Starting a RAG project today: nomic-embed-text. Smallest serious footprint, 8K chunks, and the largest ecosystem behind it — every framework you might adopt has been tested against it.
• Multilingual documents: bge-m3 (mature, MIT) or qwen3-embedding (newer, 32K context, growable to 4b/8b). Published scores will not separate these two for your corpus — run both through the harness above on 50 of your own queries and keep the winner.
• Chunks are short and you want maximum published quality: mxbai-embed-large — just keep every chunk under 512 tokens, and assert it rather than assume it.
• CPU-only box: all-minilm for prototypes and short strings; nomic-embed-text is the better production choice, because 256-token truncation will quietly cost you more accuracy than the extra parameters cost you time.
• Retrieval quality still not good enough? The answer is usually not a different embedding model. A cross-encoder reranking stage lifts quality far more than swapping one bi-encoder for another — our reranking guide covers bge-reranker-v2-m3 setup (rerankers are not in the Ollama library; they run via sentence-transformers or TEI alongside it).
• Still tempted by OpenAI's embedding API? The gap has narrowed for most retrieval work while local costs nothing per token — our local vs OpenAI embeddings comparison has the head-to-head.

A scaling note: every model on this page runs on hardware you already own, and embeddings never need a GPU upgrade. The generation model is where VRAM planning happens — and if you want one tuned specifically for answering from retrieved passages, Dragon 7B is a RAG-optimised local model worth trying alongside a general chat model.

---

What this page does not tell you

Being explicit about the edges, because "best embedding model" articles usually are not.

• There are no throughput or accuracy measurements here. Every quality figure quoted is a published score from a model card, attributed inline, and every capacity figure is either a vendor specification or arithmetic you can redo. Chunks-per-second depends entirely on your CPU, GPU, quantization and chunk length; the harness above will tell you yours in a few minutes, which is more than a number from someone else's machine ever could.
• MTEB scores across these cards are not cross-comparable. They span different MTEB generations and task mixes — mxbai's 64.68 is from the older English suite, qwen3's 64.33 is multilingual v2. Quote them; do not rank by them.
• Multilingual claims come from model cards, not from any evaluation on this site. If your corpus is not English, that is precisely the case where you should run your own eval before committing.
• Ollama's build is not always the card's build. qwen3-embedding ships quantized to Q8_0 where the card's scores were measured at BF16, and shipped num_ctx values differ from upstream defaults in both directions (nomic raised to 8192, all-minilm lowered to 256). Run ollama show on your own install rather than trusting any table, including the one above.
• Two library models get spec coverage only: snowflake-arctic-embed2 and granite-embedding.

---

Sources

• Ollama library — model pages for nomic-embed-text, mxbai-embed-large, bge-m3, qwen3-embedding, all-minilm and embeddinggemma: sizes, parameter counts, tags and live pull counts
• nomic-embed-text-v1.5 card, mxbai-embed-large-v1 card, BAAI/bge-m3 card, Qwen3-Embedding-0.6B card, embeddinggemma-300m card, all-MiniLM-L6-v2 card — dimensions, context lengths, MTEB scores and the prompt-prefix formats quoted above
• Ollama API documentation — the /api/embed request and response contract, the truncate default, and the dimensions parameter
• Snowflake Arctic Embed 2.0 announcement — arctic-embed2 specifications

---

FAQ