Best Local Models for Tab Autocomplete: FIM Models Compared

Qwen2.5-Coder 1.5B base is the best local tab-autocomplete model for most machines: a 986MB download, Apache 2.0, FIM-trained, and small enough that it clears autocomplete's sub-500ms latency budget on hardware as modest as a CPU-only laptop. Have 12GB+ of VRAM or a 32GB Mac? Run the 7B (4.7GB, also Apache 2.0). Codestral 22B is the open-weights quality ceiling, but its non-production license and 13GB size make it a niche pick, not a default.

Autocomplete is its own discipline, and it is the opposite of the "biggest model wins" logic that rules chat coding. A tab suggestion that arrives in two seconds is worthless no matter how smart it is — you already typed the line. So the models that matter here are small, FIM-trained (fill-in-the-middle), and fast enough to keep up with your pauses. This page covers which models actually support FIM as of August 2026, how to work out from a model's size whether it can possibly hit that latency budget, and the exact setup for the four clients that still do local autocomplete well: llama.vscode, Continue, Zed, and Tabby.

---

Why Autocomplete Needs FIM Models, Not Chat Models

Tab autocomplete requires a model trained on fill-in-the-middle: it sees the code before your cursor and the code after it, and predicts only the missing middle. A chat model given the same job will happily rewrite your whole file, wrap everything in markdown fences, or explain what it is about to do. Continue's documentation is blunt about this: an autocomplete model is "an LLM trained on a special format called fill-in-the-middle (FIM)," while chat models, "though larger, will often perform poorly even with extensive prompting." Size is not the lever here — Continue's own docs note that "a 3B parameter model can perform well" at this job.

The FIM format itself is three special tokens. For the Qwen coder family it looks like this:

<|fim_prefix|>code before cursor<|fim_suffix|>code after cursor<|fim_middle|>

The model generates whatever belongs at the cursor and stops. StarCoder2 uses the same idea with its own tokens (<fim_prefix>, <fim_suffix>, <fim_middle>), and Codestral has FIM built into its serving stack. Your editor plugin assembles this prompt on every keystroke pause — which is why prompt-processing speed matters as much as generation speed.

One more thing worth knowing before you shop: GitHub Copilot cannot do this locally. VS Code's own docs, as of August 2026: "Currently, you cannot connect to a local model for inline suggestions." Bring-your-own-key — including Ollama — covers chat only; completions stay on GitHub's hosted models. That single sentence is why every tool in this guide exists.

Two practical rules fall out of all this:

• Always pull the -base tag (qwen2.5-coder:1.5b-base, not qwen2.5-coder:1.5b). Instruct builds are tuned for conversation and degrade FIM output.
• Latency beats IQ. A small model that answers inside your typing pause gets accepted more often than a large one that answers after it, because you are still looking at the line when the suggestion appears.

---

The Short List: Every Local Model Worth Considering

Seven models genuinely support FIM and are worth your disk space; for almost every setup the answer is one of the two Apache-licensed Qwen2.5-Coder builds, the 1.5B or the 7B. Download sizes are Ollama's q4 builds; licenses and context are from the official model cards (checked August 2026).

Sources: Ollama model library for sizes; Hugging Face model cards for parameters, context, and licenses.

Notice what is not here: no 14B+, no chat models, no reasoning models. Chat-style coding — where the strongest model that fits your card does win — is a different list, and it lives in the related posts below. Autocomplete rewards the small end of the family.

---

How Fast Is Fast Enough?

A suggestion is useful only if it lands inside your typing pause — roughly 300-500ms end to end. You can work out whether a model can possibly hit that before you download it, using two multiplications and one published spec. No benchmark required, and unlike a benchmark, the arithmetic transfers to whatever hardware you actually own.

Step 1: how much memory do the weights occupy? At q4 quantization, weights take roughly 0.6 GB per billion parameters. That rule reproduces the published Ollama download sizes closely enough to trust it: 1.54B × 0.6 = 0.92 GB against a listed 986MB, 3.09B × 0.6 = 1.85 GB against 1.9GB, 7.61B × 0.6 = 4.57 GB against 4.7GB.

Step 2: what is the generation ceiling? Producing each token requires reading every weight out of memory once, so:

tokens/second ceiling = memory bandwidth (GB/s) / model size in memory (GB)

Memory bandwidth is a published number on every GPU and memory datasheet. Divide, and you get an upper bound on generation speed:

Step 3: turn that into a suggestion. A typical single-line-to-short-block completion is around 18 tokens, so divide 18 by the ceiling:

Read every one of those as an arithmetic upper bound — attention overhead, sampling and prompt processing all pull real output below it, often by a third or more. Which is exactly why the conclusion is robust: halve every number in the table and the 1.5B still lands inside the budget on a CPU, while the 22B still misses it on a laptop. That is the entire case for small models in this category, and it holds without anyone benchmarking anything.

The number the table leaves out is time-to-first-token, which is prompt processing rather than generation and scales with how much context the client sends. This is where client design matters more than model choice: llama.cpp's dedicated /infill endpoint reuses cached context between keystrokes, so typing within one file rarely reprocesses the whole prefix. Long files with a cold cache are where autocomplete visibly stutters.

What about discrete GPUs? The bar is low and well-documented: Tabby's docs say 1-3B completion models are fine on an NVIDIA T4, 10-series, or 20-series GPU (or an Apple M1). These models fit entirely inside even an 8GB card with room to spare. If you are choosing hardware for a coding setup, our 8GB VRAM coding picks cover this tier.

---

Model Notes: What the Table Doesn't Tell You

Qwen2.5-Coder 1.5B — the default, still

Two years after release, this is still the model that every autocomplete client's documentation points at. Continue's docs recommend it by name for local autocomplete. llama.cpp ships a one-flag preset for it (--fim-qwen-1.5b-default). It is Apache 2.0, 32K context, and small enough to leave resident in memory permanently. Full spec breakdown on our Qwen 2.5 Coder 1.5B page.

Qwen2.5-Coder 3B — the license trap

The 3B is the only Qwen2.5-Coder size that is not Apache 2.0 — it ships under the Qwen Research license, which is non-commercial. The 0.5B, 1.5B, 7B (and up) are all Apache 2.0; the 3B alone is research-only, per its Hugging Face model card. If you write code for a living, skip it: use the 1.5B, or spend the extra 3GB on the 7B and get a better model with a clean license anyway.

Qwen2.5-Coder 7B — the step-up that earns its VRAM

If your machine can hold 4.7GB of weights without evicting anything else, the 7B is the best local autocomplete quality you can get under a permissive license. It is the model Zed's docs use in their example Ollama config (qwen2.5-coder:7b-base), and llama.vscode's tiers recommend it above 16GB VRAM. On a 12-16GB GPU it shares space comfortably with your desktop; pairing it with a bigger chat model is covered in our 16GB VRAM coding guide.

StarCoder2-3B — the veteran

StarCoder2-3B still earns a slot because Tabby's model registry supports it out of the box and its OpenRAIL-M license is commercial-friendly — but it is showing its age. Trained on The Stack v2 (17 languages, 3T+ tokens) with a native FIM objective, it scores 31.7% pass@1 on HumanEval per its model card — well behind current Qwen equivalents — and its 16K context uses a 4K sliding window. Pick it when you are standardizing a team on Tabby and want the battle-tested option; otherwise the Qwen family has passed it.

Codestral 22B — the ceiling, with an asterisk

Codestral 22B is among the strongest open-weights FIM models you can download — Mistral's own comparison table scores it at 82.07% average on HumanEvalFIM — but it is licensed for non-production use only. The MNPL license means testing and research, not your day job, unless you buy a commercial license. Two more caveats: at 13GB it wants a 16GB+ GPU to itself, and the improved Codestral 25.01 (85.9% HumanEvalFIM per Mistral) never shipped as open weights — it is API and IDE-partner only. Run the 22B locally if quality-per-suggestion matters more to you than latency and licensing; our Codestral 22B analysis has the full picture.

Qwen3-Coder and the MoE question

Every version of Qwen3-Coder supports FIM, per Qwen's own repository — but the smallest one is a 19GB download, so treat it as an agent model you can also autocomplete with, not an autocomplete pick. The 30B-A3B MoE activates only 3.3B parameters per token, so once it is loaded it generates fast; if it is already resident for agentic coding, pointing your editor's completion at it costs nothing. Worth noting where the tooling still sits, though: llama.vscode's own preset tiers are all Qwen2.5-Coder FIM builds, so the dedicated-autocomplete ecosystem has not moved to the Qwen3 generation yet.

---

Editor Setup: The Four Clients That Do Local Autocomplete Right

Fastest path: install llama.cpp, run one preset flag, install the llama.vscode extension — you have local tab completion in under five minutes. Here is each client, in the order we would reach for them.

1. llama.vscode (VS Code) — the purist's pick

Built by the llama.cpp team, MIT-licensed, and the reason llama.cpp's server grew a dedicated /infill endpoint with smart context reuse. Setup:

# macOS (Windows: winget install llama.cpp)
brew install llama.cpp

# pick ONE preset — it downloads the model automatically:
llama serve --fim-qwen-1.5b-default   # less than 8GB VRAM
llama serve --fim-qwen-3b-default     # less than 16GB VRAM
llama serve --fim-qwen-7b-default     # more than 16GB VRAM
llama serve --fim-qwen-30b-default    # more than 64GB VRAM

Then install the llama-vscode extension from the marketplace and it connects to the server on its own. Those tiers are llama.vscode's own recommendations from its README. The presets are all Qwen2.5-Coder FIM builds — the ecosystem's revealed preference, again.

2. Continue (VS Code + JetBrains) — if you already use it for chat

Continue does chat and autocomplete in one extension, both against Ollama. The autocomplete role is one block in config.yaml:

models:
  - name: local-autocomplete
    provider: ollama
    model: qwen2.5-coder:1.5b-base
    roles:
      - autocomplete

Our Continue + Ollama setup guide walks through the full file, including the chat model half.

3. Zed — the nicest tab-completion UX of the group

Zed's edit prediction defaults to its hosted Zeta model (sign-in required, 2,000 free predictions/month), but the docs support pointing it at local Ollama directly:

{
  "edit_predictions": {
    "provider": "ollama",
    "ollama": {
      "api_url": "http://localhost:11434",
      "model": "qwen2.5-coder:7b-base",
      "prompt_format": "infer",
      "max_output_tokens": 512
    }
  }
}

"prompt_format": "infer" auto-detects the FIM token style from the model name — Zed knows the Qwen, StarCoder, CodeLlama, DeepSeek, CodeGemma, and GLM formats. Zed can even run its own Zeta 2 model locally via Ollama ("model": "zeta2").

4. Tabby — the self-hosted server for teams

Tabby runs completions as a server with its own extension ecosystem, which makes it the one to pick when several people share a GPU box. The official Docker one-liner:

docker run -d --name tabby --gpus all -p 8080:8080 \
  -v $HOME/.tabby:/data registry.tabbyml.com/tabbyml/tabby \
  serve --model StarCoder-1B --chat-model Qwen2-1.5B-Instruct --device cuda

Swap --model Qwen2.5-Coder-1.5B (or StarCoder2-3B) in — the registry supports the Qwen2.5-Coder line from 0.5B up to 14B, plus Codestral-22B. Our Tabby self-hosted guide covers accounts, IDE clients, and the repo-indexing features.

A word on Twinny

Twinny was the scrappy free option in this space for years, but the repository was archived on November 24, 2025 and is now read-only. It still installs and still works, but do not build a new setup on an unmaintained extension when llama.vscode and Continue are actively developed.

---

Honest Limitations

Local autocomplete in 2026 is genuinely good — and still a step behind the hosted state of the art. Know what you are trading.

• Hosted completion models are better. Copilot's completions, Zed's hosted Zeta, Mercury Coder, and the API-only Codestral 25.01 outrank anything you can download, especially at cross-file, multi-edit prediction. Local FIM is mostly single-file infilling; llama.cpp's /infill endpoint accepts extra context files, but client support varies.
• Small models produce confident near-misses. At 1.5B-3B the suggestion is usually structurally right and occasionally wrong in a detail — an inverted condition, the wrong field, an off-by-one. You accept-and-fix rather than accept-and-forget. Budget attention accordingly.
• The first completion after idle stalls. Ollama's default keep_alive unloads an idle model after five minutes, so the first suggestion after a coffee break waits for a reload from disk — roughly model size ÷ storage read speed, so about half a second for the 1.5B on NVMe and several times that on a SATA SSD. Set OLLAMA_KEEP_ALIVE=-1 (or keep_alive per request) if the stall annoys you; the 1.5B only holds ~1GB resident.
• Laptops pay in battery. Autocomplete fires on nearly every pause. On battery power, expect the fan and the drain; some users keep local autocomplete as a desk-only habit.
• Wrong tag, worse results. The most common setup mistake is pulling the instruct build instead of -base. If your completions arrive wrapped in prose or markdown fences, this is why.
• Check the license before work use. Qwen2.5-Coder 3B is research-only and Codestral 22B is non-production. The 0.5B/1.5B/7B Qwen builds and StarCoder2 are the clean choices for commercial work.

---

Verdict

Run qwen2.5-coder:1.5b-base unless you have a concrete reason not to. Concrete reasons, mapped:

The bigger takeaway: autocomplete is the cheapest place in local AI to get a daily-driver win. You do not need this year's GPU, you do not need a 20GB download, and the best model for the job costs less than a gigabyte. Spend the saved VRAM on a proper chat model — our size-by-size coding guide is the place to pick that one.

---

Sources

• Qwen2.5-Coder model cards on Hugging Face — parameters, context, and per-size licenses (0.5B/1.5B/7B Apache 2.0; 3B Qwen Research), checked August 2026
• Qwen coder GitHub repository — FIM token format; "FIM is supported in every version of Qwen3-Coder"
• StarCoder2-3B model card — FIM training objective, HumanEval 31.7%, context and license
• Mistral Codestral announcement (MNPL license) and Codestral 25.01 — the FIM benchmark table scoring Codestral-2405 22B at 82.07% and the API-only 25.01 at 85.89% HumanEvalFIM
• Ollama model library — download sizes and tags for qwen2.5-coder, codestral, and qwen3-coder
• VS Code Copilot docs — "you cannot connect to a local model for inline suggestions" (BYOK is chat-only)
• llama.vscode README and llama.cpp server docs — VRAM tiers, --fim-qwen presets, /infill endpoint
• Continue autocomplete docs · Zed edit prediction docs · Tabby model registry — client configs and recommendations
• Latency figures on this page are arithmetic ceilings, not measurements: model size from the 0.6 GB-per-billion-parameter q4 rule, divided into published memory-bandwidth specs. The formula is stated in full above so you can check it against your own hardware

---

FAQ