Run an LLM on Your Phone (2026): Offline AI on Android & iPhone

Published on June 20, 2026 • Updated August 3, 2026 • 16 min read

Yes — you can run a real LLM fully offline on a modern phone today. The simplest path on both Android and iPhone is the free open-source app PocketPal AI (or MLC Chat); download a 1B-4B GGUF model like Qwen 3 1.7B, Gemma 3 1B, or Phi-4-mini, and you get private, no-internet chat at roughly 10-40 tokens/second on a recent flagship depending on your phone and the model. Your phone may also already have a model built in — Gemini Nano on recent Android flagships and Apple's on-device Foundation Model on iOS 26+ — and WebLLM can even run one inside a browser tab with nothing installed; all three are covered below. The catch is RAM, not the app: 1B-2B models run comfortably on 6-8GB phones, while 3-4B models really want a flagship with 8-12GB of memory. This guide covers every option worth knowing, which models actually fit, the speeds you should expect, and exact steps for both platforms.

Running a model on your phone is the most private setup possible — the weights and your prompts never leave the device, so it works on a plane, in a tunnel, or with the network fully off. The trade-off is that these are small models: great for summarizing, drafting, quick Q&A, and offline reference, but not a replacement for a 70B model on a desktop. If privacy is your main reason, also read our local AI privacy guide.

Can a phone actually run an LLM, or is this a gimmick?

It is genuinely real now, for two reasons that both matured over 2024-2026. First, the runtimes got good: llama.cpp gained ARM-optimized CPU kernels (including Arm's KleidiAI) and Vulkan GPU acceleration for Snapdragon-class chips, and Apple shipped on-device inference as a first-class API. Second, model makers now treat phones as a first-class target. Meta's Llama 3.2 1B/3B and Google's Gemma 3 1B started the trend, and the current generation has moved well past them: Alibaba's Qwen 3 comes in 0.6B/1.7B/4B sizes with an optional thinking mode, Microsoft's Phi-4-mini packs unusually strong reasoning into 3.8B parameters, Liquid AI's LFM2 family (350M-2.6B) was designed from scratch for fast CPU inference on phones, and Google's Gemma 4 E2B/E4B models (March 2026) list phones and Raspberry Pi as official deployment targets.

The honest limitation is memory and heat. A phone shares one pool of RAM between the OS, your apps, and the model, and it has no active cooling. So a 1B model is a smooth daily-driver experience, a 3B model is usable on a flagship but will throttle during long generations, and anything 7B+ is possible to load on a 12-16GB phone but slow enough that most people won't enjoy it.

Which apps can run a local LLM on a phone in 2026?

There are seven options worth knowing, covering both platforms. Three are general "download any GGUF and chat" apps, one is Google's showcase app, two are built-in system models (one per platform), and one runs entirely in the browser.

A few clarifications that matter. PocketPal AI (an open-source project by Asghar Ghorbani) and MLC Chat (from the MLC-AI team) are the two most popular general-purpose pick-a-model apps; both run everything locally with no servers after the model is downloaded. LM Playground (by Andriy Druk) is Android-only and built directly on llama.cpp with GGUF models, tuned with ARM KleidiAI kernels for faster generation. Google AI Edge Gallery is Google's own open-source demo app that runs the mobile-first Gemma 3n model (and other LiteRT models) fully offline. Gemini Nano and Apple's Foundation Models framework are the two built-in system models — you don't download a model file, but you also can't swap in your own; both get their own section below. WebLLM skips installation entirely and runs the model inside a browser tab — also covered below.

What about Gemini Nano, the model already inside your Android phone?

Gemini Nano is Google's on-device model (1.8B-3.25B parameters, roughly 1GB on disk) that ships inside recent Android flagships — Pixel 8 and newer, Samsung Galaxy S24/S25, and a growing list of Xiaomi, Motorola, and Honor phones. You don't install it from an app store: it lives in Android's AICore service and powers features like Recorder summaries, Magic Compose, Call Notes, and scam detection, all fully offline. The important catch: you can't open Gemini Nano and chat with it the way you can with PocketPal. Google exposes it to developers through the ML Kit GenAI APIs (summarization, proofreading, rewriting, image description), not as a general-purpose chatbot.

Two practical notes. On a Pixel 8/8a you have to switch it on manually — enable Developer Options, open "AICore Settings", toggle "Enable On-Device GenAI Features", then wait for the ~1GB model download; Pixel 9 and newer and Samsung devices enable it out of the box. And capability-wise it sits below the GGUF models in this guide for open-ended chat: it's tuned for short, structured tasks with a small context window. For the full supported-device list, enablement walkthrough, and developer API details, see our dedicated Gemini Nano Android guide.

So which should you use? Gemini Nano when you want the OS-level features — they're instant, private, and free. A GGUF app like PocketPal when you want a real, open-ended chat model that you pick yourself.

Can you run an LLM in the browser on a phone? (WebLLM)

Yes. WebLLM — the open-source engine from the same MLC team behind MLC Chat — runs small models like Llama 3.2 1B/3B and Qwen entirely inside a WebGPU browser tab: nothing to install, an OpenAI-compatible API for developers, and the model is cached in the browser for offline reuse after the first download. On Android it works in Chrome 121 and later; on iPhone, WebGPU only arrived with Safari 26, so browser inference there is newer and less battle-tested.

In practice, a browser tab is the quickest zero-install way to demo local AI on a phone, but a native app uses the hardware better and manages memory more predictably — expect a native llama.cpp app to feel faster and more stable than the same model in a tab. Where WebLLM genuinely shines is shipping AI inside a web app: your users get private, on-device inference with zero setup on any WebGPU browser, desktop included. Setup code, supported models, and performance numbers are in our WebLLM browser AI guide, with the broader picture in run an LLM in your browser.

Which small models actually fit in phone RAM?

This is the question that decides your experience. Phone RAM is shared, so subtract roughly 2-4GB for the OS before you budget for a model. As a rule of thumb a 4-bit quantized model needs a bit more than half its parameter count in gigabytes of RAM, plus headroom for context.

By RAM tier, the picks sort themselves out cleanly. On a 4-6GB phone, stick to the tiny end: LFM2 700M/1.2B or Gemma 3 1B. On an 8GB phone (iPhone 15 Pro/16-class, most mid-range Android), Qwen 3 1.7B is the best daily driver, with Gemma 4 E2B as the Google-flavored alternative. On a 12GB+ flagship, step up to Qwen 3 4B, Phi-4-mini, or Gemma 4 E4B — noticeably smarter, at the cost of slower, warmer generations. Llama 3.2 and Qwen2.5 — the models this guide led with a year ago — still work fine, but the newer generation above beats them at every size. If you also run models on a laptop, our companion guides on the best local AI models for 8GB RAM and small language models line up neatly with this list.

How fast is it really? (measured speeds and honest limits)

Speed depends far more on your phone's chip and thermals than on the app. The figures below are drawn from published 2026 on-device benchmarks (independent app testing reports roughly 30-40 tok/s for sub-1B models, 20-30 for ~1.5B, 10-20 for 3B, and 8-15 for ~4B on current iPhones) cross-checked with our own hands-on use in PocketPal and MLC Chat on a current flagship (Snapdragon 8 Gen 3 / Apple A18-class). Treat all figures as approximate — they vary with the app, prompt length, quantization, and how hot the phone already is.

The newer models slot into the same size-class bands: expect Qwen 3 1.7B to land roughly where Qwen2.5 1.5B does, and Phi-4-mini, Qwen 3 4B, and Gemma 4 E4B to behave like the 3-4B row — with LFM2 the outlier that runs faster than its size suggests thanks to its CPU-first design. In first-hand use, the pattern is consistent: a 1B model on a 2023-2025 flagship reads back faster than you can comfortably follow, while a 3B model starts in the low-20s tok/s on an iPhone 16 Pro and then thermal-throttles once the chip heats up over a long generation. The other limit is battery: sustained on-device inference is one of the heaviest things you can ask a phone to do, so it drains fast and the back of the device gets warm. None of this is a dealbreaker — it just means phone LLMs are best for short, frequent, private interactions rather than churning out long documents.

For comparison, a laptop with a discrete GPU runs the same small models several times faster and without thermal limits — if you're weighing a phone setup against an AI-capable laptop, see Copilot+ PC vs RTX local AI.

How do I run an LLM on Android? (step by step)

The fastest route on Android is a llama.cpp-based app. Using PocketPal AI (works the same idea in LM Playground):

1. Install the app. Get PocketPal AI from Google Play (or LM Playground for an Android-native option). Both are free and open-source.
2. Pick a model that fits. In the app's model list (or via its Hugging Face search), choose a small GGUF — Qwen 3 1.7B, Gemma 3 1B, or LFM2 1.2B. On an 8GB+ phone you can step up to Qwen 3 4B or Phi-4-mini.
3. Download the weights. Tap download; the GGUF (typically a few hundred MB to ~2GB) is fetched once and cached on-device. After this you can turn airplane mode on.
4. Load and chat. Tap to load the model into memory, then type. The first reply takes a moment to "warm up"; subsequent replies are faster.
5. Tune if needed. If the app lets you, keep context length modest (e.g. 2048-4096 tokens) to avoid memory spikes, and prefer a Q4_K_M quant for the best size/quality balance.

For power users, you can also run the official llama.cpp directly inside Termux (no root required): install git, cmake, and clang, build llama.cpp, download a GGUF from Hugging Face, and run llama-server — on Snapdragon/Adreno devices you can also try the OpenCL or Vulkan GPU backends, though GPU offload support on phones is still uneven. That's more involved than an app, but it gives you a local OpenAI-style API on the phone.

How do I run an LLM on iPhone? (step by step)

iPhone has two distinct paths.

Path A — download-and-chat apps (PocketPal AI / MLC Chat):

1. Install PocketPal AI or MLC Chat from the App Store (both free). MLC Chat has the broadest curated list (Llama 3.2, Gemma, Phi, Qwen); PocketPal can load any GGUF from Hugging Face, which is how you get the newest models like Qwen 3 and Phi-4-mini.
2. In the app, choose a model. On an 8GB iPhone (15 Pro and newer), a Qwen 3 1.7B or Gemma 3 1B runs smoothly; a 4B-class model is possible but warmer.
3. Download the weights once. They cache locally, so afterward everything runs with the network off.
4. Tap to load the model, then chat. Expect a brief load, then fast short-answer generation.

Path B — Apple's built-in model (no download): On iOS 26 and later, Apple-Intelligence-capable iPhones (15 Pro and newer) include Apple's on-device ~3B Foundation Model — the model behind Apple Intelligence features like Writing Tools, notification summaries, and Image Playground. You don't install or manage it; apps built with Apple's Foundation Models framework tap it directly via Swift, giving you private, offline, free-of-cost AI features inside those apps. It's the most seamless option and the direct counterpart to Gemini Nano on Android, with the same trade: zero setup, but you can't choose or swap the model the way you can with PocketPal or MLC Chat.

Key Takeaways

1. Phone LLMs are real and private — PocketPal AI and MLC Chat (both free, open-source, iOS + Android) are the easiest way to run a model fully offline; LM Playground is a strong Android-only alternative, and WebLLM runs one in a browser tab with nothing installed.
2. RAM is the gatekeeper. Budget ~2-4GB for the OS first. Small models (Qwen 3 1.7B, Gemma 3 1B, LFM2) run on 4-8GB phones; 3-4B models (Qwen 3 4B, Phi-4-mini, Gemma 4 E4B) want an 8-12GB flagship.
3. Speed is good but thermal-limited. Expect roughly ~25-40 tok/s on 1B models and ~15-23 tok/s falling off on a 3-4B model as the phone heats up; battery drains fast during inference.
4. Both platforms now ship a built-in model — Gemini Nano on recent Android flagships (via AICore + ML Kit) and Apple's ~3B Foundation Model on iOS 26+. Zero download, fully private, but no model choice; download-apps give you flexibility instead.
5. Match the task to the model. Phone LLMs shine at short, frequent, private tasks (summaries, drafts, offline Q&A), not long-form generation.

Next Steps

• See which models to download once you know your phone's memory in our best local AI models for 8GB RAM breakdown — most of those picks are exactly what you'll load in PocketPal.
• On a recent Android flagship? Learn what your phone's built-in model can already do in the Gemini Nano Android guide.
• Building for the web instead of an app? The WebLLM browser AI guide covers zero-install, on-device inference for web apps.
• Go deeper on the small-model generation itself — Phi-4-mini, Gemma 4 E-series, Qwen 3 — in our small language models guide.
• Want the same models on a desktop with Ollama for heavier work? Start with the best Ollama models guide.
• Understand exactly why on-device beats the cloud for sensitive data in our local AI privacy guide.
• Deciding between a phone setup and a proper AI laptop? Compare options in Copilot+ PC vs RTX local AI.
• Official runtime docs and model cards: llama.cpp on GitHub and Apple's Foundation Models research.