Laptop Local AI: What Each Memory Tier Actually Runs

What a laptop can run is set by memory capacity: 8GB of RAM means 3B models, 16GB with an 8GB discrete GPU means 7-8B, and only Apple Silicon with 48GB or more of unified memory holds a 70B at Q4. How fast it runs is set by memory bandwidth, and how long it keeps running is set by a chassis that has to dissipate 35-45W continuously. This guide works through all three from vendor-published specs, so you can do the sum for the exact machine you are looking at.

You will not find a tokens-per-second table below claiming to be somebody's stopwatch. Laptop throughput depends on the SKU, the power profile, the firmware revision, the ambient temperature and whether the lid is open — which makes anyone else's number nearly worthless to you. What travels is the arithmetic, and the commands to take the reading on your own machine.

Quick Start: What Your Laptop Can Run Today

Before the detail, the cheat sheet:

# Step 1: Install Ollama (works on macOS, Windows, Linux)
curl -fsSL https://ollama.com/install.sh | sh

# Step 2: Match a model to your laptop
# 8GB RAM laptop:
ollama run llama3.2:3b
# 16GB RAM laptop:
ollama run llama3.1:8b
# 16GB+ with discrete GPU (>=8GB VRAM):
ollama run qwen2.5:14b
# 32GB+ unified memory MacBook Pro:
ollama run llama3.3:70b-instruct-q4_K_M

Integrated graphics and 8GB of RAM: you run a 3B model and you start thinking about an upgrade. 16GB plus a discrete GPU: a 7-8B model runs comfortably. Apple Silicon Pro or Max with 32GB and up: you punch above your weight. Below those tiers the honest answer is to save up or use a hosted API.

---

What Actually Limits Local AI on a Laptop?

Desktops have heat sinks the size of bricks, 850W power supplies, and PCIe x16 lanes that never share bandwidth. Laptops have none of those. Three constraints dominate everything else:

1. Memory capacity. Weights have to be resident. A model that does not fit in VRAM gets partially offloaded to system RAM, and the offloaded layers run at DDR speed instead of GDDR speed. This is a cliff, not a slope — it is the single most important number on the spec sheet.

2. Memory bandwidth. Generating one token means reading every weight once, so bandwidth — not FLOPS — sets the speed ceiling. Apple publishes 546 GB/s for the 16-core-CPU M4 Max. A Windows laptop on dual-channel DDR5-5600 SODIMMs gets roughly 90 GB/s shared between CPU and integrated GPU. A discrete laptop GPU has its own GDDR6 at 256-576 GB/s depending on bus width.

3. Thermal and power envelope. A 14-inch chassis can dissipate on the order of 35-45W of combined CPU/GPU heat continuously; the same silicon in a desktop runs at 105W or more. Plugged in, a 200W brick lets a discrete GPU pull its rated total graphics power. Unplugged, the firmware clamps it hard, which is documented OEM behaviour rather than a defect.

Keep these three in mind every time you read a laptop AI benchmark. Most published figures are minute-one numbers on AC power, and they tell you very little about a sustained workload.

How Much Model Fits in Laptop Memory?

The sizing rule for Q4_K_M, the quantization Ollama pulls by default:

VRAM needed (GB) ~= 0.6 x parameters in billions, plus 1-2 GB for context and KV cache

So a 7B needs about 5GB, a 14B about 9GB, a 32B about 20GB, a 70B about 40GB. Applied to the memory tiers laptops actually ship in:

The hard ceiling on the Windows and Linux side is discrete VRAM, and it has barely moved: the RTX 4090 laptop part tops out at 16GB, and the current top laptop SKU at 24GB. Apple is the only vendor selling a portable with enough addressable memory for a 70B, because unified memory means the GPU can use the whole pool. For a complete model-to-memory mapping, the Ollama RAM/VRAM master table lists every popular model.

How Fast Will a Model Run on a Laptop?

One division gives you the ceiling:

tokens/sec ceiling = memory bandwidth (GB/s) / model file size (GB)

This is an arithmetic upper bound, not a prediction. Real output lands below it because attention over the KV cache, sampling and framework overhead all take time the formula ignores, and on a laptop the sustained clock is lower than the boost clock the spec sheet assumes. Use it to compare machines and to sanity-check a review: any number above the ceiling is impossible.

Vendor-published bandwidth for the silicon that shows up in AI laptops:

Apply the formula to three common model sizes and you get the comparison that actually matters:

Read it as a ranking, not a promise. Two consequences fall straight out of the numbers. First, the 8GB laptop GPUs are limited by capacity long before they are limited by speed — an RTX 4070 laptop part has plenty of bandwidth for a 14B and nowhere to put it. Second, Apple's advantage is entirely a capacity advantage at the top end: an M4 Max is not faster per gigabyte than a 4090 laptop chip, it simply has somewhere to put a 40GB model. Why bandwidth rather than TFLOPS is the binding constraint has the longer version of this argument.

Why Do Apple Silicon Laptops Win This Category?

Unified memory. On an M-series chip the CPU and GPU address one pool, so a 36GB MacBook Pro can hold models that would need a discrete GPU costing more than the laptop. Apple's published bandwidth climbs steeply across the range — 150 GB/s on the M3 Pro, 273 GB/s on the M4 Pro, 546 GB/s on the top M4 Max bin — and configurations go to 128GB, which is why the 70B tier is Apple-only in portable form. Full specs are on Apple's MacBook Pro spec page; check the bandwidth for the exact bin you are buying, because the M3 Max and M4 Max each ship in two, and the cheaper one is meaningfully slower.

Two things worth knowing before you spend. The Neural Engine is not part of this story: Ollama and llama.cpp run on Metal, not the ANE, so a chip's TOPS figure tells you nothing about token throughput. And memory is soldered — the configuration you buy is the configuration you keep for the life of the machine, which makes RAM the one option worth over-buying. Our Apple Silicon buying guide works through the bins, and the Mac setup guide covers Metal configuration once the machine arrives.

What Do Windows Laptops with NVIDIA GPUs Give You?

This is the "AI-ready" gaming and creator shelf: Asus ROG, Lenovo Legion, Razer Blade, MSI Stealth, Acer Predator, typically an Intel Core Ultra or Ryzen 9 paired with a mobile RTX part. The 4090 laptop chip is essentially a desktop 4080 die in a 175W package with 16GB of VRAM. Current specs are on NVIDIA's laptop GPU page.

What you get: the best prompt-processing throughput available in a laptop. CUDA is where every inference framework optimises first, and prompt evaluation is compute-bound rather than bandwidth-bound, so long-context RAG feels snappier on these machines than the decode-rate table above suggests. Time-to-first-token is where they win.

What you give up: capacity and untethered operation. 16GB of VRAM caps you at the 14B-24B class no matter how much system RAM the laptop has. And on battery, NVIDIA Optimus and Advanced Optimus reduce discrete-GPU power to extend runtime — that is designed behaviour documented by NVIDIA and the OEMs, not throttling. Some vendors let you override it in MSI Center or Armoury Crate, at the cost of a battery that empties in under an hour. Treat these as AC-power machines and you will be happy with them.

CUDA setup is one step. Ollama for Windows bundles the CUDA runtime, so the desktop app plus a current NVIDIA driver gets you GPU acceleration with no manual toolkit install.

Can a Linux Laptop Run Local AI Well?

Yes, with the same memory arithmetic and better long-term prospects. A ThinkPad X1 Carbon with 32GB and no discrete GPU is not fast — it is running against roughly 90 GB/s of LPDDR5 shared with everything else — but it is reliable, repairable, and holds an 8B model without complaint. Use the Vulkan backend in llama.cpp to get whatever the Intel Iris Xe integrated GPU can contribute:

# ThinkPad with Intel Iris Xe (no dGPU): use the Vulkan backend in llama.cpp
git clone https://github.com/ggerganov/llama.cpp
cd llama.cpp
cmake -B build -DGGML_VULKAN=ON
cmake --build build --config Release -j

# Run a model with Vulkan acceleration
./build/bin/llama-cli \
  -m ~/models/Meta-Llama-3.1-8B-Instruct-Q4_K_M.gguf \
  -ngl 99 \
  -p "Summarize: " \
  -n 256

The Framework 16 is the more interesting Linux machine because the Radeon RX 7700S graphics module is user-replaceable. ROCm 6.1 and later support it for inference, so you get real GPU acceleration today and the option of a different module later — the only laptop on the market where the GPU is not a permanent decision. The catch is the same 8GB VRAM ceiling as the 4070 laptop part, so 14B models will not fit on the GPU.

Tuxedo and System76 machines with NVIDIA discrete GPUs behave exactly like their Windows equivalents; the story is entirely software. Install Ollama, install a current NVIDIA driver, done. If something does not light up, the Ollama troubleshooting guide covers the usual Linux causes.

Why Does My Laptop Slow Down After Ten Minutes?

Because the spec sheet quotes a boost figure and your chassis has to dissipate a sustained one. Vendors publish "total graphics power up to" numbers for exactly this reason: a 140W-rated mobile GPU in a 14-inch shell can hit 140W briefly, while the aluminium acts as a thermal capacitor, and then must settle at whatever the fans and heat pipes can actually remove. The first sixty seconds of any benchmark are measuring the capacitor, not the cooling system.

That produces three rules that hold regardless of brand:

• Chassis volume beats silicon. A 16-inch machine with a vapour chamber sustains a higher share of its rated power than a 14-inch one with the same chip. If you do sustained work — batch summarisation, long-context RAG, LoRA training — buy the bigger shell.
• Short bursts barely notice. Conversational chat is a few seconds of load at a time, so the throttled regime never arrives. The throttle tax only lands on continuous generation.
• Unplugged is a different machine. The firmware clamp on discrete-GPU power costs far more throughput than any thermal effect.

If you want the number for your specific laptop rather than someone else's, take it yourself. This logs throughput and GPU state during a sustained run:

# Shell 1: continuous generation
while true; do
  ollama run llama3.1:8b --verbose "Write a 500-word story about distributed systems."
  sleep 2
done

# Shell 2: log GPU temperature, power draw and clocks every 30s (NVIDIA)
while true; do
  echo "$(date +%T) $(nvidia-smi --query-gpu=temperature.gpu,power.draw,clocks.gr --format=csv,noheader)"
  sleep 30
done > thermal-log.txt

# macOS equivalent: package power and GPU residency
sudo powermetrics --samplers cpu_power,gpu_power -i 30000 > thermal-log.txt

The --verbose flag makes Ollama print eval rate for every response, so shell 1 gives you your own tokens-per-second series and shell 2 tells you whether a decline is thermal, power-limit, or neither. Compare minute one against minute thirty and you have a number that is true for your machine, which is the only machine that matters. If throughput is low from the very first token, the cause is usually configuration rather than heat — local LLM slow? here is the fix works through the usual suspects.

How Long Does a Laptop Battery Last Under Inference?

Another division:

runtime (hours) ~= battery capacity (Wh) / average system draw (W)

Battery capacity is published, and it is capped by regulation rather than engineering: airline carry-on rules effectively limit laptops to 100Wh, which is why nothing ships bigger. Apple lists 99.6Wh for the 16-inch MacBook Pro and 72.4Wh for the 14-inch; most large gaming laptops sit at 90-99.9Wh.

The draw figure is the one you need to measure, and it varies enormously by platform. Work it through with the numbers vendors do publish and the shape is obvious: a machine whose discrete GPU alone is rated at 140-175W cannot draw less than about 100W of total system power while that GPU is loaded, so a 99.9Wh pack gives you well under an hour. An Apple Silicon laptop running the same model draws a fraction of that, which is why M-series machines are the only ones where untethered inference is a real mode of use rather than a demo.

Get your own draw figure rather than trusting anyone's table:

# macOS: combined CPU + GPU package power, sampled every 5s
sudo powermetrics --samplers cpu_power,gpu_power -i 5000

# Windows / Linux with an NVIDIA dGPU: GPU draw only, add ~20-30W for the rest
nvidia-smi --query-gpu=power.draw --format=csv -l 5

# Linux, whole-battery discharge rate in watts
cat /sys/class/power_supply/BAT0/power_now   # microwatts

Divide your pack's Wh by the watts you see and you have your runtime. Two practical notes: ARM laptops (Snapdragon X Elite, and the Copilot+ shelf generally) run LLM inference on the CPU because Ollama and llama.cpp do not target the Adreno GPU, which is slow but extremely frugal — the longest untethered runtimes in this whole category. And discrete-GPU laptops on battery are slower and short-lived at the same time, which is the worst of both.

What Actually Helps: Power Modes and Cooling

Short list, and deliberately shorter than most:

1. Set the power mode correctly. This is free and it is the biggest single lever. On Windows, Best Performance in the battery slider plus Maximum Performance in the NVIDIA Control Panel. On macOS, make sure Low Power Mode is off. On Linux, tlp with CPU_SCALING_GOVERNOR_ON_AC=performance.

2. Stay on AC for any session over a few minutes. The most boring advice here and the most important, because the on-battery discrete-GPU clamp costs more performance than every other item on this list combined.

3. Give the intakes room. Laptop intakes are on the underside. A hard flat desk is fine; a duvet is not, and a riser that lifts the rear by a centimetre or two restores clearance a soft surface takes away. Whether a powered cooling pad does anything beyond that depends entirely on where your particular chassis draws air — some machines benefit, some are unaffected, and nobody can give you a number that generalises.

4. Keep an eye on paste and dust after a couple of years. Thermal compound degrades and intake filters clog; both show up as lower sustained clocks. Servicing is a real fix, and the size of the improvement depends on how bad the machine had got, so treat any specific figure you read with suspicion.

5. Pick a bigger chassis at purchase time. Everything above is worth a few percent. Chassis volume is worth more than all of it, and you only get to choose it once.

Which Laptop Should I Buy for Local AI?

Buy on memory capacity first, memory bandwidth second, chassis size third. Prices move monthly and vary by region, so the table lists the configuration to look for rather than a number that will be stale by the time you read it.

Three rules that fall out of the whole guide:

1. Over-buy memory, not GPU tier. On Apple it is soldered and permanent. On Windows, more system RAM will not raise the VRAM ceiling that actually gates you — so the jump from a 4070 laptop (8GB) to a 4080 laptop (12GB) matters far more than the marketing gap between them suggests.
2. Prefer 16-inch over 14-inch for sustained work. Same chip, more cooling, more of the rated power retained.
3. Avoid buying a gaming laptop for battery-mode AI. It is a good AC machine and a poor portable one, and no setting changes that.

For desktop alternatives at similar money, compare against the Mac Studio vs PC build guide and the best mini PC for Ollama. If you are shopping the Copilot+ / NPU shelf specifically, the NPU laptop table ranks the currently-shipping machines by memory ceiling and vendor-published bandwidth rather than by TOPS.

External reference: Tom's Hardware laptop reviews run thermal-camera tests on most of these chassis if you want independent measurements of a specific model.

---

FAQ

Can I run Llama 3 70B on a laptop?

Only on Apple Silicon with at least 48GB of unified memory, because the Q4 weights are roughly 40GB and everything else on the market caps out lower. Divide the chip's published bandwidth by 40GB for the ceiling: an M4 Pro at 273 GB/s comes to about 7 t/s and an M4 Max at 546 GB/s to about 14 t/s, with real output below both. No Windows or Linux laptop in 2026 ships 48GB of discrete VRAM, and CPU inference of a 70B over ~90 GB/s of DDR5 has a ceiling near 2 t/s — slower than reading speed.

Why does my laptop slow down after ten minutes of AI use?

The spec sheet quotes a boost power figure your chassis cannot sustain. The first minute runs against the thermal mass of the aluminium; after that the machine settles at whatever the fans and heat pipes can actually remove, which in a 14-inch shell is a lot less than the rated total graphics power. Bigger chassis hold a higher share of the rated figure. To find your own number, log ollama run --verbose output alongside nvidia-smi or powermetrics and compare minute one with minute thirty.

How long does Ollama drain my battery?

Divide your battery capacity by your average draw. Capacity is published — 99.6Wh for a 16-inch MacBook Pro, 72.4Wh for the 14-inch, 90-99.9Wh for most large gaming laptops, all bounded by the 100Wh airline limit. Draw you should measure with powermetrics on macOS or nvidia-smi --query-gpu=power.draw on a discrete-GPU machine. A laptop whose GPU alone is rated at 140-175W will not last an hour; an Apple Silicon machine at a fraction of that draw is the only one where untethered inference is genuinely practical.

Is 16GB of RAM enough for laptop local AI?

For 7-8B models, yes. With a discrete 8GB GPU an 8B model sits entirely in VRAM, which is the configuration you want. Without a discrete GPU, the same model runs on CPU against roughly 90 GB/s of DDR5 — usable, not snappy. What 16GB will not do is a 14B on the GPU: at roughly 9GB of weights plus context it overflows an 8GB card, and the moment layers spill to system RAM the effective bandwidth for those layers collapses. For comfortable 14B headroom you want 32GB of RAM with a 12-16GB discrete GPU, or 36GB+ of Apple unified memory.

Does the Apple Neural Engine help with Ollama?

No. Ollama uses Metal GPU acceleration; the ANE serves CoreML models and some vision and audio workflows instead. This is not a problem to solve — Apple Silicon GPUs paired with high-bandwidth unified memory are already the strongest thing in a laptop for this workload. It does mean a chip's advertised TOPS figure tells you nothing useful about token throughput.

Can I run AI on a Snapdragon X Elite, Lunar Lake, or Panther Lake laptop?

Yes, on the CPU, for 3B and 7B models, with excellent battery life. The X Elite's published 135 GB/s of LPDDR5X gives an 8B model a ceiling near 29 t/s, and the Lunar Lake class of LPDDR5X is in the same neighbourhood. The blocker is not memory, it is software: neither the integrated NPU nor the Adreno and Arc graphics are routed by Ollama or llama.cpp for LLM inference, so you get none of the accelerator you paid for. Buy these machines for general productivity with AI as a bonus, not as a primary AI workstation.

Do NPUs help with local LLMs yet?

Mostly not. Apple's Neural Engine, Intel's NPU and Qualcomm's NPU all exist and all advertise impressive TOPS, but the popular GGUF toolchain routes inference through Metal, CUDA, ROCm or Vulkan instead. NPUs do earn their keep on small specialised models — Whisper variants, some DirectML-targeted builds — just not on the mainstream chat models. Treat a TOPS number as marketing until your specific runtime says otherwise.

Are eGPU enclosures worth it for laptops?

Not on Apple Silicon: macOS dropped eGPU support after the Intel era, so this is a Windows and Linux question only. On those, the relevant spec is the link: Thunderbolt 4 provides 40 Gb/s (about 5 GB/s) and OCuLink exposes PCIe 4.0 x4 at 64 Gb/s (about 8 GB/s). Both are far slower than the GPU's own memory, which sounds fatal and mostly is not — once the weights are resident in the eGPU's VRAM, decode never crosses the cable, so the link cost lands on model loading rather than generation. Setup is fiddly and dock compatibility is real. See our eGPU local AI notes for the details.

---

Conclusion

Laptop local AI is real, but the marketing version and the lived version are different products. Plugged in, against the heat sink, with full power, any modern AI laptop handles an 8B model comfortably. Unplugged, fifteen minutes in, in a 14-inch chassis, it is a different machine — and no setting recovers what the firmware clamp takes away. Apple Silicon hides that gap better than anything else, which is why M-series machines became the default here, and unified memory is why they are the only portables that address a 70B at all.

The honest plan for most people: buy the biggest memory configuration you can afford, prefer a 16-inch chassis, and assume you will mostly work on AC. Treat battery use as bonus capability rather than the primary mode. And when you see a tokens-per-second figure in a review, divide the platform's published bandwidth by the model's file size before you believe it.

If a single takeaway saves you a mistake: do not buy a 14-inch gaming laptop and expect it to hold its review-day numbers. Buy sixteen inches, spend the difference on memory, or buy a desktop and reach it from your current laptop over Tailscale.