Does Ollama Use the Apple Neural Engine? ANE vs Metal

No. On a Mac, Ollama runs models on the GPU — through Metal, and now optionally through Apple's MLX framework — and never on the Neural Engine. Ollama's own macOS documentation states the requirement as "Apple M series (CPU and GPU support) or x86 (CPU only)": CPU and GPU, with no third option listed. The Core ML request that would change that, issue #3898, has been open since April 2024 with no backend merged. An ANE reading of zero while tokens stream is correct behaviour, not a setting you failed to find.

That is the whole answer, and it is worth understanding rather than just accepting, because the follow-up questions are the interesting ones: why can't a runtime simply target the ANE, what is that block of silicon doing instead, and is there anything you can install today that genuinely lights it up? All three have concrete answers.

If you landed here from the Windows side of the same confusion — a Copilot+ laptop sold on its TOPS rating whose NPU graph never moves — the equivalent write-up is why your laptop's NPU sits at 0% in Ollama. The cause is structurally identical across both platforms: dedicated inference accelerators are reached through vendor-specific compiled model formats, not through general-purpose inference engines.

One thing this page does not do: publish an ANE-versus-Metal speed comparison. We did not benchmark one, we are not going to invent one, and the check-it-yourself section gives you the powermetrics procedure to see what your own Mac is doing instead.

Why is my Neural Engine idle while Ollama runs?

Three layers all have to support the ANE for your tokens to reach it, and none of them do.

Layer one — Ollama itself. The Ollama GPU documentation lists the accelerators it supports: NVIDIA GPUs at compute capability 5.0+, AMD Radeon via ROCm, Metal on Apple devices, and Vulkan on Windows and Linux. The Apple entry is a single sentence — "Ollama supports GPU acceleration on Apple devices via the Metal API" — and neither Core ML nor the Neural Engine appears anywhere on the page. The macOS page is blunter still: the system requirement reads "Apple M series (CPU and GPU support) or x86 (CPU only)".

Layer two — the engines underneath. llama.cpp, which Ollama vendors (its repository carries a llama/ directory alongside the newer ml/ and mlx/ ones), publishes a backend table in its README: Metal targets Apple Silicon, CUDA targets NVIDIA, Vulkan targets GPUs, Hexagon (in progress) targets Snapdragon. There is no Core ML row and no ANE row. Its Apple statement is about a different set of frameworks entirely: "Apple silicon is a first-class citizen - optimized via ARM NEON, Accelerate and Metal frameworks."

Layer three — the ANE's API surface. This is the layer that actually blocks everyone. The Neural Engine is reachable only through Core ML; there is no public low-level API to compile a kernel and dispatch it the way you would with Metal or CUDA. Apple's own MLX team said so when they were asked directly. MLX issue #18, "ANE support", was closed in December 2023 by maintainer awni with: "at the moment we don't have plans to support ANE in MLX given it is a closed source API. If / when that changes we will be first in line to add it as a supported device."

That quote is the load-bearing fact on this page. When the framework team inside Apple's own machine learning group says the API is closed to them, no amount of configuration in a third-party tool is going to route around it.

The 2026 twist: yes, Ollama has MLX now — and MLX is still the GPU

This trips people up, so it is worth stating clearly. Ollama issue #1730, "MLX backend", was closed as fixed on 24 July 2026 by maintainer dhiltgen: "MLX support has been in for a while, and we're continuing to improve the engine, and add new models." The repository now carries an mlx/ directory, MLX-tagged model names show up in the tracker, and new MLX model PRs land regularly.

None of that is an ANE path. MLX's own README says it plainly: "Operations can run on any of the supported devices (currently the CPU and the GPU)." MLX gives you a different, Apple-native way to run the same matrix multiplications on the same GPU, with a unified-memory model — "Arrays in MLX live in shared memory. Operations on MLX arrays can be performed on any of the supported device types without transferring data." Two engines, not three. If you want the comparison against the other side of the fence, we cover it in MLX vs CUDA for local AI.

Is there an Ollama Core ML backend, or a flag I missed?

There is no flag. There is a feature request that has been open since April 2024, and a closed duplicate. Here is the entire Ollama tracker history on this question, so you can stop searching:

Two practical consequences. First, there is no environment variable, Modelfile directive or CLI flag that enables the ANE in Ollama — a backend that does not exist cannot be toggled. Second, #3898 is genuinely open, so this page has a shelf life: click through before you believe it. The discussion on that issue is mostly people pointing at adjacent projects (MLX, MLC, Metal flash-attention work, ANEMLL), which is a fair summary of where the energy has actually gone.

Which Mac AI tools actually use the Neural Engine?

This is the table worth bookmarking. The pattern it exposes: everything that runs a large language model targets the GPU, and everything that reaches the ANE does it by shipping a converted Core ML model.

Read the right-hand column as a rule: ANE access is a model-format decision, not a runtime setting. You do not point an existing engine at the Neural Engine; you convert a model into Core ML and hand it to the operating system, which then decides what to schedule where.

What is the Apple Neural Engine actually for?

Apple's own description is precise: "the energy-efficient and high-throughput engine for ML inference on Apple silicon." Note which word is missing — general-purpose. The ANE is purpose-built inference hardware with strong opinions about what it will accept.

Apple's machine learning research write-up, Deploying Transformers on the Apple Neural Engine, is the best public description of those opinions, and it explains why "just add an ANE backend" is not a weekend project:

• The data layout is not the one your model uses. "The most conducive data format for the ANE (hardware and software stack) is 4D and channels-first" — a (B, C, 1, S) tensor, rather than the (B, S, C) that a PyTorch transformer hands you.
• Buffers have alignment rules. "The last axis of an ANE buffer is not packed; it must be contiguous and aligned to 64 bytes."
• Layers have to be rewritten, not merely converted. Apple's first principle is to replace nn.Linear layers with nn.Conv2d layers to get into that 4D channels-first format.
• Attention has to be restructured. Their guidance is to "split the query, key, and value tensors to create an explicit list of single-head attention functions, each of which operates on smaller chunks of input data," and to use a specific einsum formulation (bchq,bkhc->bkhq) to avoid intermediate transposes and reshapes.

That is a per-architecture porting effort, repeated for every new model family, ending in a compiled artifact with largely fixed shapes. Compare it with a GGUF file, which any llama.cpp build loads at any context length. The economics of open-source maintenance explain the outcome on their own.

The Core ML dial, and what it does not offer

When an app does use Core ML, the only control it has over placement is MLComputeUnits. Apple's documentation defines four options:

Look at what is absent: there is no neuralEngineOnly. Even the most ANE-forward option pairs it with the CPU, because unsupported layers have to fall back somewhere. The developer expresses a preference; the operating system schedules the work. That is a very different contract from --n-gpu-layers 99.

How do I check what my ANE is doing right now?

powermetrics is the built-in answer, and it needs sudo. Its man page states that the tool "may also display estimated power consumed by various SoC subsystems, such as CPU, GPU, ANE (Apple Neural Engine)."

Run this in one terminal:

sudo powermetrics --samplers cpu_power -i 1000

ANE power is reported by the cpu_power sampler, not by a sampler of its own. That is not a guess: asitop, the nvtop-style Apple Silicon dashboard that advertises "ANE utilization (measured by power)", launches powermetrics with --samplers cpu_power,gpu_power,thermal and reads the ane_energy field out of that output.

Then run the three-part test:

1. Baseline. Watch the idle numbers for ten seconds with nothing running. ANE power on an idle Mac is the floor you are comparing against.
2. Generate with Ollama. In a second terminal, ollama run a model and give it a long prompt. GPU power should climb and stay up while tokens stream. The ANE figure should not move meaningfully. That is the observation this whole page explains.
3. Positive control. Transcribe something with a whisper.cpp build compiled using -DWHISPER_COREML=1, or any Core ML app, and watch the ANE figure during the encoder pass. Without a positive control you cannot distinguish "the ANE is idle" from "my monitoring is wrong", and that distinction is the entire point of the exercise.

Two caveats, both from primary sources. First, the powermetrics man page warns that average power values "are estimated and may be inaccurate - hence they should not be used for any comparison between devices, but can be used to help optimize apps for energy efficiency" — treat this as a signal detector, not a benchmark instrument. Second, whisper.cpp's README notes that the first Core ML run on a given device is slow because "the ANE service compiles the Core ML model to a device-specific format" — so if your positive control shows nothing on the first attempt, run it twice.

If you prefer a dashboard to a wall of text, pip install asitop then sudo asitop gives you CPU, GPU and ANE alongside each other.

Would the Neural Engine even be faster for LLMs?

We have not measured it, and we are not going to publish a number we did not produce. What we can do is set out the arithmetic that constrains the answer, because it explains why nobody has been in a hurry to build this.

Token generation is limited by memory bandwidth, not by arithmetic. Producing each new token requires reading the model's weights out of memory once. So the ceiling is simple division: memory bandwidth divided by the size of the weights. Take an 8B model at Q4_K_M, using the standard rule of thumb of about 0.6 bytes per parameter for that quantisation (roughly 4.8 bits per weight once the quantisation metadata is counted): 8 billion × 0.6 bytes ≈ 4.8GB of weights to read for every single token. Against Apple's published memory bandwidth figures:

Bandwidth figures from Apple's M4 Pro and M4 Max announcement. The tokens-per-second column is arithmetic, not a measurement — real throughput is lower once KV-cache reads, sampling and framework overhead are included. Our memory bandwidth guide works through why this ratio predicts local LLM speed so well.

Now the key point: the ANE and the GPU read from the same unified memory pool. Moving the multiply-accumulate work from one engine to the other does not raise that ceiling. A faster arithmetic unit attached to the same memory bus produces the same tokens per second, which is precisely why "38 trillion operations per second" — Apple's stated figure for the M4's Neural Engine — does not translate into a tokens-per-second promise. TOPS rates a block of silicon; it says nothing about whether any runtime can reach it, and an unreachable 38 TOPS delivers exactly zero tokens per second.

Where an ANE path could plausibly help is the other half of the workload. Prompt processing (prefill) is compute-bound rather than bandwidth-bound, and so are encoder-style models. That is not speculation about hardware we do not own — it is exactly the split whisper.cpp implements: the encoder goes to the ANE, and its README claims "more than x3 faster compared with CPU-only execution." Read that comparison carefully: it is against CPU-only, not against Metal.

The honest framing for a Mac owner is therefore the one the NPU story on Windows also lands on. The reason to want work on a dedicated inference engine is power draw and leaving the GPU free for everything else — Apple's own word for the ANE is "energy-efficient" — not peak generation speed.

What can I run today that does use the ANE?

If your goal is "make that silicon do something", pick a workload it was designed for. Four concrete options:

1. Speech-to-text with whisper.cpp's Core ML encoder. The most mature ANE path in the local-AI world. Generate the Core ML model and build with the flag:

./models/generate-coreml-model.sh base.en
cmake -B build -DWHISPER_COREML=1
cmake --build build -j --config Release

Per its README, "on Apple Silicon devices, the Encoder inference can be executed on the Apple Neural Engine (ANE) via Core ML." Our local Whisper speech-to-text guide covers models and quality tiers, and the best speech-to-text on Mac roundup compares the Mac-specific options.

2. WhisperKit — Argmax's MIT-licensed Swift framework, built on Core ML, requiring macOS 14 or later. If you are writing a Swift app rather than driving a CLI, this is the ANE-native route.

3. ANEMLL, for LLMs specifically. ANEMLL describes itself as "an open-source project focused on accelerating the porting of Large Language Models (LLMs) to tensor processors, starting with the Apple Neural Engine (ANE)." It converts models — LLaMA 3.1/3.2, Qwen 2.5 and Qwen 3, Gemma 3, DeepSeek-R1 distills and others — into compiled Core ML .mlmodelc bundles that run on the ANE. It is MIT licensed and it is the closest thing that exists to "an LLM on the Neural Engine".

Be clear-eyed about what you are signing up for: the project's own README labels it Beta Release 0.3.5, states that "quantization should be improved. LUT4 quality is fairly low due to lack of Block Quantization on Apple Neural Engine", and flags that some models — Gemma 3 in particular — "produce activations that exceed this range, causing NaN/Inf failures" in FP16. It is a conversion pipeline and a research project, not an ollama pull replacement. It is also the reason to keep an eye on this space rather than writing the ANE off entirely.

4. Core ML models generally — image classification, vision, embeddings, the everyday stuff Core ML was built around. Set MLComputeUnits.all and let the OS place the work.

What should I do on my Mac instead?

1. Stop looking for the setting. There is no ANE toggle in Ollama, LM Studio, llama.cpp or MLX. The Core ML request has sat open and unmerged since April 2024.
2. Judge your Mac on memory bandwidth and memory capacity, which is what the arithmetic above actually depends on. Our Apple Silicon AI buying guide and the M4 for AI guide are organised around exactly that, and the Neural Engine's TOPS rating deserves no weight in the decision.
3. Try MLX-tagged models in Ollama. They are a genuinely different execution path with a different performance profile — still on the GPU, but worth measuring against the Metal path on your own machine.
4. Watch your memory ceiling, not your ANE. On unified memory the thing that actually breaks your setup is running out of it; Mac memory pressure with local LLMs covers the symptoms and the fix, and the Mac local AI setup guide is the general starting point.
5. If the ANE is genuinely your goal, change the workload. Speech and vision have working ANE paths today. Text generation does not.

Honest limitations

• We did not benchmark ANE against Metal on any Mac. There is no tokens-per-second comparison on this page because we did not run one, and the arithmetic above is arithmetic, clearly labelled as such. The powermetrics procedure exists so you can produce evidence for your own machine instead of trusting ours.
• Issue #3898 is open. If a Core ML backend ever lands in Ollama, the headline answer here changes. Check the issue before you rely on this page.
• powermetrics reports estimates. Its own man page says the values "should not be used for any comparison between devices". Use it to detect activity, not to rank hardware.
• ANEMLL is beta software, on its own account, with quantisation quality caveats it publishes itself. We have not run it.
• We cannot enumerate what else on your Mac uses the ANE. macOS schedules Core ML work from the system and from third-party apps, and the only honest way to know what is running on yours is to watch the ANE power line while you use it.

FAQ

Does Ollama use the Apple Neural Engine?

No. Ollama's macOS documentation lists the requirement as "Apple M series (CPU and GPU support) or x86 (CPU only)", and its GPU documentation describes Apple acceleration as being "via the Metal API". There is no Core ML backend, no ANE backend, and no flag to enable one. The request to add Core ML support, issue #3898, has been open since April 2024.

Ollama supports MLX now — does that mean it uses the Neural Engine?

No, and this is the most common 2026 misunderstanding. MLX support landed in Ollama (issue #1730 was closed as fixed in July 2026), but MLX itself targets "currently the CPU and the GPU", in its README's words. MLX is Apple's array framework for Apple Silicon, not an ANE access layer. Apple's MLX team explicitly declined ANE support in issue #18 because the API is closed-source.

Is there a Core ML backend for llama.cpp or LM Studio?

Not for text generation. llama.cpp's backend table lists Metal for Apple Silicon with no Core ML entry, and LM Studio's Apple-native engine is MLX, which by MLX's own README targets the CPU and the GPU. The one Core ML path in the ggml family is in whisper.cpp, and it covers the Whisper encoder only, built with -DWHISPER_COREML=1.

Can I convert a GGUF into a Core ML model and run it on the ANE?

Not by flipping a switch. Apple's own guidance for putting transformers on the ANE involves rewriting nn.Linear layers as nn.Conv2d, moving to a 4D channels-first (B, C, 1, S) layout with 64-byte-aligned buffers, and splitting attention into explicit single-head chunks. That is a per-architecture porting job ending in a compiled Core ML artifact. ANEMLL exists precisely because someone has to do that work model by model.

How do I tell whether anything is using my Neural Engine?

Run sudo powermetrics --samplers cpu_power -i 1000 and watch the ANE power figure, or install asitop for a dashboard version of the same data. Establish an idle baseline, then run your workload. Crucially, test a known-good Core ML workload as a positive control, otherwise you cannot tell an idle ANE from broken monitoring.

Would using the ANE make local LLMs faster on my Mac?

Unknown to us — we have not found a like-for-like ANE-versus-Metal comparison for text generation anywhere, and we have not run one ourselves. What is knowable is the constraint: token generation is bounded by memory bandwidth divided by model size, and both engines share the same unified memory, so switching engines does not raise that ceiling. Prompt processing is compute-bound and is the part where a dedicated engine could plausibly help — which is exactly the half of Whisper that whisper.cpp offloads to the ANE.

Should I buy a Mac with a better Neural Engine for local AI?

No. Choose on unified memory capacity (which models fit) and memory bandwidth (how fast they generate). The Neural Engine's TOPS rating does not predict local LLM performance, because no LLM runtime reaches it. Apple's stated 38 trillion operations per second for the M4's Neural Engine is a peak arithmetic rating for a block of silicon your inference stack never touches.

Sources

• Ollama GPU documentation and Ollama macOS documentation — supported accelerators and the "CPU and GPU support" system requirement
• Ollama issues #3898 (Core ML request, open since Apr 2024), #4817 (closed as duplicate) and #1730 (MLX backend, closed as fixed Jul 2026)
• MLX README and MLX issue #18 — supported devices, and the maintainer statement on the closed ANE API
• llama.cpp README — the backend table and the Apple Silicon statement
• Apple Machine Learning Research — Deploying Transformers on the Apple Neural Engine — data layout, alignment and attention-chunking requirements
• Apple Developer — MLComputeUnits — the four compute-unit options, quoted verbatim
• whisper.cpp README — Core ML encoder path, -DWHISPER_COREML=1, first-run compilation note
• ANEMLL and WhisperKit — the two projects that do target the ANE
• asitop and the macOS powermetrics man page — how ANE activity is actually observed, and the accuracy caveat
• Apple Newsroom — M4 Pro and M4 Max (120/273/546 GB/s) and M4 announcement (38 TOPS Neural Engine)