Unified Memory for Local AI: 32GB to 256GB Table

Size the machine to the largest model you want fully resident, then subtract the carve-out — because the GPU never gets the whole sticker figure. The arithmetic that decides it is short: a 4-bit K-quant averages a little under 5 bits per weight, so 5 ÷ 8 = 0.625 bytes per parameter, or roughly 0.6GB per billion parameters. That makes a 70B dense model about 42GB before any KV cache, which is why 64GB is the first tier where a 70B fits at all and 96GB is the first where it fits comfortably. Below that, 32GB is a 32B-class machine and 48GB is a 32B-at-Q8 machine. Above it, 128GB buys you 70B at 8-bit or a 120B-class mixture-of-experts, and 256GB is where 235B-class MoE models become resident. The trap in every tier: unified memory buys capacity, not bandwidth, so a big model will load and then generate slowly.

This is a sizing reference, not a review. Every capacity and bandwidth figure below is quoted from the vendor's published specifications, and every model-fit number is arithmetic you can redo in your head — there are no measured tokens per second on this page, because we do not have a Mac Studio, a DGX Spark and a Strix Halo box on a bench to measure them on. Where a number would have to be invented, it is left out.

How Much Unified Memory Does a Model Actually Need?

Three numbers, added together.

1. Weights. Divide the bits per weight by 8 to get bytes per parameter:

The exact rows come out of the block formats. Q8_0 stores 32 weights as 32 int8 values plus one 16-bit scale, so (32 + 2) bytes ÷ 32 weights × 8 = 8.5 bits per weight — that one is arithmetic, not an estimate. The K-quant rows are the approximate block averages llama.cpp reports, and they vary by a few percent between models depending on how much of the tensor budget sits in embeddings.

So a 70B at Q4_K_M is 70 × 0.6 = 42GB. A 32B is 32 × 0.6 = 19.2GB. A 235B is 235 × 0.6 = 141GB. Those three products carry most of the buying decision on their own. If the quantisation names are unfamiliar, start with quantization explained — the format you pick moves the memory requirement by a factor of three or more.

2. KV cache. This is the part people forget, and it scales with context, not with model size alone. Per token, an FP16 cache costs 2 (one K, one V) × 2 bytes × layers × KV heads × head dimension. For a 70B-class model configured with 80 layers, 8 grouped-query KV heads and a 128-wide head:

2 x 2 bytes x 80 layers x 8 KV heads x 128 dims = 327,680 bytes per token
327,680 bytes x 32,768 tokens = 10,737,418,240 bytes = ~10 GB at 32K context

Ten gigabytes for the context window alone, on top of 42GB of weights. That single calculation is why a 64GB machine running a 70B feels tight and a 48GB machine cannot do it at all.

3. Everything else. The operating system, the desktop compositor, the browser you will inevitably have open, and the runtime's own working buffers. Budget 8GB on a lean Linux box and more on macOS or Windows.

Which Capacity Tier Fits Which Model?

Usable memory in this table assumes roughly 8GB set aside for the OS and runtime overhead — deliberately conservative rather than best case. Model sizes are the 0.6 GB-per-billion arithmetic above.

The two thresholds that matter. Going 32GB to 64GB changes what class of model you can run — that is a capability jump. Going 64GB to 128GB mostly buys comfort, concurrency and context on models you could already load. If your budget forces one upgrade, the 64GB step is the one that changes the answer to "can it run this?".

Where the value sits. For a single-model machine, 64GB is the cheapest tier that reaches 70B. For a machine you will keep for three years and load more than one thing into, 96GB or 128GB is the honest recommendation. 256GB is a specialist purchase; if you are not already certain you need a 235B-class model resident, you do not.

Why Is Usable Memory Smaller Than the Sticker?

Because unified memory is shared, and the GPU is only ever allowed a slice of it. Each platform draws the line differently, and each one exposes the line so you can read it rather than guess.

Apple Silicon. macOS caps how much memory can be wired for GPU use. Metal exposes the ceiling as recommendedMaxWorkingSetSize, and Apple's MLX documentation is explicit about both the constraint and the knob: the wired limit "must remain strictly less than the total available system memory size," and you raise the system limit with

sudo sysctl iogpu.wired_limit_mb=<size_in_megabytes>

MLX's device_info() returns both "max_recommended_working_set_size" (the system wired limit) and "memory_size" (your total), so you can read the real ratio on your own machine in one command instead of trusting a figure from an article. Note that the sysctl is a runtime setting: it does not survive a reboot on its own, and raising it removes a safety margin rather than creating memory.

AMD Strix Halo / Ryzen AI Max+. The iGPU's pool is set by firmware and the driver rather than by the OS at runtime. Framework publishes the ceiling for its Ryzen AI Max Desktop directly: "up to 96GB of graphics addressable memory" on the 128GB configuration. That is the number to plan against — not 128GB. On Linux, ROCm 7.14.0 lists gfx1151 (Strix Halo) as a supported target, which is what makes this platform viable at all; the wider vendor picture is in our GPU support matrix, and the platform itself in the Strix Halo guide.

NVIDIA DGX Spark. NVIDIA specifies "128 GB LPDDR5x, coherent unified system memory" and claims inference on "AI models up to 200 billion parameters," with fine-tuning up to 70 billion. Coherent means CPU and GPU address the same pool without an explicit copy; it does not mean the model gets all 128GB.

The practical rule across all three: plan on the published GPU-addressable figure where the vendor gives one, and on sticker minus 8-16GB where they do not.

What Do the Platforms Actually Ship?

Capacities and bandwidths quoted from vendor specification pages. Bandwidth is included because it is the number that decides how the machine feels, not because faster is the point of this table.

Note the spread in that bandwidth column: 819 GB/s at the top and 256 GB/s at the bottom, a factor of 3.2 across machines that all advertise "unified memory." Capacity tiers are comparable across platforms; the speed at which those tiers are usable is not. Mini-PC options at the AMD end are compared in our Strix Halo mini PC roundup, and the Apple-versus-PC framing in Mac Studio vs a PC AI build.

Does More Unified Memory Make Models Faster?

No — and this is the caveat that belongs on every unified-memory buying page.

Generating one token from a dense model requires reading every weight once. So the arithmetic ceiling on decode speed is simply bandwidth divided by the size of the weights, before any overhead at all:

Read that table carefully: those are upper bounds derived from published bandwidth, not measurements. Real output is lower — attention, sampling, memory-controller efficiency and thermal behaviour all take a cut, and none of that is captured here. What the arithmetic does establish is the shape of the problem: buying 128GB so a 70B fits does not buy you a 70B that is pleasant to chat with, and no amount of extra capacity changes the division.

This is also the case for mixture-of-experts. An MoE model only reads its active parameters per token, so a 235B model with 22B active reads roughly 22 × 0.6 = 13.2GB per token instead of 141GB. On a 256 GB/s machine that is 256 ÷ 13.2 ≈ 19 tok/s as a ceiling instead of about 1.8. MoE is the model shape unified memory was made for: it spends the capacity you paid for and skips the bandwidth you did not. If you are choosing a machine primarily for large models, weight your shortlist toward MoE-friendly capacity rather than dense-model capacity. The bandwidth mechanics are covered in more depth in GPU memory bandwidth and local LLMs.

Which Tier Should I Buy?

• You want one strong coding or chat model, always loaded. 32GB. A 32B at Q4 is 19.2GB and leaves room to work.
• You want that model at higher fidelity rather than a bigger one. 48GB gets a 32B to Q8 at 35.2GB.
• You specifically want a 70B resident. 64GB is the floor and it is a tight floor. 96GB is the version of that machine you will not regret.
• You want two models loaded at once, or 70B at 8-bit. 128GB.
• You want 235B-class MoE models resident. 256GB, which today means Apple Silicon.
• You are not sure. Buy the tier that fits the largest model you have actually run and enjoyed, plus one step. Unified memory is soldered on every platform in this table — there is no upgrade later, which is the whole reason this decision is worth an hour.

Cross-check against the discrete-GPU route before committing: for models that fit in 24GB or 32GB of VRAM, a graphics card is far faster, and the VRAM requirements reference is the comparison. Unified memory wins specifically where the model does not fit on a card at all. For Apple configurations in particular, the Apple Silicon buying guide covers chip tiers alongside capacity, and 32 vs 64 vs 128GB covers the same question for conventional system RAM.

What This Table Cannot Tell You

• No measured throughput appears here. The only speed figures on this page are divisions of a published bandwidth number by a computed model size, and they are labelled as ceilings. Anyone quoting you exact tokens per second for a specific model on a specific unified-memory box should be able to show you the run.
• Bits per weight vary by model. The 0.6 GB-per-billion rule comes from a 4-bit K-quant averaging a little under 5 bits per weight. Individual quantisations differ by a few percent, and models with unusually large embedding or vocabulary tensors differ by more. It is a planning figure, not a guarantee.
• KV cache maths depends on the architecture. The worked example uses 80 layers, 8 KV heads and a 128-wide head. A model without grouped-query attention costs several times more per token, and a cache quantised to 8-bit costs about half. Check your model's configuration rather than assuming.
• Vendor allocation ceilings change with firmware and OS releases. Framework's 96GB graphics-addressable figure and Apple's wired-limit behaviour are both current published positions, not physical constants. Read your own machine before you plan around a number.
• Nothing here says a tier is "enough" for image or video generation. Those workloads have different resident-memory profiles and are not modelled by this table.

FAQ

How much unified memory do I need to run a 70B model?

64GB is the practical floor and 96GB is the comfortable answer. A 70B at Q4_K_M is about 42GB of weights (70 × 0.6), and a 32K context on a typical 70B architecture adds roughly 10GB of KV cache, which puts you at ~52GB before the operating system takes its share. On a 64GB machine that works but leaves nothing spare; on 96GB it is relaxed and you can push context much further.

Is 128GB of unified memory worth it over 64GB?

It depends on whether you need capability or comfort. 64GB already reaches a 70B dense model, so 128GB does not unlock a new class of dense model — 235B at Q4 is 141GB and still does not fit. What 128GB buys is a second model loaded at the same time, a 70B at 8-bit instead of 4-bit (74GB), a 120B-class MoE, and far longer contexts. If you are running one model at a time, put the money into bandwidth or a discrete GPU instead.

How much of my unified memory can the GPU actually use?

Less than all of it, and the number is platform-specific. On Apple Silicon it is exposed as recommendedMaxWorkingSetSize, readable through MLX's device_info() alongside your total memory_size, and raisable with sudo sysctl iogpu.wired_limit_mb. On the AMD Ryzen AI Max+ platform, Framework publishes "up to 96GB of graphics addressable memory" on the 128GB configuration. NVIDIA does not publish a separate addressable figure for DGX Spark's 128GB of coherent unified memory. Where no figure is published, plan on sticker minus 8-16GB.

Why is my big model slow even though it fits in memory?

Because capacity and bandwidth are separate specifications, and decode speed is set by bandwidth. Every token from a dense model requires reading all of its weights, so the ceiling is bandwidth divided by model size: 42GB of 70B weights against 256 GB/s is about 6 tokens per second as an absolute upper bound, before overhead. A larger memory configuration does not change that division. Mixture-of-experts models are the exception, because only the active parameters are read per token.

Is unified memory better than a graphics card for local AI?

Only above the point where a card runs out. GDDR on a discrete card runs at bandwidths the LPDDR5x platforms here do not approach, so for models that fit on the card it is not close. Unified memory wins for the models that do not fit on a card at all — a 70B, a 120B MoE, or anything larger — where the alternative is not "slower" but "impossible without offloading to disk".

Does Strix Halo really give the GPU 128GB?

No. The system ships with up to 128GB of LPDDR5x, but the graphics-addressable ceiling on Framework's 128GB Ryzen AI Max Desktop is published as 96GB. Size your models against 96GB, not 128GB. On the software side, ROCm 7.14.0 lists gfx1151 as a supported target, so the platform is officially in AMD's compatibility matrix rather than relying on an override.

Sources

• Apple Newsroom — Mac Studio — 36GB to 128GB on M4 Max, 96GB to 512GB on M3 Ultra, "over 600 billion parameters entirely in memory"
• Apple — Mac Studio technical specifications — 410 GB/s and 546 GB/s (M4 Max), 819 GB/s (M3 Ultra)
• MLX — mlx.core.set_wired_limit — the wired limit must stay strictly below total memory; sudo sysctl iogpu.wired_limit_mb; device_info() fields
• NVIDIA DGX Spark — 128 GB LPDDR5x coherent unified system memory, 273 GB/s, models up to 200B for inference
• Framework Desktop (Ryzen AI Max) — 32GB / 64GB / 128GB LPDDR5x-8000, 256-bit memory bus, "up to 96GB of graphics addressable memory"
• AMD ROCm compatibility matrix — gfx1151 (Ryzen AI Max+) listed as a supported target in ROCm 7.14.0