Local AI Power Consumption: Cost Per Hour, Calculated

Electricity cost = (average watts x hours / 1000) x your rate per kWh. Plug your GPU's vendor-rated board power in as the watts figure and you get a defensible ceiling in under a minute: a 200 W card driven four hours a day for a month lands near 24 kWh, which is under $4 at the US residential average. The number people actually get wrong is not the inference figure — it is the idle hours.

This page gives you the arithmetic rather than a table of numbers from somebody else's machine. Every figure below is either a vendor-published rating (labelled and linked), or something you can derive yourself from two inputs you already have: a wattage and an electricity rate.

How do I calculate what local AI costs in electricity?

Two lines of arithmetic. There is no third line.

kWh  = (average watts x hours) / 1000
cost = kWh x your rate per kWh

Worked example — a desktop averaging 250 W at the wall, four hours a day, thirty days, at $0.16/kWh:

hours  = 4 x 30            = 120
kWh    = (250 x 120) / 1000 = 30
cost   = 30 x 0.16          = $4.80 per month

That is the whole model. Everything else on this page is about picking a defensible number for "average watts" and knowing when the answer stops being trustworthy.

Two inputs you need before you start:

• Your actual rate. Use the number on your own bill, not a national average. The US Energy Information Administration publishes current residential averages by state if you want a sanity check — see the EIA electricity data browser. European rates commonly run two to three times US ones, which is enough to flip conclusions.
• A duty cycle. "Watts during inference" is meaningless without the fraction of the day you are actually generating tokens. Most people massively over-estimate theirs.

What wattage should I plug into the formula?

Start with the vendor's rated board power for your GPU. It is the one number in this whole exercise that somebody with a lab has already published.

Board power (NVIDIA calls it TGP, AMD and Intel call it TBP) is the card's rated power envelope, published on the vendor's own spec page. It is an upper bound on what the card draws, not a measurement of what yours draws during inference — a card that is memory-bandwidth-bound, which is exactly what token generation is, frequently sits well below its rating.

Board-power figures are the vendors' own published ratings — check the current spec page for your exact card (NVIDIA GeForce specifications, AMD Radeon, Intel Arc), because SKUs and factory-overclocked variants differ. The right-hand column is not a measurement: it is the board rating plus 90-140 W, which is the band a typical desktop CPU, RAM, motherboard, drives and fans occupy under load, divided by a PSU efficiency of roughly 0.9. Substitute your own numbers if you know them.

The reason we give a band rather than a point estimate is that it is the honest shape of the answer. Anyone quoting you "the RTX 4090 uses 432 W" is quoting one machine, at one ambient temperature, with one PSU, running one model.

Apple Silicon and other integrated systems

Unified-memory Macs sit in a different regime, and the useful comparison is structural rather than numeric: a Mac mini's entire chassis is rated for less continuous power than an RTX 4090's board rating alone. Apple publishes per-model power-consumption tables (idle and maximum) in its support documentation — look up your exact model rather than trusting a round number from a blog. The Mac local AI setup guide covers the software side.

Why is the whole-system number higher than the GPU rating?

Because the socket does not know what a GPU is. Four things sit between your card's rating and your meter:

1. PSU conversion loss. An 80+ Gold unit is roughly 88-90% efficient near half load, worse at very low load. Wall draw = component draw / efficiency, so a 400 W system pulls closer to 445 W from the socket.
2. The rest of the machine. CPU during token decoding, RAM, chipset, NVMe, fans. This is the 90-140 W band in the table above and it does not scale down when you pick a smaller model.
3. Prompt processing versus generation. These are different workloads. Ingesting a long document is compute-heavy and pushes the card toward its rating; generating tokens one at a time is bandwidth-bound and often does not.
4. Cooling, if you are being thorough. Heat dumped into a small room in summer becomes air-conditioning load. It is a real second-order cost and nobody includes it.

Does idle power matter more than inference power?

For most people, yes — and the arithmetic proves it rather than asserting it.

Take a desktop that idles at 80 W and averages 300 W while generating, used two hours a day:

inference: 300 W x 2 h  =  600 Wh/day
idle:       80 W x 22 h = 1760 Wh/day

Idle is nearly three times the cost of the work you actually wanted. That ratio, not any specific wattage, is the finding. It also tells you exactly which lever matters: sleeping the machine when you are not using it beats every quantization and power-limit trick combined, because it attacks the 22-hour term rather than the 2-hour one.

Run the same arithmetic with your own idle figure before you spend a weekend tuning. If your box genuinely runs a workload 20 hours a day, the conclusion inverts and inference power is where to look.

Is local cheaper than cloud APIs on electricity alone?

You can answer this exactly for your own setup with two more lines of arithmetic — and the answer is not the same for a 3B model and a 70B one.

Energy per generated token, then dollars per million tokens:

joules per token   = average watts / tokens per second
kWh per M tokens   = (joules per token x 1,000,000) / 3,600,000
$ per M tokens     = kWh per M tokens x your rate

(The 3,600,000 is just the number of joules in a kilowatt-hour.)

Worked example — 300 W at 40 tok/s, at $0.16/kWh:

joules/token = 300 / 40                    = 7.5
kWh per M    = (7.5 x 1e6) / 3.6e6         = 2.08
$ per M      = 2.08 x 0.16                 = $0.33

Thirty-three cents per million output tokens, electricity only. Now do it again for a big model on the same box: at 300 W and 8 tok/s the joules-per-token figure is five times higher and you land near $1.67 per million. Model size changes your energy cost per token far more than hardware choice does, because the watts term barely moves while the tokens-per-second term collapses.

To finish the comparison, look up the current published price per million tokens on the provider's own pricing page — OpenAI, Anthropic, DeepSeek — and compare against the number you just derived. We deliberately do not reprint those prices here: API pricing moves several times a year and a stale table is worse than no table.

Three things this comparison leaves out on purpose, all of which favour the cloud: hardware amortisation, your time, and the fact that a hosted provider runs at a utilisation you will never reach. The full version, with hardware cost amortised, is in our local AI vs ChatGPT cost breakdown. If your honest answer is "some of both," the hybrid local + cloud pattern is the routing logic.

How do I measure my own numbers instead of estimating?

If you want real figures for your machine rather than a bound, there are three tiers and they answer different questions.

Two things that will wreck a measurement if you skip them: take a sustained average over several minutes, not a point reading, because draw oscillates hard between prompt processing and generation; and measure the machine headless or note the monitor separately, since a large display is a meaningful fraction of a small system's total.

Whatever you record, record the model, the quantization and the context length alongside it. A number without those three is not reproducible, including by you next month.

What actually cuts the number?

In descending order of effect, with the mechanism stated so you can judge it yourself:

1. Sleep or wake-on-LAN the machine. Attacks the idle term, which the arithmetic above shows is usually the large one. Nothing else comes close.
2. Right-size the model. From the cost-per-token derivation: dropping from a model that generates at 8 tok/s to one that generates at 40 tok/s cuts energy per token roughly five-fold, because watts stay put while throughput multiplies. Route easy work to small models — best Ollama models maps the sizes.
3. Power-limit the GPU. sudo nvidia-smi -pl 250 caps a card below its rated board power. Throughput falls too, so this only wins if the throughput loss is smaller than the power saving — measure both with the commands above rather than assuming a ratio.
4. Unload idle models. Ollama keeps a model resident for a few minutes after the last request by default; setting OLLAMA_KEEP_ALIVE=0 frees VRAM immediately and lets the card drop to its idle state. See the Ollama documentation for the current default.
5. Quantize lower where quality allows. Smaller weights mean less data streamed per token, which is the bandwidth-bound path — our quantization guide covers what you give up.
6. Right-size the PSU. A 1000 W unit running a 200 W load sits in its low-efficiency zone. Worth a few percent, no more; do it when you are buying anyway, not as a retrofit.

If your machine has no discrete GPU at all, the power question mostly evaporates and the constraint becomes speed instead — that trade is covered in running LLMs on CPU only.

Where this estimate breaks down

Three honest limits, so you know when to stop trusting the arithmetic.

• Board power is a ceiling, not a mean. Token generation is memory-bound and frequently runs below the rating. Using the rating gives you a safe upper bound on cost, which is the useful direction to be wrong in, but it is not a prediction.
• The rest-of-system band is a convention, not a measurement. We used 90-140 W because that is where typical desktop platforms sit. A Threadripper or a dual-socket server is nowhere near it, and neither is a mini PC.
• Laptops throttle, so the average moves. Sustained inference pins the package, clocks drop after a few minutes, and both your watts and your tokens-per-second change together. Short samples will mislead you in both directions.

None of this argues against running local AI. It argues for doing the arithmetic on your own machine rather than adopting somebody else's number. For hardware selection with the power-throughput trade in mind, start at the budget local AI machine guide; to check a model fits before you worry about its watts, use the VRAM calculator.

The short version

A desktop pulling 250 W for four hours a day costs single-digit dollars a month at typical US rates. The "AI is destroying the grid" story is about hyperscale datacenters, not your office. But the two things worth internalising are smaller and more useful than any headline wattage: idle hours usually dominate your bill, and energy per token is set by model size, not by which card you bought.