hostinglinks.
PRACTICAL TECH, EXPLAINED

GPU MEMORY

How much VRAM do you need for local AI?

Choose a model, format and conversation length first. Then budget for weights, attention cache and runtime space. A GPU’s advertised capacity alone cannot tell you whether your task will fit.

Editorial update: 24 October 2026. The worked figures use our versioned calculator, not measurements of a particular GPU.

Build the memory budget

Model weights are one part. The attention cache holds information used during generation and usually grows with context. Runtime allocations and other applications need space too. Our estimator adds a runtime allowance and leaves headroom rather than aiming to fill the device exactly.

A worked example: Llama 3.1 8B

This comparison keeps Q4_K_M weights, one conversation, an FP16 KV cache and the Ollama planning model fixed. Only context changes. The table uses the same calculation as our homepage tool.

Planning estimates in GiB, with all weights loaded on one GPU
ContextWeightsKV cacheGPU target
4,096 tokens4.520.506.7
8,192 tokens4.521.007.2
32,768 tokens4.524.0010.6

The weights stay constant while the cache increases. That is why a short chat fitting does not prove a long document will fit. See the model’s formats, architecture source and limitations.

What do 8, 12, 16 or 24 GB change?

These are ways to approach the calculator, not universal model-size guarantees. Check units: GiB is 1,024 cubed bytes; decimal GB is one billion bytes. Compare like-for-like capacity, including any memory already occupied.

System RAM and unified memory

System RAM can be needed while loading even when inference runs on a dedicated GPU. On an Apple Silicon Mac, applications and the operating system share unified memory, so do not compare its full capacity directly with a dedicated GPU target. The calculator reports a separate unified-memory planning target.

If you are close to the limit

  1. Close other GPU-heavy applications and check available memory.
  2. Start with a shorter context or a smaller compatible weight format.
  3. Load one conversation and inspect actual runtime memory use.
  4. Test a representative document before buying hardware for it.

CPU offloading, multiple GPUs and different cache formats can change the result; this simple estimate does not model them. For Ollama, the official FAQ explains using ollama ps to inspect whether a loaded model uses CPU, GPU or both.

Compare one change at a time

Use the upgrade comparison, then read the GPU-memory buying guide and loading-failure checklist. Establish the workload before spending on capacity.