GPU MEMORY
How much VRAM do you need for local AI?
Choose a model, format and conversation length first. Then budget for weights, attention cache and runtime space. A GPU’s advertised capacity alone cannot tell you whether your task will fit.
Editorial update: 24 October 2026. The worked figures use our versioned calculator, not measurements of a particular GPU.
Build the memory budget
Model weights are one part. The attention cache holds information used during generation and usually grows with context. Runtime allocations and other applications need space too. Our estimator adds a runtime allowance and leaves headroom rather than aiming to fill the device exactly.
A worked example: Llama 3.1 8B
This comparison keeps Q4_K_M weights, one conversation, an FP16 KV cache and the Ollama planning model fixed. Only context changes. The table uses the same calculation as our homepage tool.
| Context | Weights | KV cache | GPU target |
|---|---|---|---|
| 4,096 tokens | 4.52 | 0.50 | 6.7 |
| 8,192 tokens | 4.52 | 1.00 | 7.2 |
| 32,768 tokens | 4.52 | 4.00 | 10.6 |
The weights stay constant while the cache increases. That is why a short chat fitting does not prove a long document will fit. See the model’s formats, architecture source and limitations.
What do 8, 12, 16 or 24 GB change?
- 8 GB: start with a smaller quantised model and a short context, then compare the target with memory actually available.
- 12 to 16 GB: more capacity can allow a larger format or longer context. Change one setting at a time to see what uses the space.
- 24 GB and above: more models can become possible, but a model that fits is not automatically fast or useful for your task.
These are ways to approach the calculator, not universal model-size guarantees. Check units: GiB is 1,024 cubed bytes; decimal GB is one billion bytes. Compare like-for-like capacity, including any memory already occupied.
System RAM and unified memory
System RAM can be needed while loading even when inference runs on a dedicated GPU. On an Apple Silicon Mac, applications and the operating system share unified memory, so do not compare its full capacity directly with a dedicated GPU target. The calculator reports a separate unified-memory planning target.
If you are close to the limit
- Close other GPU-heavy applications and check available memory.
- Start with a shorter context or a smaller compatible weight format.
- Load one conversation and inspect actual runtime memory use.
- Test a representative document before buying hardware for it.
CPU offloading, multiple GPUs and different cache formats can change the result; this simple estimate does not model them. For Ollama, the official FAQ explains using ollama ps to inspect whether a loaded model uses CPU, GPU or both.
Compare one change at a time
Use the upgrade comparison, then read the GPU-memory buying guide and loading-failure checklist. Establish the workload before spending on capacity.