Fix local AI out-of-memory errors
Separate VRAM pressure, system RAM limits and runtime problems, then change one setting at a time.
Published 16 September 2026 · HostingLinks · Editorial approach
Identify when the failure happens
Record the exact error, model tag, format and runtime version. Does the model fail while loading, after a long prompt, or only when another conversation starts? Those cases point to different causes. A download error or unsupported model format is not automatically a memory shortage. Keep the original error message so you can distinguish a fix from a different failure.
Check the memory that is actually available
Open your system’s memory tools using our Windows, Mac and Linux guide. Look at both dedicated GPU memory and system RAM on a PC. Save your work, then close applications you do not need. Enter the available capacity into the finder rather than assuming the full advertised amount is free.
On a Mac, CPU and GPU share memory. Do not add the same unified capacity twice. Memory pressure can come from other applications even if the model weights look small. On a PC, a small model download can still require extra GPU cache and temporary loading space in RAM.
Reduce one demand at a time
- Try a short prompt and one active conversation. This establishes whether the model can load at all.
- Lower context length, for example from 32,768 to 4,096 tokens. Make the same change in the calculator and the runtime.
- Check that you downloaded the intended format. Choosing Q4_K_M in our calculator does not convert an FP16 file already on your computer.
- Try a smaller model. If that works, keep the successful configuration as a baseline before adding complexity again.
For our Llama 3.1 8B Q4_K_M example, reducing context from 32,768 to 4,096 lowers the calculated GPU target from 10.6 to 6.7 GiB. The weights stay unchanged; the cache becomes smaller. That is why lowering context may help a borderline workload without changing the model file.
Check loading and runtime support
In Ollama, ollama ps helps identify where a loaded model runs. A CPU or mixed result can explain unexpected slowness even when the request succeeds. The calculator assumes full GPU loading; it does not estimate CPU-offload speed. Check the official hardware support list before assuming your GPU is in use.
If a small supported model still fails, inspect the runtime logs. Ollama’s troubleshooting documentation lists log locations and GPU-discovery checks. Match instructions to your operating system and runtime version. Avoid changing several driver and runtime settings at once because that makes the cause harder to identify.
Know when a larger machine would help
Use the upgrade comparison only after checking format and runtime compatibility. Extra memory helps a capacity shortfall, but it will not repair an incompatible backend or corrupt download. When seeking support, include settings and the relevant error, and remove private prompts, access tokens and personal paths from logs. Never post an entire private conversation just to demonstrate a loading error.