Llama 3.1 405B VRAM and RAM requirements
A very large model requiring specialist infrastructure.
This Meta model has approximately 405 billion parameters. The estimates below use 8,192 context tokens, one conversation, FP16 KV cache and all model weights on the GPU.
| Precision | GPU VRAM target | System RAM target | Estimated weight size |
|---|---|---|---|
| Q4_K_M | 283.9 GiB | 240 GiB | 245.53 GB |
| Q8_0 | 494.2 GiB | 416 GiB | 430.31 GB |
| FP16 | 926.4 GiB | 768 GiB | 810.00 GB |
How much context can I use?
This calculator supports up to 131,072 tokens for this model. Longer prompts and generated responses share that budget. KV cache grows with context length; additional conversations need additional cache memory.
Can I run Llama 3.1 405B in Ollama, LM Studio or vLLM?
Q4_K_M and Q8_0 estimates represent GGUF-style quantizations commonly used with Ollama and LM Studio. The calculator supports 16-bit weight estimates for vLLM. Check the runtime version, model format and hardware support before downloading. A memory fit alone does not guarantee compatibility or speed.
What do these estimates include?
Weights are estimated from parameter count and bits per weight, not measured files. We add FP16 attention cache, at least 1 GiB runtime allowance and 10% GPU headroom. System RAM includes an 8 GiB loading allowance. CPU offload, image processing and multi-GPU setups are not modelled.