Previous generation · sources checked 30 October 2026
Llama 3.1 8B VRAM and RAM requirements
Run this model with Ollama or LM Studio ↓
Check a model download using its actual file size.
New to local AI? Find your GPU memory and understand these settings.
If a desktop is asked to run Llama, the file that arrives is usually this one. On 30 October 2026, Ollama's llama3.1:latest tag and llama3.1:8b-instruct-q4_K_M were the same 4.9 GB download. Meta released the checkpoint on 23 July 2024 as the smallest of six text models: a pretrained and an instruction-tuned pair at 8B, 70B and 405B.
What Meta actually shipped
The Llama 3.1 model card describes an 8 billion parameter model that takes multilingual text and returns multilingual text and code. Context length is 128,000 tokens. Attention is grouped-query. Pretraining used more than 15 trillion tokens, and the knowledge cutoff printed on the card is December 2023. The eight languages Meta lists as supported are English, German, French, Italian, Portuguese, Hindi, Spanish and Thai. The card also says the training mix contained further languages, and that quality outside the eight is the developer's problem to measure.
This page is the text instruct model. It does not see images. Llama 3.2's vision checkpoints are a later, separate release and they are not estimated here. The Llama 3.2 1B and Llama 3.2 3B text models are in the catalogue, share the 128,000 token window, and are the ones to open when 8B is already too large. They are not a compressed copy of these weights.
The Q4_K_M instruct file on Ollama records 32 layers, 32 query heads, 8 key-value heads, an embedding width of 4,096, a feed-forward width of 14,336, a RoPE dimension of 128 and a vocabulary of 128,256. Those layer and KV-head counts are the ones this calculator uses, with a head dimension of 128. The same GGUF header stores a context length of 8,192. That header field is not Meta's limit. The model card says 128,000 tokens, and Ollama's tag page advertises 128K for the file.
The licence is a contract, not an open-source grant
These weights are not Apache 2.0, MIT, or any other OSI-approved licence. Meta distributes them under the Llama 3.1 Community License. Ordinary commercial use, modification and redistribution are allowed, with conditions that are easy to miss. If you distribute the weights, or a product that contains them, you have to include the agreement and display "Built with Llama". If you train another model on Llama outputs and then distribute that model, its name has to begin with "Llama". An acceptable-use policy sits beside the licence and bans a listed set of harmful uses.
There is a scale clause. If, on the Llama 3.1 release date, the products or services made available by you or your affiliates had more than 700 million monthly active users in the preceding calendar month, you must request a licence from Meta, and Meta may refuse. A home server and a normal company are nowhere near that line. A product at the scale of a global consumer platform is. Read the licence before you ship a derivative, rather than relying on this summary. The text lives with the model card on GitHub.
Scores Meta published, including one that went backwards
The figures below are copied from Meta's instruction-tuned table for Llama 3.1 8B. HostingLinks has not rerun them. They also are not the harness Qwen or DeepSeek use, so they cannot be subtracted from those labs' numbers as if they were the same exam.
- MMLU, 5-shot macro accuracy: 69.4, against 68.5 for Llama 3 8B Instruct.
- MMLU with 0-shot chain-of-thought: 73.0, against 65.3.
- MMLU-Pro, 5-shot chain-of-thought: 48.3.
- IFEval: 80.4. HumanEval 0-shot pass@1: 72.6, against 60.4. MBPP EvalPlus base pass@1: 72.8.
- GSM8K, 8-shot chain-of-thought: 84.5. MATH, 0-shot: 51.9, against 29.1 for Llama 3 8B Instruct.
- Multilingual MGSM: 68.9. API-Bank: 82.6. BFCL: 76.1.
GPQA, reported as 0-shot exact match, is the result that should stop a casual upgrade story. Llama 3.1 8B Instruct scores 30.4. Llama 3 8B Instruct scored 34.6 on the same row of Meta's table. Post-training improved maths and code and made this particular science set worse. A four-choice exam at 30.4 is not a basis for trusting the model as a tutor. MATH at 51.9 is the other ceiling: fine for a first local model, weak beside later 8B-class models that were trained to spend tokens on reasoning.
Q4, Q8 and FP16, and the context that changes the GPU
Ollama's tag list on 30 October 2026 shows three sizes worth planning around for this instruct checkpoint: Q4_K_M at 4.9 GB, Q8_0 at 8.5 GB and FP16 at 16 GB. The table underneath does not copy those files. It estimates the weight store from 8 billion parameters, then adds an FP16 KV cache for one conversation, at least 1 GiB of runtime allowance and 10 percent GPU headroom. The method is set out on the methodology page. The Q4 allowance comes out at 4.85 GB, close to the 4.9 GB package. Q8_0 and FP16 land on 8.50 GB and 16.00 GB, the same round figures Ollama publishes.
| Precision | GPU VRAM target | System RAM target | Weight allowance |
|---|---|---|---|
| Q4_K_M | 7.2 GiB | 16 GiB | 4.85 GB |
| Q8_0 | 11.0 GiB | 16 GiB | 8.50 GB |
| FP16 | 19.3 GiB | 24 GiB | 16.00 GB |
At 8,192 tokens, the length used for that table, Q4_K_M is a 7.2 GiB GPU target, 16 GiB of system RAM and a 16 GiB Mac unified target. The cache itself is only 1.0 GiB, because eight KV heads are stored rather than all 32 query heads. Q8_0 moves the GPU target to 11.0 GiB and the unified target to 24 GiB. FP16 moves them to 19.3 GiB and 32 GiB, with a system RAM target of 24 GiB.
An 8 GB graphics card is the wrong place to feel safe. The Q4 planning target is already 7.2 GiB before the display and the runtime take a share. A 12 GB card covers the Q4 estimate at this context, and it is tight for Q8. FP16 is a 24 GB decision. System RAM is a loading allowance in this calculator, not a second copy of the GPU budget, which is why Q4 and Q8 both show 16 GiB. A unified-memory Mac does not get that split. Weights, cache and the operating system share one pool, which is why the Mac column is a separate number.
The 4.9 GB download says nothing about a long chat. Stretch Q4 to 32,768 tokens and the cache becomes 4.0 GiB, with a GPU target of 10.6 GiB and a unified target of 24 GiB. At the full 131,072 tokens the cache is 16.0 GiB and the GPU target is 23.9 GiB. A 128K conversation of this small model is a 24 GB-class problem. The same pattern, for the first quantisation only, is below.
| Context | KV cache | GPU target | Mac unified target |
|---|---|---|---|
| 1,024 tokens | 0.13 GiB | 6.3 GiB | 16 GiB |
| 4,096 tokens | 0.50 GiB | 6.7 GiB | 16 GiB |
| 8,192 tokens | 1.00 GiB | 7.2 GiB | 16 GiB |
| 32,768 tokens | 4.00 GiB | 10.6 GiB | 24 GiB |
| 131,072 tokens | 16.00 GiB | 23.9 GiB | 32 GiB |
Our separate benchmark notes measure Llama 2 7B at Q4_0 on a handful of NVIDIA cards. Those tokens-per-second figures are a different model, a different quantisation and a different program. They are not a speed for Llama 3.1 8B.
Use it as the first model, not the last
Llama 3.1 8B is a sensible first checkpoint when you are proving that Ollama or LM Studio can see the GPU, and for short rewriting, extraction and simple tool-calling experiments. API-Bank at 82.6 on Meta's card is the reason to try the last of those. Keep the context at the shortest length that holds the task. Do not treat it as a private stand-in for a current hosted assistant when the question needs maths beyond this card, or any fact after December 2023. The model will answer anyway. The cutoff will not announce itself.
The neighbour that matters in this catalogue is Qwen3 8B. The Q4 download is a similar size, and the licence is Apache 2.0. In Qwen's technical report, non-thinking Qwen3 8B is well ahead of Llama 3.1 8B Instruct on MMLU-Redux (79.5 against 61.7), MATH-500 (87.4 against 54.8), AIME 2024 (29.1 against 6.3) and Arena-Hard (79.6 against 30.1). That is Qwen's harness, published by Qwen, and it is not a HostingLinks bake-off. It is still the comparison to run on your own prompts before you standardise on Llama. Llama 3.3 70B is the quality step inside the Llama text family, and it is a different computer: the Q4_K_M file is 43 GB.
Ollama also publishes Q5 and Q6 tags between 4.9 GB and 8.5 GB. This table does not estimate them. If the Q4 target is close to the memory you have free, shorten the context before you hunt a more aggressive file the calculator cannot see.
Sources for this page
- Meta, Llama 3.1 model card: size, 128K context, cutoff, languages and the instruction-tuned benchmark table
- Hugging Face Llama 3.1 release note, published 23 July 2024
- Llama 3.1 Community License, including the 700 million monthly-active-user clause
- Ollama llama3.1 tags: 4.9 GB Q4_K_M, 8.5 GB Q8_0 and 16 GB FP16, read 30 October 2026
- Qwen3 technical report, Table 18, for the non-thinking comparison with Llama 3.1 8B Instruct
Run Llama 3.1 8B locally
These steps use Q4_K_M weights. Compare the Q4_K_M row above with your available memory before downloading. The command does not choose Q8 or 16-bit weights, even if you explored those in the calculator.
View the exact Ollama download, file size and licence. Download tags checked 16 September 2026; publishers can update their files. File size is not the same as total running memory.
Option 1: Ollama
- Install and start Ollama from its official website. Check its supported GPUs.
- Open Terminal on Mac or Linux, or PowerShell on Windows. Run the command below. On first use, Ollama downloads the model before opening a chat.
ollama run llama3.1:8b-instruct-q4_K_MInside the Ollama chat, enter this setting before your first prompt to use the same 8,192-token context as the table above:
/set parameter num_ctx 8192Then try a short prompt such as “Explain how a solar panel works in three sentences.” In a second terminal, run ollama ps to inspect whether the model is on the GPU or partly on the CPU. CPU offload changes performance and falls outside this calculator's full-GPU assumptions. See Ollama's context and GPU-loading instructions.
Option 2: LM Studio
- Install LM Studio and open its model discovery screen.
- Search for Llama 3.1 8B. Match the family, size and variant exactly. Check the publisher and model card; community conversions are not necessarily published by the original model creator.
- Choose a supported GGUF download labelled Q4_K_M to compare with our balanced estimate. Check the actual file size before downloading.
- Load the model into a chat. Set the context length to 8,192 tokens and check the GPU offload settings. Try a short prompt before adding a long conversation or document.
LM Studio's official model download guide. Search results and available formats can change; we have verified the Ollama tags, not every community LM Studio conversion.
If the model will not load
Check your runtime version and GPU support, then close other memory-heavy apps. Try a shorter context or a smaller model and recalculate. Confirm that the downloaded format matches the estimate. Extra conversations, vision processing and runtime-specific allocations can increase memory use.
Find a smaller model for my computer · Check my memory and settings