hostinglinks.
PRACTICAL TECH, EXPLAINED

Current · sources checked 30 October 2026

Qwen3 8B VRAM and RAM requirements

Run this model with Ollama or LM Studio ↓

Check a model download using its actual file size.

New to local AI? Find your GPU memory and understand these settings.

Qwen3 8B is a dense 8.2 billion parameter model from Alibaba's Qwen team, with 6.95 billion parameters outside the embeddings. The model card lists 36 layers and grouped-query attention with 32 query heads and 8 KV heads. The config.json read from the Hugging Face repository on 30 October 2026 adds the figures the card leaves out: hidden size 4,096, head dimension 128, intermediate size 12,288, vocabulary 151,936, and rope_scaling set to null. The weights are Apache 2.0. That licence, not the benchmark table, is why a product team picks this file over a Llama checkpoint of a similar size.

Thinking is on unless you turn it off

The useful trick is that one checkpoint does both a reasoning mode and a direct mode. Qwen's model card says enable_thinking defaults to true. In that mode the model writes a thinking trace and then an answer. Set it to false and the trace is suppressed, which is the behaviour you want for a short chat, a UI that has to stay responsive, or a batch of simple rewrites. With thinking left on, /think and /no_think in the user text can switch mode from turn to turn. With thinking forced off, those flags do nothing.

The sampling notes are part of running the model, not optional colour. For thinking mode Qwen specifies temperature 0.6, top-p 0.95, top-k 20 and min-p 0, and tells you not to use greedy decoding because it can collapse into repetition. Non-thinking mode is temperature 0.7, top-p 0.8, top-k 20 and min-p 0. Qwen also says a presence penalty between 0 and 2 can reduce repetition, and that a higher penalty can mix languages and dent quality. Recommended output length for most queries is 32,768 tokens, and 38,912 tokens for hard maths and programming items. A thinking trace of that length is a memory event, not just a latency event. The KV cache grows with every token of the trace, not only with the question you typed.

The two modes are different models in the scores

Qwen's technical report, arXiv 2505.09388, evaluates the same 8B weights twice. These are Qwen's numbers. They are not Meta's harness and not DeepSeek's. In thinking mode, Table 17 puts Qwen3 8B at MMLU-Redux 87.5, GPQA Diamond 62.0, IFEval strict prompt 85.0, MATH-500 97.4, AIME 2024 76.0, AIME 2025 67.3, LiveCodeBench v5 57.5 and a Codeforces rating of 1,785. AIME is an average of 64 samples per question, not a single greedy pass.

Turn thinking off and Table 18 is a different product: MMLU-Redux 79.5, GPQA Diamond 39.3, IFEval 83.0, MATH-500 87.4, AIME 2024 29.1, AIME 2025 20.9, LiveCodeBench v5 22.8, Arena-Hard 79.6. The maths collapse is the one to plan for. AIME 2024 falling from 76.0 to 29.1 is what you give up for a short answer. GPQA Diamond at 39.3 in non-thinking mode is also below Qwen2.5 14B Instruct at 45.5 in that same table, so a larger older model can still win a science item when this 8B is not allowed to think. Qwen 2.5 14B remains in the catalogue for that reason. Qwen 2.5 7B is the closer ancestor on size, and Qwen's non-thinking table has this 8B ahead of it on most of the rows they printed.

32,768 native tokens, a 40,960 config cap, and 131,072 only with YaRN

Three context numbers are in circulation, and they are not interchangeable. The model card says 32,768 tokens natively and 131,072 with YaRN. The technical report's architecture table prints 128K for Qwen3 8B. The config file sets max_position_embeddings to 40,960 and does not enable YaRN. Qwen's own note explains the 40,960: it reserves 32,768 tokens for the answer and 8,192 for a typical prompt. Ollama's qwen3:8b tag, read on 30 October 2026, advertises a 40K window, which matches that config cap rather than the YaRN length.

YaRN is a RoPE scaling change you add, either by editing rope_scaling onto the config or by passing it to vLLM, SGLang or llama.cpp. Qwen's example uses a factor of 4.0 and an original context of 32,768 to reach 131,072. The same note says the common frameworks implement static YaRN, so the scale stays on even for short prompts and can hurt those short prompts. Qwen's advice is to leave YaRN off unless the conversation really needs it. This calculator still allows a slider up to 131,072 tokens, because that is the length Qwen says it validated. The slider does not turn YaRN on, and the cache maths do not include any savings or quality loss from scaling. A 131,072 token estimate is a memory ceiling for a configuration you must enable elsewhere.

Downloads, and the GPU each one actually needs

Ollama's tags for this size, same date, are Q4_K_M at 5.2 GB, Q8_0 at 8.9 GB and FP16 at 16 GB. Our weight allowances are 4.97 GB, 8.71 GB and 16.40 GB, from 8.2 billion parameters rather than from those files. They are close enough to plan with, and they are not a promise that a particular GGUF matches the byte.

Estimated memory for Qwen3 8B
PrecisionGPU VRAM targetSystem RAM targetWeight allowance
Q4_K_M7.5 GiB16 GiB4.97 GB
Q8_011.4 GiB24 GiB8.71 GB
FP1619.9 GiB24 GiB16.40 GB

For a chat of 8,192 tokens, read Q4_K_M as GPU 7.5 GiB, of which the cache is only 1.1 GiB, plus a 16 GiB RAM allowance and a 16 GiB shared pool on a Mac. Q8_0 lifts the GPU load to 11.4 GiB and the Mac pool to 24 GiB, and the RAM allowance becomes 24 GiB. FP16 is a 19.9 GiB graphics load and a 32 GiB Mac. A 12 GB card is the comfortable Q4 home at this context. An 8 GB card sits under that Q4 graphics figure once the desktop takes its share.

Move Q4 to the native 32,768 token budget Qwen recommends for answers and the GPU target becomes 11.3 GiB, with 4.5 GiB of cache. That still fits a 16 GB card and is the wrong assumption for an 8 GB one. The YaRN length, 131,072 tokens, pushes the same Q4 weights to a 26.3 GiB GPU target and 18.0 GiB of cache. Long context on this "small" model is a 32 GB-class GPU or a Mac with a 40 GiB unified target. The first-quantisation sweep is below. Remember that the 131,072 row is the planning figure, not a switch that enables YaRN.

Qwen3 8B: context and memory trade-offs
ContextKV cacheGPU targetMac unified target
1,024 tokens0.14 GiB6.4 GiB16 GiB
4,096 tokens0.56 GiB6.9 GiB16 GiB
8,192 tokens1.13 GiB7.5 GiB16 GiB
32,768 tokens4.50 GiB11.3 GiB24 GiB
131,072 tokens18.00 GiB26.3 GiB40 GiB

Where it beats an 8B Llama, and where a 32B distill still matters

Against Llama 3.1 8B Instruct in Qwen's non-thinking table, this model is ahead by a wide margin on MMLU-Redux, MATH-500, AIME and Arena-Hard, under the Apache licence rather than the Llama community licence. That is the default local 8B to try in 2026 if the machine is a 12 GB GPU and the task is ordinary chat plus the option of a thinking pass. It is not a vision model. It is not the 32B or the 235B Qwen3, and the technical report is explicit that the smaller dense models, this one included in that lightweight group, take their capability from strong-to-weak distillation off larger Qwen3 teachers as well as from their own training.

The reasoning comparison inside the same paper is DeepSeek R1 Distill Qwen 32B. In Table 17, thinking Qwen3 8B is slightly behind that 32B distill on MMLU-Redux (87.5 against 88.2) and tied on GPQA Diamond (62.0 against 62.1), and ahead on AIME 2024 (76.0 against 72.6), AIME 2025 (67.3 against 49.6), LiveCodeBench v5 (57.5 against 54.5) and IFEval (85.0 against 72.5). Those rows are why an 8B with a thinking mode can replace a 32B distill for many of Qwen's published tasks, at a fraction of the memory. They are not a reason to ignore the 32B when you need the DeepSeek paper's own LiveCodeBench protocol, which is a different number and is discussed on that model's page. Qwen3 32B remains the dense step up when the 8B thinking trace is not enough and you have the memory.

Sources for the Qwen3 8B figures

Run Qwen3 8B locally

These steps use Q4_K_M weights. Compare the Q4_K_M row above with your available memory before downloading. The command does not choose Q8 or 16-bit weights, even if you explored those in the calculator.

View the exact Ollama download, file size and licence. Download tags checked 16 September 2026; publishers can update their files. File size is not the same as total running memory.

Option 1: Ollama

  1. Install and start Ollama from its official website. Check its supported GPUs.
  2. Open Terminal on Mac or Linux, or PowerShell on Windows. Run the command below. On first use, Ollama downloads the model before opening a chat.
ollama run qwen3:8b-q4_K_M

Inside the Ollama chat, enter this setting before your first prompt to use the same 8,192-token context as the table above:

/set parameter num_ctx 8192

Then try a short prompt such as “Explain how a solar panel works in three sentences.” In a second terminal, run ollama ps to inspect whether the model is on the GPU or partly on the CPU. CPU offload changes performance and falls outside this calculator's full-GPU assumptions. See Ollama's context and GPU-loading instructions.

Option 2: LM Studio

  1. Install LM Studio and open its model discovery screen.
  2. Search for Qwen3 8B. Match the family, size and variant exactly. Check the publisher and model card; community conversions are not necessarily published by the original model creator.
  3. Choose a supported GGUF download labelled Q4_K_M to compare with our balanced estimate. Check the actual file size before downloading.
  4. Load the model into a chat. Set the context length to 8,192 tokens and check the GPU offload settings. Try a short prompt before adding a long conversation or document.

LM Studio's official model download guide. Search results and available formats can change; we have verified the Ollama tags, not every community LM Studio conversion.

If the model will not load

Check your runtime version and GPU support, then close other memory-heavy apps. Try a shorter context or a smaller model and recalculate. Confirm that the downloaded format matches the estimate. Extra conversations, vision processing and runtime-specific allocations can increase memory use.

Find a smaller model for my computer · Check my memory and settings

Compare hardware and change context in the VRAM calculator

Compare other local LLM models · Privacy and cookies