MEASURED SPEED REFERENCES
Local LLM GPU benchmarks
Memory tells you whether a model might fit. Benchmarks show how fast one specific setup ran a specific workload.
Community measurements, source checked
These published results were checked against the llama.cpp community scoreboard on 16 September 2026. HostingLinks has not independently rerun them. They are historical reference measurements, not guaranteed speeds or a controlled hardware ranking.
Llama 2 7B, Q4_0, CUDA
All rows use llama-bench with GPU offload and Flash Attention disabled. Prompt processing uses 512 tokens (pp512); generation uses 128 tokens (tg128). These are separate tests, not a combined 512-token chat followed by a reply. Values are tokens per second, with the source-reported variation.
| GPU | VRAM (GB) | Prompt tokens/s | Generated tokens/s | llama.cpp commit |
|---|---|---|---|---|
| RTX 3060 | 12 | 2137.50 ± 10.12 | 75.57 ± 0.07 | baa9255 |
| RTX 3090 | 24 | 5174.69 ± 21.83 | 158.16 ± 0.21 | c76b420 |
| RTX 4090 | 24 | 11992.70 ± 107.99 | 186.21 ± 0.13 | 2241453 |
| RTX 5060 Ti | 16 | 3737.25 ± 6.79 | 90.94 ± 0.02 | 89d1029 |
Original measurements and full test discussion, llama.cpp CUDA scoreboard (no FA).
Read these results correctly
Prompt processing measures how quickly input is read; generation measures how quickly new tokens are produced. Neither number includes every source of user-visible waiting time, such as loading a model.
The tested Llama 2 model and Q4_0 format differ from our catalogue’s newer models and Q4_K_M estimates. Do not transfer these speeds to Qwen, DeepSeek, Gemma or gpt-oss, or to Ollama, LM Studio or vLLM. Software commits differ between rows; the summary does not provide a uniform operating system, driver, CPU or system RAM configuration. Longer conversations, concurrent users and CPU offload can change performance.
What makes a useful benchmark?
Record the exact model file and quantization, GPU and VRAM, CPU and RAM, operating system, driver, runtime version, GPU offload, cache format, prompt length, generated tokens, concurrency and repeated-run variation. Keep prompt and generation speeds separate. Link the raw output so others can inspect the result.
Our first snapshot covers four NVIDIA GPUs. We will only add further hardware or model results when their source and workload can be checked; an absent result does not mean a device is unsupported.