Run your first local model with Ollama
A small-model walkthrough: check memory, choose an exact download, start a local chat and confirm GPU loading.
Published 16 September 2026 · HostingLinks · Editorial approach
Start with a small, repeatable experiment
Your first goal is to get one short conversation working and understand where the model runs. Choosing the largest model that might fit makes troubleshooting harder. Start with the DeepSeek R1 Distill Qwen 1.5B Q4_K_M example below, then compare larger models after you have a working baseline. It is a learning example, not a claim that this model gives the best answers.
Before installing anything, check your dedicated GPU memory and system RAM. A Mac uses unified memory instead. Find the model in our calculator, select Ollama and balanced weights, and begin with 4,096 context tokens. The estimate assumes one conversation and GPU loading; it does not certify your GPU or driver.
Install and choose the exact file
Download Ollama from its official site and follow the installation instructions for your operating system. Check Ollama’s supported hardware. Open the application before using the command below. On a Linux installation without an active service, follow the official instructions for starting the server.
The exact model listing identifies Q4_K_M weights and a download of roughly 1.1 GB when checked on 16 September 2026. Allow disk space beyond the download. Download size is not the memory needed to run a model, and publishers can update files.
Open a local conversation
Open PowerShell on Windows, or Terminal on Mac or Linux. Enter:
ollama run deepseek-r1:1.5b-qwen-distill-q4_K_MThe first run downloads the weights. Inside the resulting chat, enter:
/set parameter num_ctx 4096Try: “Explain the difference between RAM and storage in three sentences.” Check whether the answer is understandable and correct. A model can sound confident while making mistakes. This distilled reasoning model may also produce reasoning text before its answer.
Confirm what happened
In a second terminal, run ollama ps. Its processor column shows GPU, CPU or mixed loading. CPU offload may let a model run beyond our full-GPU estimate, but speed can differ substantially. Use /bye in the chat to leave. See the Ollama FAQ for context and loading controls, and the official quickstart for the current installation flow.
Decide what to try next
Keep your prompt and context unchanged when trying another model. Write down the exact tag, weight format and whether it used the GPU. Compare the usefulness of its answer as well as speed. If loading fails, use our memory troubleshooting guide before buying hardware. This walkthrough was checked against documentation; it is not a claim that HostingLinks tested Ollama on every supported computer.