How to install Ollama and run your first local model

Ollama is an open-source tool that lets you download, manage, and run large language models on your own machine with a single terminal command. It handles all the complexity of GGUF quantisation, llama.cpp compilation, and model management so you don't have to.

Once installed, running a model is as simple as `ollama run llama3.3`. Models download automatically on first run and are cached locally for offline use. Ollama also exposes a local HTTP server at port 11434 that is compatible with the OpenAI API, so you can point any OpenAI SDK client at it.

On macOS, download the Ollama app from ollama.com and drag it to your Applications folder. The menu-bar icon confirms it is running. Alternatively install via Homebrew: `brew install ollama`.

On Linux, run the official one-liner: `curl -fsSL https://ollama.com/install.sh | sh`. This installs the `ollama` binary and registers a systemd service that starts automatically on boot.

On Windows, download the installer from ollama.com. It installs as a background service and adds `ollama` to your PATH so you can use it from PowerShell or Command Prompt.

Open a terminal and run `ollama pull phi4-mini` to download Microsoft's 3.8B model — it is under 2.5 GB and runs comfortably on 4 GB of RAM. For something more capable on 8 GB RAM, try `ollama pull deepseek-r1:7b`.

Once downloaded, start an interactive chat session with `ollama run phi4-mini`. Type your message and press Enter. Type `/bye` to exit the session. To run a quick one-shot query without entering interactive mode, use: `ollama run phi4-mini 'Explain transformer attention in one paragraph'`.

Ollama runs a local server at `http://localhost:11434`. You can call it from any HTTP client: `curl http://localhost:11434/api/generate -d '{"model": "phi4-mini", "prompt": "Why is the sky blue?", "stream": false}'`.

Because the server exposes an OpenAI-compatible `/v1/chat/completions` endpoint, you can reuse any existing OpenAI SDK code by changing the base URL: `client = OpenAI(base_url='http://localhost:11434/v1', api_key='ollama')`. This makes it trivial to switch between local and cloud models during development.

Use `ollama list` to see all downloaded models with their sizes, `ollama rm <model>` to delete one, and `ollama ps` to see which models are currently loaded in memory. Ollama keeps a model warm in GPU/memory for a few minutes after use to speed up subsequent requests.

To see available models, visit the Ollama model library at ollama.com/library. It lists hundreds of open-source models with their sizes, quantisation levels, and recommended hardware.