Skip to main content

Ollama Setup — Local AI

Ollama lets you run AI models entirely on your own machine — no internet connection or API key required after initial setup. This is ideal for offline environments, air-gapped networks, or when you prefer to keep all data local.

Pro feature

Selecting Ollama as the active AI provider requires a Pro licence. The Ollama settings panel is visible on the Free tier but saving the configuration is gated.

How it works​

Ollama runs as a local server on your Windows machine (port 11434 by default). Conio sends prompts to it over the local network — from inside Docker to your Windows host. Because of this Docker-to-host path, two extra configuration steps are required beyond a standard Ollama install.

Step 1 — Install Ollama​

Download and run the Windows installer from ollama.com/download/windows.

After installation, Ollama starts automatically and places an icon in the system tray. You do not need to run any terminal commands to start it — it runs in the background.

Step 2 — Pull a model​

Open PowerShell and pull the model you want to use. Recommended starting points:

# Fast, low memory (4 GB VRAM / 8 GB RAM)
ollama pull llama3.2

# Balanced accuracy / speed
ollama pull mistral

# Higher accuracy, slower (8 GB VRAM / 16 GB RAM)
ollama pull llama3.1:8b

The model downloads once and is cached locally. Subsequent starts are instant.

To see all models you have pulled:

ollama list
Choosing a model

For construction bid estimation tasks (description refinement, keyword extraction, rate matching), mistral and llama3.2 produce reliable results on most hardware. Larger models improve quality on complex or ambiguous descriptions but require more RAM and take longer per request.

Step 3 — Allow Ollama to accept Docker connections​

This is the most important step and the most common source of connection errors.

By default, Ollama binds only to 127.0.0.1 (the Windows loopback interface). Conio runs inside a Docker container, which connects to your Windows host through a virtual network adapter — a different interface. Ollama will refuse those connections unless you tell it to listen on all interfaces.

Set the OLLAMA_HOST environment variable​

  1. Press Win + S and search for "Edit the system environment variables". Open it.
  2. Click Environment Variables….
  3. Under System variables, click New.
  4. Set:
    • Variable name: OLLAMA_HOST
    • Variable value: 0.0.0.0
  5. Click OK on all dialogs.

Restart Ollama​

Right-click the Ollama icon in the system tray and choose Quit. Then launch Ollama again from the Start Menu.

To verify it is now listening on all interfaces, open PowerShell and run:

netstat -ano | findstr ":11434"

You should see a line beginning with TCP 0.0.0.0:11434 in the output.

Step 4 — (Optional) Windows Firewall rule​

If your machine has a strict Windows Firewall policy, Docker's virtual network adapter may be blocked from reaching port 11434 even after Step 3. Add an explicit inbound rule to allow it:

Open PowerShell as Administrator and run:

New-NetFirewallRule `
  -DisplayName "Ollama — Docker access" `
  -Direction Inbound `
  -Protocol TCP `
  -LocalPort 11434 `
  -Action Allow

This is a one-time step. You can remove the rule later with:

Remove-NetFirewallRule -DisplayName "Ollama — Docker access"

Step 5 — Configure Conio​

  1. Open Settings → AI Provider in Conio.
  2. Click the Ollama tab.
  3. Leave the Ollama Host field as localhost:11434 — Conio automatically rewrites this to the correct Docker-to-host address internally.
  4. Click Fetch to load your available local models.
  5. Select a model from the dropdown.
  6. Click Save Config.
  7. Click Test Connection to confirm the end-to-end path works.

A successful test returns a live response confirmation with a short snippet from the model.

Updating models​

To pull a newer version of a model:

ollama pull llama3.2

Ollama handles versioning automatically — if the model is already up to date, it exits immediately.

Using cloud models via Ollama​

Ollama also supports a curated list of cloud-hosted models (no local GPU required). These are listed in the Model dropdown under the cloud sections. Cloud models require an active internet connection but do not need the OLLAMA_HOST or firewall steps — they are proxied through Ollama's cloud service rather than your local machine.

Next step​

Licensing →