Connect a local model.

Install Ollama

# macOS / Linux
curl -fsSL https://ollama.com/install.sh | sh

# or via Homebrew
brew install ollama

Once installed, Ollama runs a local API at http://localhost:11434. Rabbit IDE detects it automatically.

Pull a model

ollama pull llama3.1:70b
ollama pull qwen2.5-coder:32b
ollama pull deepseek-coder-v2:16b

Any model that supports tool calling works with the Rabbit agent out of the box: Llama 3.1, Qwen 2.5 Coder, DeepSeek Coder V2, Mistral Large, Phi-3.5, Gemma 2.

Select in the IDE

Open the Model Picker (Cmd+Shift+M), switch to the Local tab, and pick your model. The agent will use it for all steps unless you configure per-step routing.

Apple Silicon acceleration

On M-series Macs, Ollama uses Metal / MLX kernels automatically. Expect 30-60 tokens/sec on a 70B model with 64GB RAM. No GPU drivers or CUDA needed.

Custom endpoints

Any OpenAI-compatible local server works. Set the base URL in Settings > Rabbit AI > Local Provider URL. Examples:llama.cpp, vLLM, LM Studio,Jan.