Run pi.dev with Gemma4 using local Llama.cpp

1. Install Llama.ccp

OSX
brew install llama.cpp

2. Start the server

OSX
llama-server -hf ggml-org/gemma-4-E2B-it-GGUF

3. Install pi.dev

curl -fsSL https://pi.dev/install.sh | sh

4. Create the config files:

.pi/agent/models.json

{
  "providers": {
    "local-llama": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "local-bypass",
      "models": [
        {
          "id": "gemma-4",
          "name": "Gemma 4 Local",
          "input": ["text"],
          "contextWindow": 128000,
          "maxTokens": 4096
        }
      ]
    }
  }
}

~/.pi/agent/settings.json

{
  "lastChangelogVersion": "0.80.6",
  "theme": "dark",
  "defaultProvider": "local-llama",
  "defaultModel": "gemma-4",
  "defaultThinkingLevel": "medium"
}

References:

https://ai.google.dev/gemma/docs/integrations/llamacpp

https://github.com/ggml-org/llama.cpp/blob/master/docs/install.md