> ## Documentation Index
> Fetch the complete documentation index at: https://mintlify.com/open-webui/open-webui/llms.txt
> Use this file to discover all available pages before exploring further.

# LLM Provider Integrations

> Connect to Anthropic, Google AI, and other LLM providers

## Overview

Open WebUI supports multiple LLM providers through OpenAI-compatible API endpoints. This includes native integrations with Anthropic Claude, Google Gemini, and many others.

## Supported Providers

<CardGroup cols={2}>
  <Card title="Anthropic Claude" icon="comment">
    Claude 3 Opus, Sonnet, and Haiku models
  </Card>

  <Card title="Google AI" icon="google">
    Gemini Pro, Gemini Ultra, and PaLM 2
  </Card>

  <Card title="OpenRouter" icon="route">
    Access to 100+ models from multiple providers
  </Card>

  <Card title="Groq" icon="bolt">
    Ultra-fast LLM inference
  </Card>

  <Card title="Mistral AI" icon="wind">
    Mistral, Mixtral, and specialized models
  </Card>

  <Card title="Together AI" icon="users">
    Open source models with fast inference
  </Card>
</CardGroup>

## Anthropic Claude

### Configuration

<CodeGroup>
  ```bash Environment Variables theme={null}
  OPENAI_API_BASE_URL=https://api.anthropic.com/v1
  OPENAI_API_KEY=sk-ant-...
  ENABLE_OPENAI_API=True
  ```

  ```bash Docker theme={null}
  docker run -d -p 3000:8080 \
    -e OPENAI_API_BASE_URL=https://api.anthropic.com/v1 \
    -e OPENAI_API_KEY=sk-ant-... \
    -v open-webui:/app/backend/data \
    ghcr.io/open-webui/open-webui:main
  ```

  ```json Admin Panel theme={null}
  {
    "OPENAI_API_BASE_URLS": [
      "https://api.anthropic.com/v1"
    ],
    "OPENAI_API_KEYS": [
      "sk-ant-..."
    ]
  }
  ```
</CodeGroup>

### Available Models

* `claude-3-opus-20240229` - Most powerful model
* `claude-3-sonnet-20240229` - Balanced performance and speed
* `claude-3-haiku-20240307` - Fastest and most affordable
* `claude-2.1` - Previous generation

<Note>
  Open WebUI automatically detects Anthropic URLs and applies appropriate model mappings.

  *File: backend/open\_webui/utils/anthropic.py*
</Note>

### Features

* System prompts
* Tool/function calling
* Vision capabilities (Claude 3)
* Long context (200K tokens)

## Google AI (Gemini)

### Configuration

<CodeGroup>
  ```bash Dedicated Variables theme={null}
  GEMINI_API_KEY=your-api-key
  GEMINI_API_BASE_URL=https://generativelanguage.googleapis.com/v1
  ```

  ```bash OpenAI-Compatible theme={null}
  OPENAI_API_BASE_URL=https://generativelanguage.googleapis.com/v1
  OPENAI_API_KEY=your-gemini-key
  ```

  ```python Python SDK (via requirements.txt) theme={null}
  # Open WebUI includes:
  google-genai==1.62.0
  ```
</CodeGroup>

### Available Models

* `gemini-1.5-pro` - Latest and most capable
* `gemini-1.5-flash` - Optimized for speed
* `gemini-1.0-pro` - Previous generation

### Features

* Multimodal (text, image, video, audio)
* 2M context window (Gemini 1.5 Pro)
* Code execution
* Function calling

## OpenRouter

Access 100+ models from multiple providers through a single API.

### Configuration

<CodeGroup>
  ```bash Environment theme={null}
  OPENAI_API_BASE_URL=https://openrouter.ai/api/v1
  OPENAI_API_KEY=sk-or-v1-...
  ```

  ```json With Custom Headers theme={null}
  {
    "OPENAI_API_BASE_URLS": ["https://openrouter.ai/api/v1"],
    "OPENAI_API_KEYS": ["sk-or-v1-..."],
    "OPENAI_API_CONFIGS": {
      "0": {
        "headers": {
          "HTTP-Referer": "https://openwebui.com/",
          "X-Title": "Open WebUI"
        }
      }
    }
  }
  ```
</CodeGroup>

<Note>
  OpenRouter requires HTTP-Referer and X-Title headers. Open WebUI automatically adds these when it detects an OpenRouter URL.

  *File: backend/open\_webui/routers/openai.py:134*
</Note>

### Popular Models

* `anthropic/claude-3-opus`
* `google/gemini-pro-1.5`
* `meta-llama/llama-3-70b-instruct`
* `mistralai/mixtral-8x7b-instruct`
* `openai/gpt-4-turbo`

## Groq

Ultra-fast inference for open source models.

### Configuration

```bash theme={null}
OPENAI_API_BASE_URL=https://api.groq.com/openai/v1
OPENAI_API_KEY=gsk_...
```

### Available Models

* `llama-3.1-70b-versatile`
* `llama-3.1-8b-instant`
* `mixtral-8x7b-32768`
* `gemma-7b-it`

### Features

* Extremely fast inference (500+ tokens/sec)
* Free tier available
* OpenAI-compatible API

## Mistral AI

### Configuration

```bash theme={null}
OPENAI_API_BASE_URL=https://api.mistral.ai/v1
OPENAI_API_KEY=your-mistral-key
```

### Available Models

* `mistral-large-latest` - Most capable
* `mistral-medium-latest` - Balanced
* `mistral-small-latest` - Fast and efficient
* `mixtral-8x7b` - Open source MoE
* `codestral-latest` - Code generation specialist

## Together AI

Open source models with fast inference.

### Configuration

```bash theme={null}
OPENAI_API_BASE_URL=https://api.together.xyz/v1
OPENAI_API_KEY=your-together-key
```

### Features

* 50+ open source models
* Fine-tuning support
* Fast inference
* Competitive pricing

## Perplexity AI

### Configuration

```bash theme={null}
OPENAI_API_BASE_URL=https://api.perplexity.ai
OPENAI_API_KEY=pplx-...
```

### Models

* `llama-3.1-sonar-large-128k-online` - With web search
* `llama-3.1-sonar-small-128k-online` - Faster with web search
* `llama-3.1-70b-instruct` - Base model

## Cohere

### Configuration

```bash theme={null}
OPENAI_API_BASE_URL=https://api.cohere.ai/v1
OPENAI_API_KEY=your-cohere-key
```

### Models

* `command-r-plus` - Most capable
* `command-r` - Balanced
* `command-light` - Fast and efficient

## Hugging Face Inference

### Configuration

```bash theme={null}
OPENAI_API_BASE_URL=https://api-inference.huggingface.co/models
OPENAI_API_KEY=hf_...
```

### Usage

Access any model on Hugging Face:

```
model: meta-llama/Llama-2-70b-chat-hf
model: mistralai/Mixtral-8x7B-Instruct-v0.1
```

## Multiple Provider Setup

Configure multiple providers simultaneously:

<CodeGroup>
  ```bash Environment Variables theme={null}
  OPENAI_API_BASE_URLS="https://api.openai.com/v1;https://api.anthropic.com/v1;https://api.groq.com/openai/v1"
  OPENAI_API_KEYS="sk-openai-key;sk-ant-key;gsk-groq-key"
  ```

  ```json Configuration theme={null}
  {
    "OPENAI_API_BASE_URLS": [
      "https://api.openai.com/v1",
      "https://api.anthropic.com/v1",
      "https://api.groq.com/openai/v1"
    ],
    "OPENAI_API_KEYS": [
      "sk-openai-key",
      "sk-ant-key",
      "gsk-groq-key"
    ],
    "OPENAI_API_CONFIGS": {
      "0": {
        "prefix_id": "openai",
        "tags": ["paid", "gpt4"]
      },
      "1": {
        "prefix_id": "claude",
        "tags": ["anthropic", "vision"]
      },
      "2": {
        "prefix_id": "groq",
        "tags": ["fast", "free"]
      }
    }
  }
  ```
</CodeGroup>

## Provider-Specific Features

### Vision Models

Providers with vision support:

* OpenAI: `gpt-4-vision-preview`, `gpt-4-turbo`
* Anthropic: `claude-3-opus`, `claude-3-sonnet`, `claude-3-haiku`
* Google: `gemini-1.5-pro`, `gemini-1.5-flash`

### Function Calling

Providers with function/tool calling:

* OpenAI: All GPT models
* Anthropic: Claude 3 models
* Google: Gemini models
* Mistral: Most models

### Streaming

All providers support streaming responses through Server-Sent Events (SSE).

## Cost Optimization

<CardGroup cols={2}>
  <Card title="Model Selection" icon="sliders">
    Use smaller models (haiku, flash, small) for simple tasks
  </Card>

  <Card title="Provider Comparison" icon="scale-balanced">
    Compare costs across providers for equivalent capabilities
  </Card>

  <Card title="Caching" icon="database">
    Enable model caching to reduce duplicate API calls
  </Card>

  <Card title="Free Tiers" icon="gift">
    Leverage free tiers from Groq, Hugging Face, etc.
  </Card>
</CardGroup>

## Troubleshooting

<AccordionGroup>
  <Accordion title="Provider Not Responding">
    1. Verify API key is valid
    2. Check base URL is correct
    3. Ensure provider service is operational
    4. Check firewall/network restrictions
  </Accordion>

  <Accordion title="Models Not Appearing">
    Some providers require specific model IDs in configuration:

    ```json theme={null}
    {
      "OPENAI_API_CONFIGS": {
        "0": {
          "model_ids": ["model-name-1", "model-name-2"]
        }
      }
    }
    ```
  </Accordion>

  <Accordion title="Rate Limiting">
    Each provider has different rate limits:

    * Use multiple API keys
    * Implement exponential backoff
    * Monitor usage through provider dashboard
  </Accordion>
</AccordionGroup>

## Best Practices

1. **API Key Security**: Store keys in environment variables
2. **Model Prefixing**: Use prefixes to distinguish provider models
3. **Cost Monitoring**: Track usage across providers
4. **Fallback Providers**: Configure multiple providers for redundancy
5. **Model Tagging**: Use tags to categorize models by capability

## References

* Anthropic API: [docs.anthropic.com](https://docs.anthropic.com)
* Google AI: [ai.google.dev](https://ai.google.dev)
* OpenRouter: [openrouter.ai/docs](https://openrouter.ai/docs)
* Groq: [console.groq.com/docs](https://console.groq.com/docs)
* Mistral: [docs.mistral.ai](https://docs.mistral.ai)