> ## Documentation Index
> Fetch the complete documentation index at: https://mintlify.com/open-webui/open-webui/llms.txt
> Use this file to discover all available pages before exploring further.

# Web Search

> Integrate live web search into your AI conversations with 15+ providers

## Overview

Open WebUI's web search integration allows AI models to access real-time information from the internet, enhancing responses with current data, facts, and sources.

## Supported Search Providers

Open WebUI integrates with 15+ search providers, offering flexibility for different use cases and privacy preferences.

### Self-Hosted Options

<CardGroup cols={2}>
  <Card title="SearXNG" icon="server">
    **Privacy-focused metasearch engine**

    * Aggregates results from multiple sources
    * Self-hosted, no tracking
    * Highly customizable
    * Free and open source
  </Card>

  <Card title="YaCy" icon="network-wired">
    **Decentralized search**

    * P2P search network
    * Full control over infrastructure
    * No central authority
  </Card>
</CardGroup>

### Commercial APIs

<Tabs>
  <Tab title="Premium">
    **High-quality paid services:**

    * **Google PSE** (Programmable Search Engine)
      * Reliable, comprehensive results
      * Custom search engines
      * Usage-based pricing
    * **Brave Search**
      * Privacy-focused
      * Independent index
      * Competitive pricing
    * **Kagi**
      * Premium search quality
      * No ads or tracking
      * Advanced features
  </Tab>

  <Tab title="AI-Optimized">
    **Search APIs designed for AI:**

    * **Tavily**
      * Optimized for LLMs
      * Structured results
      * Rich metadata
    * **Perplexity**
      * AI-powered answers
      * Citation support
      * Context-aware results
    * **Exa**
      * Neural search
      * Semantic understanding
      * Developer-friendly
  </Tab>

  <Tab title="Traditional">
    **Established search APIs:**

    * **Bing Search API**
    * **DuckDuckGo**
    * **Serper**
    * **SerpApi**
    * **SearchAPI**
    * **Mojeek**
    * **Jina**
  </Tab>
</Tabs>

## Configuration

### Basic Setup

Enable and configure web search through the admin panel:

```python theme={null}
# From routers/retrieval.py:534-597
{
  "ENABLE_WEB_SEARCH": true,
  "WEB_SEARCH_ENGINE": "searxng",  // Choose your provider
  "WEB_SEARCH_RESULT_COUNT": 5,
  "WEB_SEARCH_CONCURRENT_REQUESTS": 3
}
```

### Provider-Specific Configuration

<Tabs>
  <Tab title="SearXNG">
    ```python theme={null}
    {
      "WEB_SEARCH_ENGINE": "searxng",
      "SEARXNG_QUERY_URL": "http://searxng:8080/search",
      "SEARXNG_LANGUAGE": "en"
    }
    ```

    <Tip>
      Deploy SearXNG using Docker for easy self-hosting:

      ```bash theme={null}
      docker run -d -p 8080:8080 searxng/searxng
      ```
    </Tip>
  </Tab>

  <Tab title="Google PSE">
    ```python theme={null}
    {
      "WEB_SEARCH_ENGINE": "google_pse",
      "GOOGLE_PSE_API_KEY": "your-api-key",
      "GOOGLE_PSE_ENGINE_ID": "your-engine-id"
    }
    ```

    **Setup:**

    1. Create a Programmable Search Engine at [Google CSE](https://programmablesearchengine.google.com/)
    2. Get your API key from [Google Cloud Console](https://console.cloud.google.com/)
    3. Note your Search Engine ID
  </Tab>

  <Tab title="Brave Search">
    ```python theme={null}
    {
      "WEB_SEARCH_ENGINE": "brave",
      "BRAVE_SEARCH_API_KEY": "your-api-key"
    }
    ```

    Sign up at [Brave Search API](https://brave.com/search/api/)
  </Tab>

  <Tab title="Tavily">
    ```python theme={null}
    {
      "WEB_SEARCH_ENGINE": "tavily",
      "TAVILY_API_KEY": "your-api-key"
    }
    ```

    Get your key at [Tavily AI](https://tavily.com/)
  </Tab>

  <Tab title="Perplexity">
    ```python theme={null}
    {
      "WEB_SEARCH_ENGINE": "perplexity",
      "PERPLEXITY_API_KEY": "your-api-key",
      "PERPLEXITY_MODEL": "sonar-medium-online",
      "PERPLEXITY_SEARCH_API_URL": "https://api.perplexity.ai"
    }
    ```
  </Tab>
</Tabs>

### Advanced Settings

<CardGroup cols={2}>
  <Card title="Performance" icon="gauge">
    ```python theme={null}
    {
      "WEB_SEARCH_CONCURRENT_REQUESTS": 3,
      "WEB_LOADER_CONCURRENT_REQUESTS": 5,
      "WEB_LOADER_TIMEOUT": "30"
    }
    ```
  </Card>

  <Card title="Security" icon="shield">
    ```python theme={null}
    {
      "WEB_SEARCH_TRUST_ENV": true,
      "ENABLE_WEB_LOADER_SSL_VERIFICATION": true
    }
    ```
  </Card>

  <Card title="Filtering" icon="filter">
    ```python theme={null}
    {
      "WEB_SEARCH_DOMAIN_FILTER_LIST": [
        "wikipedia.org",
        "github.com"
      ]
    }
    ```
  </Card>

  <Card title="Processing" icon="gear">
    ```python theme={null}
    {
      "BYPASS_WEB_SEARCH_WEB_LOADER": false,
      "BYPASS_WEB_SEARCH_EMBEDDING_AND_RETRIEVAL": false
    }
    ```
  </Card>
</CardGroup>

## Web Content Loading

### Loading Engines

Choose how web pages are fetched and processed:

<Tabs>
  <Tab title="Default">
    **Standard HTTP requests**

    * Fast and lightweight
    * Works for most sites
    * No JavaScript execution

    ```python theme={null}
    {
      "WEB_LOADER_ENGINE": "default"
    }
    ```
  </Tab>

  <Tab title="Playwright">
    **Browser automation**

    * JavaScript rendering
    * Handles dynamic content
    * Screenshot capabilities

    ```python theme={null}
    {
      "WEB_LOADER_ENGINE": "playwright",
      "PLAYWRIGHT_WS_URL": "ws://playwright:3000",
      "PLAYWRIGHT_TIMEOUT": 30000
    }
    ```
  </Tab>

  <Tab title="Firecrawl">
    **Specialized scraping**

    * Advanced extraction
    * Anti-bot bypass
    * Rich metadata

    ```python theme={null}
    {
      "WEB_LOADER_ENGINE": "firecrawl",
      "FIRECRAWL_API_KEY": "your-key",
      "FIRECRAWL_API_BASE_URL": "https://api.firecrawl.dev",
      "FIRECRAWL_TIMEOUT": "60"
    }
    ```
  </Tab>

  <Tab title="External">
    **Custom loader service**

    * Bring your own scraper
    * Custom processing logic
    * API-based integration

    ```python theme={null}
    {
      "WEB_LOADER_ENGINE": "external",
      "EXTERNAL_WEB_LOADER_URL": "http://your-loader:8000",
      "EXTERNAL_WEB_LOADER_API_KEY": "your-key"
    }
    ```
  </Tab>
</Tabs>

## Using Web Search in Chat

### Basic Usage

Trigger web search by mentioning URLs or topics:

```
# Search and cite sources
What are the latest developments in quantum computing?

# Direct URL loading
Summarize https://example.com/article

# Multiple URLs
Compare these articles:
- https://source1.com
- https://source2.com
```

### Search Workflow

<Steps>
  <Step title="Query Detection">
    AI determines if web search would enhance the response
  </Step>

  <Step title="Search Execution">
    Query sent to configured search provider

    * Returns top N results (configurable)
    * Concurrent requests for multiple queries
  </Step>

  <Step title="Content Fetching">
    Web loader retrieves page content

    * Parallel loading of multiple URLs
    * Timeout protection
    * SSL verification
  </Step>

  <Step title="Processing">
    Content prepared for RAG:

    * Text extraction
    * Chunking (if enabled)
    * Embedding generation (if enabled)
  </Step>

  <Step title="Response Generation">
    AI uses retrieved content to formulate answer

    * Cites sources
    * Combines multiple perspectives
    * Provides attribution
  </Step>
</Steps>

## YouTube Integration

Extract and search YouTube video transcripts:

```python theme={null}
{
  "YOUTUBE_LOADER_LANGUAGE": ["en", "es", "fr"],
  "YOUTUBE_LOADER_PROXY_URL": "http://proxy:8080",
  "YOUTUBE_LOADER_TRANSLATION": "en"
}
```

**Features:**

* Automatic transcript extraction
* Multi-language support
* Translation capabilities
* Proxy support for restricted regions

**Usage:**

```
Summarize this video: https://youtube.com/watch?v=VIDEO_ID
```

## Domain Filtering

Control which domains are searched:

```python theme={null}
{
  "WEB_SEARCH_DOMAIN_FILTER_LIST": [
    "wikipedia.org",       # Encyclopedic content
    "github.com",          # Code repositories
    "stackoverflow.com",   # Technical Q&A
    "arxiv.org"            # Academic papers
  ]
}
```

<Note>
  When filter list is populated, only results from these domains are returned. Leave empty to search all domains.
</Note>

## Performance Optimization

### Concurrent Requests

Balance speed and resource usage:

```python theme={null}
{
  "WEB_SEARCH_CONCURRENT_REQUESTS": 3,    // Parallel searches
  "WEB_LOADER_CONCURRENT_REQUESTS": 5     // Parallel page loads
}
```

**Recommendations:**

* **Small deployments**: 2-3 concurrent searches, 3-5 loaders
* **Medium deployments**: 3-5 concurrent searches, 5-10 loaders
* **Large deployments**: 5-10 concurrent searches, 10-20 loaders

### Timeouts

```python theme={null}
{
  "WEB_LOADER_TIMEOUT": "30",           // Seconds
  "PLAYWRIGHT_TIMEOUT": 30000,          // Milliseconds
  "FIRECRAWL_TIMEOUT": "60"             // Seconds
}
```

<Warning>
  Set reasonable timeouts to prevent resource exhaustion. Slow sites can hang requests indefinitely.
</Warning>

## Bypass Options

### Skip Web Loader

Use search results without fetching full content:

```python theme={null}
{
  "BYPASS_WEB_SEARCH_WEB_LOADER": true
}
```

**When to use:**

* Search summaries sufficient
* Reduce API calls/bandwidth
* Faster responses needed

### Skip Embedding

Disable RAG processing for web content:

```python theme={null}
{
  "BYPASS_WEB_SEARCH_EMBEDDING_AND_RETRIEVAL": true
}
```

**When to use:**

* Direct content injection
* Avoid embedding costs
* Real-time freshness required

## External Search Integration

Connect custom search services:

```python theme={null}
{
  "WEB_SEARCH_ENGINE": "external",
  "EXTERNAL_WEB_SEARCH_URL": "http://your-search-api:8000",
  "EXTERNAL_WEB_SEARCH_API_KEY": "your-key"
}
```

**API Contract:**

```json theme={null}
// Request
POST /search
{
  "queries": ["search query 1", "query 2"],
  "count": 5
}

// Response
{
  "results": [
    {
      "title": "Result Title",
      "url": "https://example.com",
      "content": "Snippet..."
    }
  ]
}
```

## API Endpoints

### Configuration

```bash theme={null}
# Get web search config
GET /api/v1/retrieval/config

# Response includes web search settings
{
  "web": {
    "ENABLE_WEB_SEARCH": true,
    "WEB_SEARCH_ENGINE": "searxng",
    ...
  }
}
```

### Update Configuration

```bash theme={null}
POST /api/v1/retrieval/config/update
{
  "web": {
    "ENABLE_WEB_SEARCH": true,
    "WEB_SEARCH_ENGINE": "brave",
    "BRAVE_SEARCH_API_KEY": "new-key"
  }
}
```

## Best Practices

<CardGroup cols={2}>
  <Card title="Choose Right Provider" icon="magnifying-glass">
    **Consider:**

    * Privacy requirements
    * Budget constraints
    * Result quality needs
    * Geographic coverage
    * API rate limits
  </Card>

  <Card title="Optimize Result Count" icon="list">
    **Balance:**

    * More results = better coverage
    * Fewer results = faster responses
    * Recommended: 3-7 results
    * Adjust based on use case
  </Card>

  <Card title="Use Domain Filtering" icon="filter">
    **Benefits:**

    * Improve result quality
    * Reduce irrelevant content
    * Focus on trusted sources
    * Faster processing
  </Card>

  <Card title="Monitor Costs" icon="dollar-sign">
    **Track:**

    * API usage per provider
    * Bandwidth consumption
    * Processing time
    * Set up alerts for limits
  </Card>
</CardGroup>

## Troubleshooting

<AccordionGroup>
  <Accordion title="No search results returned">
    **Check:**

    * Provider API key is valid
    * Search engine is properly configured
    * Rate limits not exceeded
    * Domain filter not too restrictive
    * Network connectivity to provider
  </Accordion>

  <Accordion title="Slow web loading">
    **Solutions:**

    * Reduce `WEB_LOADER_TIMEOUT`
    * Increase concurrent requests
    * Use faster web loader engine
    * Enable `BYPASS_WEB_SEARCH_WEB_LOADER`
    * Check network latency
  </Accordion>

  <Accordion title="SSL certificate errors">
    **Options:**

    * Ensure `ENABLE_WEB_LOADER_SSL_VERIFICATION` is true
    * Update CA certificates
    * Configure proxy with valid certs
    * For internal sites, use custom CA
  </Accordion>

  <Accordion title="YouTube videos not working">
    **Verify:**

    * Video has captions/transcript
    * Language is in `YOUTUBE_LOADER_LANGUAGE`
    * Proxy configured if in restricted region
    * Video is publicly accessible
  </Accordion>
</AccordionGroup>

## Security Considerations

<Warning>
  **Important security practices:**

  1. **SSL Verification**: Always enable in production
  2. **API Keys**: Use environment variables, never commit
  3. **Domain Filtering**: Prevent access to internal networks
  4. **Rate Limiting**: Implement to prevent abuse
  5. **Timeout Settings**: Prevent resource exhaustion
</Warning>

### URL Validation

Open WebUI validates URLs to prevent SSRF attacks:

* Blocks private IP ranges (10.0.0.0/8, 192.168.0.0/16, etc.)
* Prevents localhost access
* Validates URL format
* Enforces timeout limits