> ## Documentation Index
> Fetch the complete documentation index at: https://mintlify.com/open-webui/open-webui/llms.txt
> Use this file to discover all available pages before exploring further.

# RAG Integration

> Retrieval-Augmented Generation with document libraries and advanced search capabilities

## Overview

Open WebUI's RAG (Retrieval-Augmented Generation) system enables powerful document-based chat interactions by combining vector search, hybrid retrieval, and multiple content extraction engines.

## Document Upload & Processing

### Supported File Types

Open WebUI supports extensive file format compatibility:

<Tabs>
  <Tab title="Documents">
    * PDF (with OCR support)
    * Word (DOC, DOCX)
    * PowerPoint (PPT, PPTX)
    * Excel (XLS, XLSX)
    * Plain text (TXT, MD)
    * Rich text (RTF)
  </Tab>

  <Tab title="Code & Data">
    * Source code (PY, JS, TS, etc.)
    * JSON, XML, YAML
    * CSV, TSV
    * Configuration files
  </Tab>

  <Tab title="Images">
    * PNG, JPEG, GIF
    * SVG, WebP
    * TIFF, BMP
    * Image OCR extraction
  </Tab>

  <Tab title="Web Content">
    * URLs (automatic scraping)
    * HTML files
    * YouTube videos (transcript extraction)
  </Tab>
</Tabs>

### Content Extraction Engines

Choose from multiple extraction engines based on your needs:

<CardGroup cols={2}>
  <Card title="Tika" icon="file">
    **Apache Tika** - Universal document parser

    * Supports 1000+ file formats
    * Metadata extraction
    * Self-hosted option
  </Card>

  <Card title="Docling" icon="wand-magic-sparkles">
    **IBM Docling** - AI-powered extraction

    * Advanced layout understanding
    * Table structure preservation
    * High accuracy for complex documents
  </Card>

  <Card title="Document Intelligence" icon="brain">
    **Azure Document Intelligence**

    * Cloud-based OCR
    * Form recognition
    * Layout analysis
    * Custom model support
  </Card>

  <Card title="Mistral OCR" icon="image">
    **Mistral OCR API**

    * AI-powered image text extraction
    * Multi-language support
    * High-quality results
  </Card>
</CardGroup>

### Configuration

Configure content extraction settings:

```python theme={null}
# From routers/retrieval.py:481-511
{
  "CONTENT_EXTRACTION_ENGINE": "tika",  // tika, docling, azure, mistral
  "PDF_EXTRACT_IMAGES": true,
  "PDF_LOADER_MODE": "auto",  // auto, fast, quality
  
  // Tika settings
  "TIKA_SERVER_URL": "http://tika:9998",
  
  // Docling settings
  "DOCLING_SERVER_URL": "http://docling:5000",
  "DOCLING_API_KEY": "your-api-key",
  "DOCLING_PARAMS": {...},
  
  // Azure Document Intelligence
  "DOCUMENT_INTELLIGENCE_ENDPOINT": "https://your-instance.cognitiveservices.azure.com",
  "DOCUMENT_INTELLIGENCE_KEY": "your-key",
  "DOCUMENT_INTELLIGENCE_MODEL": "prebuilt-read",
  
  // Mistral OCR
  "MISTRAL_OCR_API_BASE_URL": "https://api.mistral.ai/v1",
  "MISTRAL_OCR_API_KEY": "your-key"
}
```

## Vector Database Support

Open WebUI supports 9 vector database options:

<Tabs>
  <Tab title="ChromaDB">
    **Default embedded database**

    * No external dependencies
    * Perfect for single-node deployments
    * Persistent storage
  </Tab>

  <Tab title="PostgreSQL">
    **PGVector extension**

    * Use existing PostgreSQL
    * ACID compliance
    * Familiar SQL interface
  </Tab>

  <Tab title="Qdrant">
    **High-performance vector search**

    * Advanced filtering
    * Horizontal scaling
    * Rich query capabilities
  </Tab>

  <Tab title="Cloud Options">
    **Managed services:**

    * **Milvus**: Distributed vector DB
    * **Elasticsearch**: Full-text + vector
    * **OpenSearch**: AWS-compatible
    * **Pinecone**: Serverless vector DB
    * **Oracle 23ai**: Enterprise database
  </Tab>
</Tabs>

## Embedding Configuration

### Embedding Models

Configure embedding generation:

<Steps>
  <Step title="Choose Engine">
    ```python theme={null}
    {
      "RAG_EMBEDDING_ENGINE": "ollama",  // "", ollama, openai, azure_openai
      "RAG_EMBEDDING_MODEL": "nomic-embed-text"
    }
    ```
  </Step>

  <Step title="Configure Provider">
    **Ollama:**

    ```python theme={null}
    {
      "RAG_OLLAMA_BASE_URL": "http://ollama:11434",
      "RAG_OLLAMA_API_KEY": ""
    }
    ```

    **OpenAI:**

    ```python theme={null}
    {
      "RAG_OPENAI_API_BASE_URL": "https://api.openai.com/v1",
      "RAG_OPENAI_API_KEY": "sk-..."
    }
    ```

    **Azure OpenAI:**

    ```python theme={null}
    {
      "RAG_AZURE_OPENAI_BASE_URL": "https://your-instance.openai.azure.com",
      "RAG_AZURE_OPENAI_API_KEY": "your-key",
      "RAG_AZURE_OPENAI_API_VERSION": "2023-05-15"
    }
    ```
  </Step>

  <Step title="Optimize Performance">
    ```python theme={null}
    {
      "RAG_EMBEDDING_BATCH_SIZE": 100,
      "ENABLE_ASYNC_EMBEDDING": true,
      "RAG_EMBEDDING_CONCURRENT_REQUESTS": 4
    }
    ```
  </Step>
</Steps>

<Warning>
  Changing embedding models requires re-embedding all existing documents. Plan migrations carefully.
</Warning>

## Chunking Strategies

Optimize document chunking for better retrieval:

### Text Splitters

<Tabs>
  <Tab title="Recursive">
    **RecursiveCharacterTextSplitter** (Default)

    * Splits on multiple separators hierarchically
    * Preserves semantic meaning
    * Best for general content

    ```python theme={null}
    {
      "TEXT_SPLITTER": "recursive",
      "CHUNK_SIZE": 1500,
      "CHUNK_OVERLAP": 200
    }
    ```
  </Tab>

  <Tab title="Token-Based">
    **TokenTextSplitter**

    * Splits by token count
    * Precise size control
    * Better for LLM context limits

    ```python theme={null}
    {
      "TEXT_SPLITTER": "token",
      "CHUNK_SIZE": 1000,  // tokens
      "CHUNK_OVERLAP": 100
    }
    ```
  </Tab>

  <Tab title="Markdown">
    **MarkdownHeaderTextSplitter**

    * Splits by markdown structure
    * Preserves headers
    * Maintains hierarchy

    ```python theme={null}
    {
      "ENABLE_MARKDOWN_HEADER_TEXT_SPLITTER": true,
      "TEXT_SPLITTER": "markdown"
    }
    ```
  </Tab>
</Tabs>

### Chunking Parameters

<Note>
  Optimal chunking balances:

  * **Chunk Size**: Larger = more context, fewer chunks
  * **Overlap**: Prevents information loss at boundaries
  * **Min Size**: Ensures chunks contain meaningful content
</Note>

```python theme={null}
{
  "CHUNK_SIZE": 1500,              // Characters/tokens per chunk
  "CHUNK_MIN_SIZE_TARGET": 100,    // Minimum viable chunk
  "CHUNK_OVERLAP": 200             // Overlap between chunks
}
```

## Hybrid Search

Combine vector and keyword search for better results.

### Enabling Hybrid Search

```python theme={null}
{
  "ENABLE_RAG_HYBRID_SEARCH": true,
  "ENABLE_RAG_HYBRID_SEARCH_ENRICHED_TEXTS": true,
  "HYBRID_BM25_WEIGHT": 0.5  // 0=vector only, 1=BM25 only
}
```

### How It Works

<Steps>
  <Step title="Vector Search">
    Semantic similarity using embeddings
  </Step>

  <Step title="BM25 Keyword Search">
    Traditional keyword matching with TF-IDF
  </Step>

  <Step title="Score Fusion">
    Combine scores using configured weight
  </Step>

  <Step title="Reranking">
    Optionally rerank results with dedicated model
  </Step>
</Steps>

## Reranking

Improve retrieval quality with reranking models.

### Configuration

<Tabs>
  <Tab title="Local Models">
    ```python theme={null}
    {
      "RAG_RERANKING_ENGINE": "",
      "RAG_RERANKING_MODEL": "BAAI/bge-reranker-large",
      "TOP_K_RERANKER": 10
    }
    ```

    **Supported models:**

    * BAAI/bge-reranker-\*
    * jinaai/jina-colbert-v2
    * CrossEncoder models
  </Tab>

  <Tab title="External API">
    ```python theme={null}
    {
      "RAG_RERANKING_ENGINE": "external",
      "RAG_RERANKING_MODEL": "your-model",
      "RAG_EXTERNAL_RERANKER_URL": "http://reranker:8000",
      "RAG_EXTERNAL_RERANKER_API_KEY": "key",
      "RAG_EXTERNAL_RERANKER_TIMEOUT": "30"
    }
    ```
  </Tab>
</Tabs>

### Reranking Process

1. **Initial Retrieval**: Get top N candidates (e.g., 50)
2. **Rerank**: Score candidates with reranking model
3. **Filter**: Keep top K (e.g., 10) best matches
4. **Threshold**: Optionally filter by relevance score

```python theme={null}
{
  "TOP_K": 50,                    // Initial retrieval
  "TOP_K_RERANKER": 10,           // After reranking
  "RELEVANCE_THRESHOLD": 0.0      // Minimum score (0-1)
}
```

## Web Search Integration

Enhance RAG with live web search.

### Supported Providers

<CardGroup cols={2}>
  <Card title="SearXNG" icon="magnifying-glass">
    Self-hosted metasearch engine
  </Card>

  <Card title="Google PSE" icon="google">
    Programmable Search Engine
  </Card>

  <Card title="Brave Search" icon="shield">
    Privacy-focused search API
  </Card>

  <Card title="Kagi" icon="key">
    Premium search API
  </Card>

  <Card title="Tavily" icon="globe">
    AI-optimized search
  </Card>

  <Card title="Perplexity" icon="brain">
    AI-powered answers
  </Card>
</CardGroup>

### Configuration Example

```python theme={null}
{
  "ENABLE_WEB_SEARCH": true,
  "WEB_SEARCH_ENGINE": "searxng",
  "SEARXNG_QUERY_URL": "http://searxng:8080/search",
  "WEB_SEARCH_RESULT_COUNT": 5,
  "WEB_SEARCH_CONCURRENT_REQUESTS": 3
}
```

### Web Search Workflow

<Steps>
  <Step title="Search Execution">
    Query configured search provider for relevant URLs
  </Step>

  <Step title="Content Loading">
    Fetch and extract text from web pages
  </Step>

  <Step title="Processing">
    Chunk and embed web content
  </Step>

  <Step title="RAG Integration">
    Combine with document library results
  </Step>
</Steps>

## Using RAG in Chat

### Accessing Documents

Reference documents in chat using the `#` command:

```
# Single document
#document-name What is the summary of this report?

# Multiple documents
#doc1 #doc2 #doc3 Compare these three documents

# Web URL
#https://example.com/article What does this article say about AI?
```

### RAG Template

Customize how retrieved context is presented:

```python theme={null}
{
  "RAG_TEMPLATE": """Use the following context to answer the question:

Context:
{{CONTEXT}}

Question: {{QUERY}}"""
}
```

<Tip>
  The `{{CONTEXT}}` placeholder is replaced with retrieved document chunks, and `{{QUERY}}` with the user's question.
</Tip>

## Advanced Features

### Full Context Mode

Bypass chunking for small documents:

```python theme={null}
{
  "RAG_FULL_CONTEXT": true
}
```

When enabled:

* Small documents sent in full
* Better context preservation
* Higher token usage

### Bypass Embedding

Skip vector search for specific use cases:

```python theme={null}
{
  "BYPASS_EMBEDDING_AND_RETRIEVAL": true
}
```

<Warning>
  This disables RAG functionality. Documents won't be searchable.
</Warning>

### YouTube Integration

Extract transcripts from YouTube videos:

```python theme={null}
{
  "YOUTUBE_LOADER_LANGUAGE": ["en", "es", "fr"],
  "YOUTUBE_LOADER_PROXY_URL": "http://proxy:8080",
  "YOUTUBE_LOADER_TRANSLATION": "en"
}
```

Usage:

```
#https://youtube.com/watch?v=VIDEO_ID Summarize this video
```

## Cloud Storage Integration

Import documents from cloud services:

<Tabs>
  <Tab title="Google Drive">
    ```python theme={null}
    {
      "ENABLE_GOOGLE_DRIVE_INTEGRATION": true
    }
    ```

    Features:

    * OAuth authentication
    * File picker interface
    * Automatic download and processing
  </Tab>

  <Tab title="OneDrive">
    ```python theme={null}
    {
      "ENABLE_ONEDRIVE_INTEGRATION": true
    }
    ```

    Features:

    * SharePoint support
    * Microsoft Graph API
    * Enterprise integration
  </Tab>
</Tabs>

## Performance Optimization

### Async Embedding

Parallelize embedding generation:

```python theme={null}
{
  "ENABLE_ASYNC_EMBEDDING": true,
  "RAG_EMBEDDING_CONCURRENT_REQUESTS": 4,
  "RAG_EMBEDDING_BATCH_SIZE": 100
}
```

**Benefits:**

* Faster document processing
* Better resource utilization
* Configurable concurrency

### Web Loader Optimization

```python theme={null}
{
  "WEB_LOADER_CONCURRENT_REQUESTS": 5,
  "WEB_LOADER_TIMEOUT": "30",
  "ENABLE_WEB_LOADER_SSL_VERIFICATION": true
}
```

## API Reference

### RAG Configuration

```bash theme={null}
# Get current settings
GET /api/v1/retrieval/config

# Update configuration
POST /api/v1/retrieval/config/update
```

### Embedding Management

```bash theme={null}
# Get embedding config
GET /api/v1/retrieval/embedding

# Update embedding model
POST /api/v1/retrieval/embedding/update
```

### Web Search

```bash theme={null}
# Perform web search
POST /api/v1/retrieval/web/search
{
  "queries": ["search query 1", "search query 2"]
}
```

## Best Practices

<CardGroup cols={2}>
  <Card title="Choose Right Chunk Size" icon="scissors">
    * Too small: Loss of context
    * Too large: Poor retrieval precision
    * Start with 1500 characters
    * Adjust based on content type
  </Card>

  <Card title="Use Hybrid Search" icon="magnifying-glass">
    * Better than vector-only for many queries
    * Combines semantic + keyword matching
    * Tune weight based on use case
  </Card>

  <Card title="Enable Reranking" icon="arrow-up-arrow-down">
    * Significantly improves result quality
    * Small performance cost
    * Worth it for production use
  </Card>

  <Card title="Monitor Embedding Costs" icon="dollar-sign">
    * Track API usage for cloud providers
    * Consider local models for volume
    * Batch processing reduces costs
  </Card>
</CardGroup>