> ## Documentation Index
> Fetch the complete documentation index at: https://mintlify.com/open-webui/open-webui/llms.txt
> Use this file to discover all available pages before exploring further.
# Ollama Integration
> Connect Open WebUI with Ollama for local LLM inference
## Overview
Ollama is a local LLM runner that allows you to run large language models on your own hardware. Open WebUI provides native integration with Ollama, supporting both local and remote Ollama instances.
## Quick Start
Download and install Ollama from [ollama.ai](https://ollama.ai)
```bash theme={null}
ollama pull llama2
```
Set the Ollama base URL in your environment or admin settings
## Configuration
### Environment Variables
```bash Local Ollama theme={null}
OLLAMA_BASE_URL=http://localhost:11434
ENABLE_OLLAMA_API=True
```
```bash Remote Ollama theme={null}
OLLAMA_BASE_URL=https://ollama.example.com
ENABLE_OLLAMA_API=True
```
```bash Multiple Ollama Instances theme={null}
OLLAMA_BASE_URLS="http://localhost:11434;http://server2:11434;http://server3:11434"
ENABLE_OLLAMA_API=True
```
```bash Docker Network theme={null}
# For Docker installations
OLLAMA_BASE_URL=http://host.docker.internal:11434
```
### Admin Panel Configuration
Navigate to **Admin Panel > Settings > Connections** to configure Ollama:
1. **Enable Ollama API**: Toggle to enable/disable Ollama integration
2. **Base URLs**: Add one or more Ollama server URLs
3. **API Configurations**: Configure advanced settings per instance
## Advanced Configuration
### Multiple Ollama Instances
Open WebUI supports load balancing across multiple Ollama instances:
```json Configuration Example theme={null}
{
"OLLAMA_BASE_URLS": [
"http://localhost:11434",
"http://gpu-server-1:11434",
"http://gpu-server-2:11434"
],
"OLLAMA_API_CONFIGS": {
"0": {
"enable": true,
"key": "",
"prefix_id": "",
"tags": [],
"connection_type": "local"
},
"1": {
"enable": true,
"key": "your-api-key",
"prefix_id": "gpu1",
"tags": ["gpu", "fast"],
"connection_type": "external"
}
}
}
```
```bash Environment Variables theme={null}
OLLAMA_BASE_URLS="http://localhost:11434;http://gpu-server:11434"
```
### Authentication
For secured Ollama instances:
```json theme={null}
{
"OLLAMA_API_CONFIGS": {
"0": {
"enable": true,
"key": "your-bearer-token"
}
}
}
```
The API key will be sent as: `Authorization: Bearer {key}`
### Model Filtering
Filter specific models from an Ollama instance:
```json theme={null}
{
"OLLAMA_API_CONFIGS": {
"0": {
"model_ids": ["llama2", "mistral", "codellama"]
}
}
}
```
### Model Prefixing
Add prefixes to distinguish models from different instances:
```json theme={null}
{
"OLLAMA_API_CONFIGS": {
"0": {
"prefix_id": "local"
},
"1": {
"prefix_id": "remote"
}
}
}
```
Models will appear as `local.llama2` and `remote.llama2`.
## API Endpoints
Open WebUI proxies the following Ollama API endpoints:
### Model Management
* `GET /ollama/api/tags` - List available models\
*File: backend/open\_webui/routers/ollama.py:448*
* `POST /ollama/api/pull` - Pull a model from registry\
*File: backend/open\_webui/routers/ollama.py:708*
* `POST /ollama/api/create` - Create a model from Modelfile\
*File: backend/open\_webui/routers/ollama.py:784*
* `DELETE /ollama/api/delete` - Delete a model\
*File: backend/open\_webui/routers/ollama.py:874*
* `POST /ollama/api/show` - Show model information\
*File: backend/open\_webui/routers/ollama.py:943*
### Inference
* `POST /ollama/api/generate` - Generate completion\
*File: backend/open\_webui/routers/ollama.py:1192*
* `POST /ollama/api/chat` - Chat completion\
*File: backend/open\_webui/routers/ollama.py:1281*
* `POST /ollama/api/embed` - Generate embeddings\
*File: backend/open\_webui/routers/ollama.py:1014*
### OpenAI Compatible
* `POST /ollama/v1/chat/completions` - OpenAI-compatible chat\
*File: backend/open\_webui/routers/ollama.py:1496*
* `POST /ollama/v1/completions` - OpenAI-compatible completions\
*File: backend/open\_webui/routers/ollama.py:1412*
## Docker Integration
### All-in-One Container
```bash theme={null}
# With bundled Ollama
docker run -d -p 3000:8080 \
--gpus=all \
-v ollama:/root/.ollama \
-v open-webui:/app/backend/data \
--name open-webui \
ghcr.io/open-webui/open-webui:ollama
```
### Separate Containers
```yaml docker-compose.yml theme={null}
version: '3'
services:
ollama:
image: ollama/ollama:latest
ports:
- "11434:11434"
volumes:
- ollama:/root/.ollama
deploy:
resources:
reservations:
devices:
- driver: nvidia
count: all
capabilities: [gpu]
open-webui:
image: ghcr.io/open-webui/open-webui:main
ports:
- "3000:8080"
environment:
- OLLAMA_BASE_URL=http://ollama:11434
volumes:
- open-webui:/app/backend/data
depends_on:
- ollama
volumes:
ollama:
open-webui:
```
## Troubleshooting
### Connection Errors
**Docker Network Issues**:
If using Docker, ensure you're using the correct hostname:
* Same machine: `http://host.docker.internal:11434`
* Different container: `http://ollama:11434`
* Network mode host: `http://localhost:11434`
**Firewall**: Ensure port 11434 is accessible
1. Verify Ollama is running: `ollama list`
2. Check ENABLE\_OLLAMA\_API is set to `True`
3. Refresh the models list in the UI
4. Check browser console for errors
Open WebUI will automatically try port 12434 as fallback.
*File: backend/open\_webui/config.py:1046*
### Performance Optimization
**Load Balancing**: The current implementation uses random selection for routing requests.
For production deployments, consider implementing weighted round-robin or least-connections algorithms.
*File: backend/open\_webui/routers/ollama.py:1*
## User Info Forwarding
Forward user information to Ollama for logging and access control:
```bash theme={null}
ENABLE_FORWARD_USER_INFO_HEADERS=true
```
Headers sent:
* `X-OpenWebUI-User-Name`
* `X-OpenWebUI-User-Id`
* `X-OpenWebUI-User-Email`
* `X-OpenWebUI-User-Role`
* `X-OpenWebUI-Chat-Id`
*File: backend/open\_webui/routers/ollama.py:93*
## Best Practices
Distinguish models from different instances with prefixes
Use `GET /ollama/api/ps` to see loaded models and memory usage
Models are cached for better performance (default: 5 minutes TTL)
Configure model-specific GPU allocation in Ollama Modelfile