System Architecture

Understanding the Open WebUI architecture and technology stack

Overview

Open WebUI is a full-stack application built with a modern, scalable architecture. The system consists of a SvelteKit frontend and a FastAPI backend, designed to provide a responsive AI chat interface with extensive customization capabilities.

Technology Stack

Frontend

• Framework: SvelteKit 5.x
• Build Tool: Vite 5.x
• Language: TypeScript 5.x
• Styling: Tailwind CSS 4.x
• State Management: Svelte stores
• Real-time Communication: Socket.IO Client
• Code Editor: CodeMirror 6
• Rich Text: TipTap (ProseMirror)

Backend

• Framework: FastAPI 0.128.x
• Server: Uvicorn with workers support
• Language: Python 3.11-3.12
• Database ORM: SQLAlchemy 2.x + Peewee 3.x
• Migrations: Alembic + Peewee-migrate
• Authentication: PyJWT, python-jose
• Real-time: Python-SocketIO 5.x
• Task Scheduling: APScheduler 3.x

Architecture Layers

1. Presentation Layer (Frontend)

┌─────────────────────────────────────┐
│      SvelteKit Application          │
│  ┌───────────┐    ┌──────────────┐  │
│  │   Pages   │    │  Components  │  │
│  └───────────┘    └──────────────┘  │
│  ┌───────────┐    ┌──────────────┐  │
│  │  Stores   │    │   Services   │  │
│  └───────────┘    └──────────────┘  │
└─────────────────────────────────────┘
          │
          ▼
    HTTP/WebSocket

The frontend handles:

• User interface rendering
• Client-side state management
• Real-time chat updates via WebSockets
• File uploads and document handling
• Code execution with Pyodide (browser-based Python)

2. Application Layer (Backend)

┌─────────────────────────────────────┐
│       FastAPI Application           │
│  ┌───────────┐    ┌──────────────┐  │
│  │  Routers  │    │  Middleware  │  │
│  └───────────┘    └──────────────┘  │
│  ┌───────────┐    ┌──────────────┐  │
│  │  Models   │    │   Utils      │  │
│  └───────────┘    └──────────────┘  │
└─────────────────────────────────────┘
          │
          ▼
   Database / Redis

Core Backend Components

Located in backend/open_webui/:

Main Application (main.py):

• FastAPI app initialization
• Middleware configuration (CORS, compression, sessions)
• WebSocket integration
• Startup/shutdown lifecycle management

Routers (routers/):

• auths.py - Authentication and authorization
• chats.py - Chat management
• models.py - Model configuration
• users.py - User management
• files.py - File handling
• retrieval.py - RAG (Retrieval-Augmented Generation)
• pipelines.py - Pipeline integration
• tools.py - Tool/function calling
• images.py - Image generation
• audio.py - Speech-to-text and text-to-speech

Models (models/):

• Database models using Peewee ORM
• User, Chat, Model, Function, File entities

Socket (socket/):

• WebSocket connection handling
• Real-time message streaming
• Collaborative features

3. Data Layer

┌──────────────┐  ┌──────────────┐  ┌──────────────┐
│   SQLite /   │  │    Redis     │  │   Vector DB  │
│  PostgreSQL  │  │   (Session)  │  │  (ChromaDB)  │
└──────────────┘  └──────────────┘  └──────────────┘

Primary Database:

• SQLite (default) or PostgreSQL
• Stores users, chats, models, configurations
• Encrypted SQLite option available

Session Storage:

• Redis (optional) for session management
• Required for horizontal scaling
• WebSocket state synchronization

Vector Database (RAG):

• ChromaDB (default)
• Alternatives: Qdrant, Milvus, PGVector, Elasticsearch, OpenSearch, Pinecone
• Document embeddings and similarity search

4. Integration Layer

┌─────────────────────────────────────┐
│        External Services            │
│  ┌───────────┐    ┌──────────────┐  │
│  │  Ollama   │    │   OpenAI     │  │
│  └───────────┘    └──────────────┘  │
│  ┌───────────┐    ┌──────────────┐  │
│  │ Pipelines │    │    Tools     │  │
│  └───────────┘    └──────────────┘  │
└─────────────────────────────────────┘

LLM Providers:

• Ollama integration (local models)
• OpenAI API compatibility (OpenAI, Anthropic, etc.)
• Custom model endpoints

Pipelines:

• Plugin framework for custom logic
• Python-based middleware
• Rate limiting, monitoring, filtering
• Repository: open-webui/pipelines

Tools & Functions:

• Native Python function calling
• MCP (Model Context Protocol) support
• Code editor with built-in support

Key Features Implementation

Authentication & Authorization

• JWT-based authentication
• Role-Based Access Control (RBAC)
• OAuth2/OIDC support (LDAP, SSO)
• SCIM 2.0 provisioning
• API key management

Location: backend/open_webui/routers/auths.py, utils/auth.py

Real-Time Communication

• Socket.IO for WebSocket connections
• Streaming chat completions
• Collaborative editing with Y.js
• Redis pub/sub for multi-instance deployments

Location: backend/open_webui/socket/main.py

RAG (Retrieval-Augmented Generation)

• Document ingestion and chunking
• Embedding generation (local or API)
• Vector similarity search
• Hybrid search with BM25
• Reranking support
• Multiple content extractors (Tika, Docling, Azure Document Intelligence)

Location: backend/open_webui/routers/retrieval.py, backend/open_webui/retrieval/

Code Execution

• Browser-based: Pyodide (Python in WebAssembly)
• Server-based: Jupyter integration
• Sandboxed execution environment
• Support for multiple languages via CodeMirror

Location: Frontend uses @pyscript/core, backend config in main.py

Image Generation & Editing

• OpenAI DALL-E integration
• Google Gemini support
• Local: ComfyUI, AUTOMATIC1111
• Prompt-based editing workflows

Location: backend/open_webui/routers/images.py

Audio Processing

Speech-to-Text:

• Local: Faster Whisper
• API: OpenAI, Azure, Deepgram, Mistral

Text-to-Speech:

• OpenAI TTS
• Azure Speech Services
• ElevenLabs
• Transformers (local)
• Web Speech API

Location: backend/open_webui/routers/audio.py

Scalability & Production

Horizontal Scaling

• Redis-backed sessions for stateless workers
• Multi-worker support (Uvicorn workers)
• Load balancer compatible
• WebSocket sticky sessions support

Observability

• OpenTelemetry integration
• Traces, metrics, and logs
• Audit logging middleware
• Custom monitoring hooks

Configuration: ENABLE_OTEL=true

Storage Options

File Storage:

• Local filesystem (default)
• S3-compatible storage
• Google Cloud Storage
• Azure Blob Storage

Database:

• SQLite with encryption
• PostgreSQL for production
• Connection pooling

Configuration Management

Configuration is managed through:

1. Environment Variables - Primary configuration method
2. AppConfig (config.py) - Runtime configuration state
3. Database - User-modifiable settings stored in DB
4. Redis - Distributed configuration for multi-instance

Location: backend/open_webui/config.py, backend/open_webui/env.py

Security Architecture

• Security headers middleware
• CORS configuration
• Content Security Policy
• API key restrictions
• Input validation with Pydantic
• SQL injection prevention (parameterized queries)
• XSS protection (DOMPurify on frontend)

Location: backend/open_webui/utils/security_headers.py

Request Flow Example

Chat Message Flow

1. User sends message (Frontend)
   ↓
2. WebSocket connection (Socket.IO)
   ↓
3. Authentication middleware
   ↓
4. Chat router processes request
   ↓
5. RAG retrieval (if documents attached)
   ↓
6. Tool/function calling (if enabled)
   ↓
7. LLM API call (Ollama/OpenAI)
   ↓
8. Stream response to client
   ↓
9. Save to database
   ↓
10. Render in UI (Frontend)

Development Workflow

1. Frontend changes: Edit Svelte components, hot-reload updates automatically
2. Backend changes: Modify Python files, restart server (or use --reload)
3. Database changes: Update models, migrations run on startup
4. API changes: Update routers, OpenAPI docs auto-generated at /docs

API Documentation

When running in development mode (ENV=dev):

• Swagger UI: http://localhost:8080/docs
• OpenAPI JSON: http://localhost:8080/openapi.json

Next Steps

• Set up your Local Development Environment
• Review Contributing Guidelines
• Explore Plugin Development