RagLeap Core is bring-your-own-key only there is no system-provided key for any provider. Set LLM_PROVIDER in .env to choose which one to use for the generation (chat) step. Embeddings are configured separately with EMBEDDING_PROVIDER (default gemini, using gemini-embedding-001; ollama, openai, mistral, together, openrouter, qwen, zhipu and custom are also supported). The vector size must match your embedding model (EMBEDDING_DIMENSIONS): empty tables are resized at startup, tables that already hold embeddings are never changed.
LLM_PROVIDER value |
Required env vars | Notes |
|---|---|---|
gemini (default) |
GEMINI_API_KEY |
Get a key at aistudio.google.com/apikey |
anthropic |
ANTHROPIC_API_KEY, ANTHROPIC_MODEL (optional) |
Get a key at console.anthropic.com |
openai |
OPENAI_API_KEY, OPENAI_MODEL |
|
mistral |
MISTRAL_API_KEY, MISTRAL_MODEL |
|
groq |
GROQ_API_KEY, GROQ_MODEL |
Free tier available |
together |
TOGETHER_API_KEY, TOGETHER_MODEL |
|
openrouter |
OPENROUTER_API_KEY, OPENROUTER_MODEL |
|
ollama |
OLLAMA_MODEL (no API key needed) |
Self-hosted; requires Ollama running locally |
deepseek |
DEEPSEEK_API_KEY, DEEPSEEK_MODEL |
|
demonroute |
DEMONROUTE_API_KEY, DEMONROUTE_MODEL |
Hosted open-weight models through a third-party gateway. DemonRoute states that it does not retain prompts, but it forwards them to the upstream host that serves the model, and its privacy page does not describe that host's retention: avoid sensitive business data |
cheaperinference |
CHEAPERINFERENCE_API_KEY, CHEAPERINFERENCE_MODEL |
OpenAI-compatible gateway to models from several labs, through a third-party service that forwards prompts to the upstream provider. Its retention policy is not reviewed by RagLeap: avoid sensitive business data |
xai |
XAI_API_KEY, XAI_MODEL |
|
cohere |
COHERE_API_KEY, COHERE_MODEL |
|
perplexity |
PERPLEXITY_API_KEY, PERPLEXITY_MODEL |
|
qwen |
QWEN_API_KEY, QWEN_MODEL |
|
moonshot |
MOONSHOT_API_KEY, MOONSHOT_MODEL |
|
zhipu |
ZHIPU_API_KEY, ZHIPU_MODEL |
|
yi |
YI_API_KEY, YI_MODEL |
|
baidu |
BAIDU_API_KEY, BAIDU_MODEL |
|
minimax |
MINIMAX_API_KEY, MINIMAX_MODEL |
|
custom |
CUSTOM_API_KEY, CUSTOM_MODEL, CUSTOM_BASE_URL |
Any OpenAI-compatible endpoint |
Example, switching to Groq in .env:
LLM_PROVIDER=groq
GROQ_API_KEY=your-groq-key
GROQ_MODEL=llama-3.3-70b-versatile
Ollama in Docker: known gotchas
No internet, or a GPU? See the offline and GPU guide.
Running Ollama as a fallback/local provider from inside this app's Docker
container (rather than bare metal) has real, verified setup steps beyond
just setting OLLAMA_MODEL:
host.docker.internalisn't reliable on plain Linux Docker Engine (unlike Docker Desktop) — it can resolve to the wrong bridge network's gateway. If Ollama connections fail with this alias, overrideOLLAMA_BASE_URLwith the container's actual compose-network gateway IP directly, e.g.OLLAMA_BASE_URL=http://172.18.0.1:11434/v1(find your real gateway withdocker network inspect <network> | grep Gateway).- Ollama binds to
127.0.0.1by default, which a container can't reach even with the correct gateway IP. Override it to listen on all interfaces:
mkdir -p /etc/systemd/system/ollama.service.d cat > /etc/systemd/system/ollama.service.d/override.conf << 'EOF' [Service] Environment="OLLAMA_HOST=0.0.0.0:11434" EOF systemctl daemon-reload && systemctl restart ollama
- The host firewall may silently drop container→host traffic even
after the above — container-to-bridge-gateway traffic hits ufw's
INPUTchain, notFORWARD/DOCKER-USER. Allow it explicitly:ufw allow from 172.18.0.0/16 to any port 11434 proto tcp(adjust the subnet to match your actual Docker network).