RagLeap DocsRagLeap Core is bring-your-own-key only there is no system-provided key for any provider. Set LLM_PROVIDER in .env to choose which one to use for the generation (chat) step. Embeddings currently always use Gemini (gemini-embedding-001), regardless of LLM_PROVIDER.
LLM_PROVIDER value |
Required env vars | Notes |
|---|---|---|
gemini (default) |
GEMINI_API_KEY |
Get a key at aistudio.google.com/apikey |
anthropic |
ANTHROPIC_API_KEY, ANTHROPIC_MODEL (optional) |
Get a key at console.anthropic.com |
openai |
OPENAI_API_KEY, OPENAI_MODEL |
|
mistral |
MISTRAL_API_KEY, MISTRAL_MODEL |
|
groq |
GROQ_API_KEY, GROQ_MODEL |
Free tier available |
together |
TOGETHER_API_KEY, TOGETHER_MODEL |
|
openrouter |
OPENROUTER_API_KEY, OPENROUTER_MODEL |
|
ollama |
OLLAMA_MODEL (no API key needed) |
Self-hosted; requires Ollama running locally |
deepseek |
DEEPSEEK_API_KEY, DEEPSEEK_MODEL |
|
xai |
XAI_API_KEY, XAI_MODEL |
|
cohere |
COHERE_API_KEY, COHERE_MODEL |
|
perplexity |
PERPLEXITY_API_KEY, PERPLEXITY_MODEL |
|
qwen |
QWEN_API_KEY, QWEN_MODEL |
|
moonshot |
MOONSHOT_API_KEY, MOONSHOT_MODEL |
|
zhipu |
ZHIPU_API_KEY, ZHIPU_MODEL |
|
yi |
YI_API_KEY, YI_MODEL |
|
baidu |
BAIDU_API_KEY, BAIDU_MODEL |
|
minimax |
MINIMAX_API_KEY, MINIMAX_MODEL |
|
custom |
CUSTOM_API_KEY, CUSTOM_MODEL, CUSTOM_BASE_URL |
Any OpenAI-compatible endpoint |
Example, switching to Groq in .env:
LLM_PROVIDER=groq
GROQ_API_KEY=your-groq-key
GROQ_MODEL=llama-3.3-70b-versatile
Ollama in Docker: known gotchas
Running Ollama as a fallback/local provider from inside this app's Docker
container (rather than bare metal) has real, verified setup steps beyond
just setting OLLAMA_MODEL:
host.docker.internalisn't reliable on plain Linux Docker Engine (unlike Docker Desktop) — it can resolve to the wrong bridge network's gateway. If Ollama connections fail with this alias, overrideOLLAMA_BASE_URLwith the container's actual compose-network gateway IP directly, e.g.OLLAMA_BASE_URL=http://172.18.0.1:11434/v1(find your real gateway withdocker network inspect <network> | grep Gateway).- Ollama binds to
127.0.0.1by default, which a container can't reach even with the correct gateway IP. Override it to listen on all interfaces:
mkdir -p /etc/systemd/system/ollama.service.d cat > /etc/systemd/system/ollama.service.d/override.conf << 'EOF' [Service] Environment="OLLAMA_HOST=0.0.0.0:11434" EOF systemctl daemon-reload && systemctl restart ollama
- The host firewall may silently drop container→host traffic even
after the above — container-to-bridge-gateway traffic hits ufw's
INPUTchain, notFORWARD/DOCKER-USER. Allow it explicitly:ufw allow from 172.18.0.0/16 to any port 11434 proto tcp(adjust the subnet to match your actual Docker network).