RagLeap Docs
Docs for ragleap-core v0.13.6 · latest
Configure

LLM providers

RagLeap Core is bring-your-own-key only there is no system-provided key for any provider. Set LLM_PROVIDER in .env to choose which one to use for the generation (chat) step. Embeddings are configured separately with EMBEDDING_PROVIDER (default gemini, using gemini-embedding-001; ollama, openai, mistral, together, openrouter, qwen, zhipu and custom are also supported). The vector size must match your embedding model (EMBEDDING_DIMENSIONS): empty tables are resized at startup, tables that already hold embeddings are never changed.

LLM_PROVIDER value Required env vars Notes
gemini (default) GEMINI_API_KEY Get a key at aistudio.google.com/apikey
anthropic ANTHROPIC_API_KEY, ANTHROPIC_MODEL (optional) Get a key at console.anthropic.com
openai OPENAI_API_KEY, OPENAI_MODEL
mistral MISTRAL_API_KEY, MISTRAL_MODEL
groq GROQ_API_KEY, GROQ_MODEL Free tier available
together TOGETHER_API_KEY, TOGETHER_MODEL
openrouter OPENROUTER_API_KEY, OPENROUTER_MODEL
ollama OLLAMA_MODEL (no API key needed) Self-hosted; requires Ollama running locally
deepseek DEEPSEEK_API_KEY, DEEPSEEK_MODEL
demonroute DEMONROUTE_API_KEY, DEMONROUTE_MODEL Hosted open-weight models through a third-party gateway. DemonRoute states that it does not retain prompts, but it forwards them to the upstream host that serves the model, and its privacy page does not describe that host's retention: avoid sensitive business data
cheaperinference CHEAPERINFERENCE_API_KEY, CHEAPERINFERENCE_MODEL OpenAI-compatible gateway to models from several labs, through a third-party service that forwards prompts to the upstream provider. Its retention policy is not reviewed by RagLeap: avoid sensitive business data
xai XAI_API_KEY, XAI_MODEL
cohere COHERE_API_KEY, COHERE_MODEL
perplexity PERPLEXITY_API_KEY, PERPLEXITY_MODEL
qwen QWEN_API_KEY, QWEN_MODEL
moonshot MOONSHOT_API_KEY, MOONSHOT_MODEL
zhipu ZHIPU_API_KEY, ZHIPU_MODEL
yi YI_API_KEY, YI_MODEL
baidu BAIDU_API_KEY, BAIDU_MODEL
minimax MINIMAX_API_KEY, MINIMAX_MODEL
custom CUSTOM_API_KEY, CUSTOM_MODEL, CUSTOM_BASE_URL Any OpenAI-compatible endpoint

Example, switching to Groq in .env:

LLM_PROVIDER=groq
GROQ_API_KEY=your-groq-key
GROQ_MODEL=llama-3.3-70b-versatile

Ollama in Docker: known gotchas

No internet, or a GPU? See the offline and GPU guide.

Running Ollama as a fallback/local provider from inside this app's Docker container (rather than bare metal) has real, verified setup steps beyond just setting OLLAMA_MODEL:

  1. host.docker.internal isn't reliable on plain Linux Docker Engine (unlike Docker Desktop) — it can resolve to the wrong bridge network's gateway. If Ollama connections fail with this alias, override OLLAMA_BASE_URL with the container's actual compose-network gateway IP directly, e.g. OLLAMA_BASE_URL=http://172.18.0.1:11434/v1 (find your real gateway with docker network inspect <network> | grep Gateway).
  2. Ollama binds to 127.0.0.1 by default, which a container can't reach even with the correct gateway IP. Override it to listen on all interfaces:
   mkdir -p /etc/systemd/system/ollama.service.d
   cat > /etc/systemd/system/ollama.service.d/override.conf << 'EOF'
   [Service]
   Environment="OLLAMA_HOST=0.0.0.0:11434"
   EOF
   systemctl daemon-reload && systemctl restart ollama
  1. The host firewall may silently drop container→host traffic even after the above — container-to-bridge-gateway traffic hits ufw's INPUT chain, not FORWARD/DOCKER-USER. Allow it explicitly: ufw allow from 172.18.0.0/16 to any port 11434 proto tcp (adjust the subnet to match your actual Docker network).

Generated from the ragleap-core v0.13.6 source. The repository is the source of truth and may be newer.