Channels & integrations

Voice Channel (Twilio)

RagLeap Core includes a real-time voice channel: Twilio Media Streams connects via WebSocket, your speech is transcribed with OpenAI Whisper, answered by the core RAG pipeline, and spoken back with OpenAI TTS. The voice-activity detection and echo-suppression logic is carried over from a production system tuned against real call traffic.

Runs as a separate service on port 8765 (see docker-compose.yml), since Twilio's real-time audio protocol needs a raw WebSocket server, not an HTTP route.

Setup:

1. Set OPENAI_API_KEY in .env (used for both Whisper STT and TTS in v1)
2. Optionally set VOICE_BOT_NAME, VOICE_GREETING, VOICE_TTS_VOICE
3. Point a Twilio phone number's <Connect><Stream> TwiML at wss://your-domain.com:8765

Honest status: the WebSocket server, Twilio event protocol handling, and error handling are verified working. The full Whisper/TTS round-trip has not yet been live-tested end-to-end (requires OpenAI API credits). If you try it and hit issues, please open one — this is exactly the kind of real-world testing this project needs.

Known limitations, carried over from production and not yet fixed here:

• Only OpenAI Whisper (STT) and OpenAI TTS are supported in v1 — Deepgram and ElevenLabs (multi-language support) are good-first-issue candidates
• Non-English TTS quality varies since OpenAI's TTS voices are English-tuned
• Typical round-trip latency in production was 6-8 seconds

Integrations

flowchart TD
    subgraph Connectors["10 connectors, one shared interface"]
        C1[MySQL] --- C2[PostgreSQL] --- C3[MongoDB]
        C4[REST API] --- C5[Salesforce] --- C6[HubSpot]
        C7[Shopify] --- C8[Google Sheets] --- C9[Stripe]
        C10[CSV Upload]
    end
    Connectors --> Svc["RealTimeExternalDataService<br/>real-time query, 5-min cache, no separate sync step"]
    Owner["Owner configures an action:<br/>trigger phrase or auto-match on SQL/field names<br/>+ a query template (SELECT / UPDATE / INSERT / DELETE)"] --> Svc
    Svc --> Match["match_and_execute_action(workspace_id, user_message, user_identifier)<br/>extracts order_id / email / phone from the message,<br/>substitutes into the template, runs the query"]
    Match --> Chat["Chat channels (WhatsApp/Telegram/Discord/Web)<br/>api/personal_bot_views.py"]
    Match --> Voice["Voice channel<br/>memory/voice_views.py::twilio_voice_speech<br/>wired in 2026-08"]
    Chat --> RAGCtx["Result injected as context<br/>into the RAG prompt"]
    Voice --> RAGCtx
    RAGCtx --> Answer["AI answers with real account/order/appointment<br/>data, not just document knowledge"]

Loading

Verified live, read from api/addon_realtime.py's RealTimeExternalDataService. Two things worth being direct about: match_and_execute_action genuinely supports write queries (UPDATE/INSERT/DELETE), not just read-only lookups — owner-configured, so the safety boundary is whatever SQL the owner writes into the template, not something the framework restricts on its own. And until 2026-08, this action-matching step only ran on chat channels; voice calls had no equivalent, which is the gap closed in the Voice Channel Routing diagram above.

RagLeap Core connects to external databases and business tools, syncing per-user context to personalize RAG responses. Nine connectors are included: MySQL, PostgreSQL, MongoDB, generic REST APIs, Salesforce, HubSpot, Shopify, Google Sheets, and Stripe.

Every CRM/SaaS connector uses credentials you provide directly — a username/password, a private-app token, an admin API token, a service-account JSON file, or a secret key, depending on the service. None require registering an OAuth app; nothing here depends on RagLeap owning any third-party developer account.

Credentials are encrypted at rest (Fernet/AES-128) before being stored.

Setup:

1. Generate an encryption key: python3 -c "from cryptography.fernet import Fernet; print(Fernet.generate_key().decode())"
2. Set ADDON_ENCRYPTION_KEY in .env to that value
3. Install the SDK for the connector(s) you want (each is optional — see requirements.txt)
4. Create a data source: POST /integrations with name, source_type, and the relevant credential fields
5. Test it: POST /integrations/{id}/test
6. Sync it: POST /integrations/{id}/sync

Honest status: verified end-to-end against a real public API — connection testing, syncing, correct identifier-field matching, and credential encryption (checked as actual ciphertext in the database, not just assumed) all confirmed working.

Known limitations:

• CSV Upload, Snowflake, BigQuery, WooCommerce, Airtable, Notion, Razorpay, Slack, and Gmail are good-first-issue candidates for anyone wanting to add one
• Sync is on-demand only (POST /integrations/{id}/sync) — no scheduled background sync yet, though the schema tracks sync_interval_minutes for a future Celery-beat-equivalent
• Synced context isn't automatically injected into chat responses yet — each channel adapter would need to know its own user's identifier first, which is a reasonable next contribution