LibreChat

LibreChat is an "enhanced ChatGPT clone". You can deploy it internally (or externally) to instantly spin up a Chat UI for any of your AI use cases.

Portkey natively integrates with LibreChat and makes your LibreChat deployments production-grade and reliable with our suite of features:

• Full-stack observability and tracing for all requests

• Interoperability across 250+ LLMs

• Built-in 50+ state-of-the-art guardrails

• Simple & semantic caching to save costs & time

• Conditional request routing with fallbacks, load-balancing, automatic retries, and more

• Continuous improvement based on user feedback

---

Integrate Portkey with LibreChat

Step 1. Create the docker-compose-override.yaml file, following the instructions here.

This file will point to the librechat.yaml file where we will configure our Portkey settings (in Step 3).

docker-compose.override.yml

services:
  api:
    volumes:
    - type: bind
      source: ./librechat.yaml
      target: /app/librechat.yaml

Step 2: Configure the .env file

Edit your existing .env file at the project root (if the file does not exist, copy the .env.example file and rename to .env). We will add:

.env

PORTKEY_API_KEY=YOUR_PORTKEY_API_KEY
PORTKEY_GATEWAY_URL=https://api.portkey.ai/v1

Step 3: Edit the librechat.yaml file, following the instructions here.

Here, you can either pass your Config (containing provider/model configurations) or direct provider Virtual key saved on Portkey.

LibreChat requires that the API key field is present. Since we don't need it for the Portkey integration, we can pass a dummy string for it.

librechat.yaml with Portkey Virtual Key

version: 1.1.4
cache: true
endpoints:
  custom:
    - name: "Portkey"
      apiKey: "dummy"
      baseURL: ${PORTKEY_GATEWAY_URL}
      headers:
        x-portkey-api-key: "${PORTKEY_API_KEY}"
        x-portkey-virtual-key: "PORTKEY_OPENAI_VIRTUAL_KEY"
      models:
        default: ["gpt-4o-mini"]
        fetch: true
      titleConvo: true
      titleModel: "current_model"
      summarize: false
      summaryModel: "current_model"
      forcePrompt: false
      modelDisplayLabel: "Portkey:OpenAI"

librechat.yaml with Portkey Config

version: 1.1.4
cache: true
endpoints:
  custom:
    - name: "Portkey"
      baseURL: ${PORTKEY_GATEWAY_URL}
      headers:
        x-portkey-api-key: "${PORTKEY_API_KEY}"
        x-portkey-config: "pc-libre-xxx"
      models:
        default: ["llama-3.2"]
        fetch: true
      titleConvo: true
      titleModel: "current_model"
      summarize: false
      summaryModel: "current_model"
      forcePrompt: false
      modelDisplayLabel: "Portkey:Llama"

---

Support for the LibreChat Integration

If you face any issues in integrating Portkey with LibreChat, please ping the Portkey team on our commuinty forum here.