Bring Your Own Guardrails

Portkey supports bringing your own Guardrail using a custom webhook:

You can add Webhook as a Guardrail Check and setup any Guardrail Actions along with it. This is useful when you have an existing custom guardrail pipeline in your app and you are sending your LLM inputs or outputs to it for sync or async evaluation. Now, you can bring that onto Portkey's Gateway, make it production-grade, and enforce LLM behavior in real-time, easily.

How Custom Webhook Works

In the Guardrail check, Portkey expects your Webhook URL and any headers (including Authorization) you need to send.

Headers

Headers here should be a JSON object, like below:

{
    "Authorization":"MY_API_KEY",
    "Content-Type":"Application/JSON"
}

Portkey makes a POST request to your webhook URL and expects two objects in the response: verdict and data.

Webhook Response

Portkey expects two objects: verdict and data

Object	Type	Required?
verdict	boolean	Yes
data	any	No

Here's a sample webhook response:

{
  "verdict": true,
  "data": {
    "reason": "User passed all security checks",
    "score": 0.95,
    "additionalInfo": {
      "userStatus": "verified",
      "lastCheckTimestamp": "2024-08-21T14:30:00Z"
    }
  }
}

Check out the Webhook implementation here:

https://github.com/Portkey-AI/gateway/blob/main/plugins/default/webhook.ts

import { PluginContext, PluginHandler, PluginParameters } from '../types';
import { post } from '../utils';

export const handler: PluginHandler = async (
  context: PluginContext,
  parameters: PluginParameters
) => {
  let error = null;
  let verdict = false;
  let data = null;

  try {
    let url = parameters.webhookURL;
    let headers: Record<string, string> = parameters?.headers
      ? JSON.parse(parameters.headers)
      : {};
    ({ verdict, data } = await post(url, context, { headers }, 3000));
  } catch (e: any) {
    delete e.stack;
    error = e;
  }

  return { error, verdict, data };
};

Based on the verdict value, the Guardrail Check will PASS or FAIL, and will have subsequent impact on the Guardrail Actions you've set.

The webhook request automatically time out after 3 seconds - this can not be changed. So, if a webhook request times out, the Guardrail verdict will return PASS for that request.

Head over to the Portkey Discord community if you are building out custom webhooks and need any help!