Universal API

The Portkey API and SDKs are integrated with all the popular LLM providers. This allows you to make a request to any hosted or private model with the same signature, and we take care of the request and response transformations automatically.

This universal API is powered by Portkey's battle-tested open-source AI Gateway. It converts all incoming requests to the OpenAI signature and is tuned to always return an OpenAI compliant response.

Let's try sending a completion request to OpenAI using Portkey's gateway

import Portkey from 'portkey-ai';

// Initialize the Portkey client
const portkey = new Portkey({
    apiKey: "PORTKEY_API_KEY",  // Replace with your Portkey API key
    virtualKey: "VIRTUAL_KEY"   // The OpenAI virtual key
});

// Function to generate a text completion
async function getTextCompletion() {
    const completion = await portkey.completions.create({
        prompt: "Say this is a test",
        model: "gpt-3.5-turbo-instruct",
    });

    return completion;
}

const response = await getTextCompletion();
console.log(response);

When we decide that we just want to try out Anthropic instead, we can just change this to Anthropic's virtual key and update the model.

import Portkey from 'portkey-ai';

// Initialize the Portkey client
const portkey = new Portkey({
    apiKey: "PORTKEY_API_KEY",  // Replace with your Portkey API key
    virtualKey: "VIRTUAL_KEY"   // The Anthropic virtual key
});

// Function to generate a text completion
async function getTextCompletion() {
    const completion = await portkey.completions.create({
        prompt: "Say this is a test",
        model: "claude-2",
        max_tokens: 250 //required parameter for Anthropic
    });

    return completion;
}

const response = await getTextCompletion();
console.log(response);

You can also use config objects for a whole host of gateway strategies.

The AI Gateway supports APIs outside plain text and chat completions as well. Check out the guides on Image Generation, Function Calling, Using Vision APIs and Working with Audio.

Last updated