What is Portkey?

Teams use Portkey to monitor and improve the cost, performance, and accuracy of their Gen AI apps.

It takes 2 mins to integrate and with that, it starts monitoring all of your LLM requests and makes your app resilient, secure, performant, and more accurate at the same time.

Here's a product walkthrough (3 mins):

Integrate in 3 Lines of Code

OpenAI Python

# pip install portkey-ai

from openai import OpenAI
from portkey_ai import PORTKEY_GATEWAY_URL, createHeaders

client = OpenAI(
    base_url=PORTKEY_GATEWAY_URL,
    default_headers=createHeaders(
        provider="openai", 
        api_key="PORTKEY_API_KEY"
    )
)

chat_complete = client.chat.completions.create(
    model="gpt-4",
    messages=[{"role": "user", "content": "Say this is a test"}],
)

print(chat_complete.choices[0].message.content)

OpenAI Node

// npm i portkey-ai

import OpenAI from 'openai';
import { PORTKEY_GATEWAY_URL, createHeaders } from 'portkey-ai'

const openai = new OpenAI({
  baseURL: PORTKEY_GATEWAY_URL,
  defaultHeaders: createHeaders({
    provider: "openai", 
    apiKey: "PORTKEY_API_KEY"
    })
});

async function main() {
  const chatCompletion = await openai.chat.completions.create({
    messages: [{ role: 'user', content: 'Say this is a test' }],
    model: 'gpt-3.5-turbo',
  });

  console.log(chatCompletion.choices);
}

main();

🚀 Quickstart

Setting up Portkey takes less than 2 mintues.

🤝 Integrations

Find the best integration for you with 200+ models across LLM providers and multiple frameworks.

✨ Product Features

Jump to the product section to learn more about the Portkey modules and the use cases they solve.

📔 API Reference

Head to the API reference and code samples for all Portkey functionality available through REST APIs and SDKs

While you're here, why not give us a star? It helps us a lot!

Languages Supported

Language	Supported Library
Javascript	portkey-node-sdk openai-node
Python	portkey-python-sdk openai-python
Go	go-openai
Java	openai-java
Rust	async-openai
Ruby	ruby-openai

AI Providers Supported

Portkey is multimodal by default - along with chat and text models, we also support audio, vision, and image generation models.

AI Provider	Status
OpenAI	fully supportedpublic
Anthropic	fully supportedpublic
Azure OpenAI	fully supportedpublic
Cohere	fully supportedpublic
Anyscale	fully supportedpublic
Google Palm	fully supportedpublic
Google Gemini	fully supportedpublic
Together AI	fully supportedpublic
Perplexity	fully supportedpublic
Mistral	fully supportedpublic
Stability	fully supportedpublic
Nomic	fully supportedpublic
AWS Bedrock	fully supportedpublic
Ollama	fully supportedpublic
AzureML	partially supported
BYOLLM	fully supportedpublic
Jina AI	fully supportedpublic
Fireworks AI	fully supportedpublic
LocalAI	partially supportedpublic
Predibase	fully supportedpublic
ZhipuAI (ChatGLM)	fully supportedpublic
Deepinfra	fully supportedpublic
Lingyi (01.ai)	fully supportedpublic
Openrouter	fully supportedpublic
Moonshot	fully supportedpublic
AI21	fully supportedpublic
Reka AI	fully supportedpublic
Workers AI	fully supportedpublic
Deepbricks	fully supportedpublic
SiliconFlow	fully supportedpublic

View all the supported integration guides.

Frameworks Supported

Framework	Status
Langchain	nativepythontypescript
Llamaindex	nativepythontypescript
Autogen	nativepython
Vercel	nativetypescript
Instructor	nativepythontypescript
Promptfoo	nativetypescript
CrewAI	nativepython