Image Generation

Portkey's AI gateway supports image generation capabilities that many foundational model providers offer. The most common use case is that of text-to-image where the user sends a prompt which the image model processes and returns an image.

The guide for vision models is available here.

Text-to-Image Usage

Portkey supports the OpenAI signature to make text-to-image requests.

NodeJS

import Portkey from 'portkey-ai';

// Initialize the Portkey client
const portkey = new Portkey({
    apiKey: "PORTKEY_API_KEY",  // Replace with your Portkey API key
    virtualKey: "VIRTUAL_KEY"   // Add your provider's virtual key
});

async function main() {
  const image = await portkey.images.generate({ 
    model: "dall-e-3", 
    prompt: "Lucy in the sky with diamonds" 
  });
  
  console.log(image.data);
}

main();

Python

from portkey_ai import Portkey
from IPython.display import display, Image

# Initialize the Portkey client
portkey = Portkey(
    api_key="PORTKEY_API_KEY",  # Replace with your Portkey API key
    virtual_key="VIRTUAL_KEY"   # Add your provider's virtual key
)

image = portkey.images.generate(
  model="dall-e-3",
  prompt="Lucy in the sky with diamonds"
)

# Display the image
display(Image(url=image.data[0].url))

OpenAI Node.js

import OpenAI from 'openai'; // We're using the v4 SDK
import { PORTKEY_GATEWAY_URL, createHeaders } from 'portkey-ai'

const openai = new OpenAI({
  apiKey: 'OPENAI_API_KEY', // defaults to process.env["OPENAI_API_KEY"],
  baseURL: PORTKEY_GATEWAY_URL,
  defaultHeaders: createHeaders({
    provider: "openai",
    apiKey: "PORTKEY_API_KEY" // defaults to process.env["PORTKEY_API_KEY"]
  })
});

async function main() {
  const image = await openai.images.generate({ 
    model: "dall-e-3", 
    prompt: "Lucy in the sky with diamonds" 
  });
  
  console.log(image.data);
}

main();

OpenAI Python

from openai import OpenAI
from portkey_ai import PORTKEY_GATEWAY_URL, createHeaders
from IPython.display import display, Image

client = OpenAI(
    api_key='OPENAI_API_KEY',
    base_url=PORTKEY_GATEWAY_URL,
    default_headers=createHeaders(
        provider="openai",
        api_key="PORTKEY_API_KEY"
    )
)

image = client.images.generate(
  model="dall-e-3",
  prompt="Lucy in the sky with diamonds",
  n=1,
  size="1024x1024"
)

# Display the image
display(Image(url=image.data[0].url))

REST

curl "https://api.portkey.ai/v1/chat/completions" \
  -H "Content-Type: application/json" \
  -H "x-portkey-api-key: $PORTKEY_API_KEY" \
  -H "x-portkey-virtual-key: openai-virtual-key" \
  -d '{
    "model": "dall-e-3",
    "prompt": "Lucy in the sky with diamonds"
  }'

API Reference

Create Image

On completion, the request will get logged in the logs UI where the image can be viewed.

(Note that providers may remove the hosted image after a period of time, so some logs might only contain the url)

Supported Providers and Models

The following providers are supported for image generation with more providers getting added soon. Please raise a request or a PR to add model or provider to the AI gateway.

Provider	Models	Functions
OpenAI	dall-e-2, dall-e-3	Create Image (text to image)
Azure OpenAI	dall-e-2, dall-e-3	Create Image (text to image)
Stability	stable-diffusion-v1-6, stable-diffusion-xl-1024-v1-0	Create Image (text to image)
Segmind	Refer here	Create Image (text to image)
Together AI (Coming Soon)
Monster API (Coming Soon)
Replicate (Coming Soon)

Cookbook

Here's a detailed cookbook on image generation using Portkey which demonstrates the use of multiple providers and routing between them through Configs.