Speech-to-Text

Portkey's AI gateway supports STT models like Whisper by OpenAI.

Transcription & Translation Usage

Portkey supports both Transcription and Translation methods for STT models and follows the OpenAI signature where you can send the file (in flac, mp3, mp4, mpeg, mpga, m4a, ogg, wav, or webm formats) as part of the API request.

Here's an example:

OpenAI NodeJS

import fs from "fs";
import OpenAI from "openai";
import { PORTKEY_GATEWAY_URL, createHeaders } from 'portkey-ai'

const openai = new OpenAI({
  baseURL: PORTKEY_GATEWAY_URL,
  defaultHeaders: createHeaders({
    apiKey: "PORTKEY_API_KEY",
    virtualKey: "OPENAI_VIRTUAL_KEY"
  })
});

// Transcription

async function transcribe() {
  const transcription = await openai.audio.transcriptions.create({
    file: fs.createReadStream("/path/to/file.mp3"),
    model: "whisper-1",
  });

  console.log(transcription.text);
}
transcribe();

// Translation

async function translate() {
    const translation = await openai.audio.translations.create({
        file: fs.createReadStream("/path/to/file.mp3"),
        model: "whisper-1",
    });
    console.log(translation.text);
}
translate();

OpenAI Python

from openai import OpenAI
from portkey_ai import PORTKEY_GATEWAY_URL, createHeaders

client = OpenAI(
    base_url=PORTKEY_GATEWAY_URL,
    default_headers=createHeaders(
        api_key="PORTKEY_API_KEY",
        virtual_key="OPENAI_VIRTUAL_KEY"
    )
)

audio_file= open("/path/to/file.mp3", "rb")

# Transcription

transcription = client.audio.transcriptions.create(
  model="whisper-1", 
  file=audio_file
)
print(transcription.text)

# Translation

translation = client.audio.translations.create(
  model="whisper-1", 
  file=audio_file
)
print(translation.text)

REST

For Transcriptions:

curl "https://api.portkey.ai/v1/audio/transcriptions" \
  -H "x-portkey-api-key: $PORTKEY_API_KEY" \
  -H "x-portkey-provider: openai" \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -H 'Content-Type: multipart/form-data' \
  --form file=@/path/to/file/audio.mp3 \
  --form model=whisper-1

For Translations:

curl "https://api.portkey.ai/v1/audio/translations" \
  -H "x-portkey-api-key: $PORTKEY_API_KEY" \
  -H "x-portkey-provider: openai" \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -H 'Content-Type: multipart/form-data' \
  --form file=@/path/to/file/audio.mp3 \
  --form model=whisper-1

On completion, the request will get logged in the logs UI where you can see trasncribed or translated text, along with the cost and latency incurred.

Supported Providers and Models

The following providers are supported for speech-to-text with more providers getting added soon. Please raise a request or a PR to add model or provider to the AI gateway.

Provider	Models	Functions
OpenAI	whisper-1	Transcription Translation