Create Batch

Example Usage

OpenAI Node

import OpenAI from 'openai';
import { PORTKEY_GATEWAY_URL, createHeaders } from 'portkey-ai'

const client = new OpenAI({
  baseURL: PORTKEY_GATEWAY_URL,
  defaultHeaders: createHeaders({
    apiKey: "PORTKEY_API_KEY",
    virtualKey: "PROVIDER_VIRTUAL_KEY"
  })
});

async function main() {
  const batch = await client.batches.create({
    input_file_id: "file-abc123",
    endpoint: "/v1/chat/completions",
    completion_window: "24h"
  });

  console.log(batch);
}

main();

OpenAI Python

from openai import OpenAI
from portkey_ai import PORTKEY_GATEWAY_URL, createHeaders

client = OpenAI(
    base_url=PORTKEY_GATEWAY_URL,
    default_headers=createHeaders(
        api_key="PORTKEY_API_KEY",
        virtual_key="PROVIDER_VIRTUAL_KEY"
    )
)

batch = client.batches.create(
  input_file_id="file-abc123",
  endpoint="/v1/chat/completions",
  completion_window="24h"
)

Portkey Node

import Portkey from 'portkey-ai';

const client = new Portkey({
  apiKey: 'PORTKEY_API_KEY',
  virtualKey: 'PROVIDER_VIRTUAL_KEY'
});

async function main() {
  const batch = await client.batches.create({
    input_file_id: "file-abc123",
    endpoint: "/v1/chat/completions",
    completion_window: "24h"
  });

  console.log(batch);
}

main();

Portkey Python

from portkey_ai import Portkey

client = Portkey(
  api_key = "PORTKEY_API_KEY",
  virtual_key = "PROVIDER_VIRTUAL_KEY"
)

batch = client.batches.create(
  input_file_id="file-abc123",
  endpoint="/v1/chat/completions",
  completion_window="24h"
)

CURL

curl https://api.portkey.ai/v1/batches \
  -H "x-portkey-api-key: $PORTKEY_API_KEY" \
  -H "x-portkey-virtual-key: $PORTKEY_PROVIDER_VIRTUAL_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "input_file_id": "file-abc123",
    "endpoint": "/v1/chat/completions",
    "completion_window": "24h"
  }'

Creates and executes a batch from an uploaded file of requests

POSThttps://api.portkey.ai/v1/batches

Body

application/json

input_file_id*string

The ID of an uploaded file that contains requests for the new batch.

See upload file for how to upload a file.

Your input file must be formatted as a JSONL file, and must be uploaded with the purpose batch. The file can contain up to 50,000 requests, and can be up to 100 MB in size.

endpoint*enum

The endpoint to be used for all requests in the batch. Currently /v1/chat/completions, /v1/embeddings, and /v1/completions are supported. Note that /v1/embeddings batches are also restricted to a maximum of 50,000 embedding inputs across all requests in the batch.

/v1/chat/completions/v1/embeddings/v1/completions

completion_window*enum

The time frame within which the batch should be processed. Currently only 24h is supported.

24h

metadatanullable object

Optional custom metadata for the batch.

Response

200

Batch created successfully.

Body

application/json

id*string

object*enum

The object type, which is always batch.

batch

endpoint*string

The Portkey API endpoint used by the batch.

errorsobject

input_file_id*string

The ID of the input file for the batch.

completion_window*string

The time frame within which the batch should be processed.

status*enum

The current status of the batch.

validatingfailedin_progressfinalizingcompletedexpiredcancellingcancelled

output_file_idstring

The ID of the file containing the outputs of successfully executed requests.

error_file_idstring

The ID of the file containing the outputs of requests with errors.

created_at*integer

The Unix timestamp (in seconds) for when the batch was created.

in_progress_atinteger

The Unix timestamp (in seconds) for when the batch started processing.

expires_atinteger

The Unix timestamp (in seconds) for when the batch will expire.

finalizing_atinteger

The Unix timestamp (in seconds) for when the batch started finalizing.

completed_atinteger

The Unix timestamp (in seconds) for when the batch was completed.

failed_atinteger

The Unix timestamp (in seconds) for when the batch failed.

expired_atinteger

The Unix timestamp (in seconds) for when the batch expired.

cancelling_atinteger

The Unix timestamp (in seconds) for when the batch started cancelling.

cancelled_atinteger

The Unix timestamp (in seconds) for when the batch was cancelled.

request_countsobject

The request counts for different statuses within the batch.

metadatanullable object

Set of 16 key-value pairs that can be attached to an object. This can be useful for storing additional information about the object in a structured format. Keys can be a maximum of 64 characters long and values can be a maxium of 512 characters long.

Response

{
  "id": "text",
  "object": "batch",
  "endpoint": "text",
  "errors": {
    "object": "text",
    "data": [
      {
        "code": "text",
        "message": "text",
        "param": "text"
      }
    ]
  },
  "input_file_id": "text",
  "completion_window": "text",
  "status": "validating",
  "output_file_id": "text",
  "error_file_id": "text",
  "request_counts": {}
}