The Portkey SDK client enables various features of Portkey in an easy to use config-as-code paradigm.
Install the Portkey SDK
Add the Portkey SDK to your application to interact with Portkey's gateway.
npminstall--saveportkey-ai
pipinstallportkey-ai
Export Portkey API Key
export PORTKEY_API_KEY=""
Basic Client Setup
The basic Portkey SDK client needs 2 required parameters
The Portkey Account's API key to authenticate all your requests
The virtual key of the AI provider you want to use
OR
The config being used
This is achieved through headers when you're using the REST API.
For example,
import Portkey from'portkey-ai';// Construct a client with a virtual keyconstportkey=newPortkey({ apiKey:"PORTKEY_API_KEY", virtualKey:"VIRTUAL_KEY"})// Construct a client with a config idconstportkey=newPortkey({ apiKey:"PORTKEY_API_KEY", config:"cf-***"// Supports a string config slug or a config object})
Find more info on what's available through configs here.
from portkey_ai import Portkey# Construct a client with a virtual keyportkey =Portkey( api_key="PORTKEY_API_KEY", virtual_key="VIRTUAL_KEY")# Construct a client with provider and provider API keyportkey =Portkey( api_key="PORTKEY_API_KEY", config="cf-***"# Supports a string config slug or a config object)
Find more info on what's available through configs here.
Find more info on what's available through configs here.
Making a Request
You can then use the client to make completion and other calls like this
constchatCompletion=awaitportkey.chat.completions.create({ messages: [{ role:'user', content:'Say this is a test' }], model:'gpt-3.5-turbo',});console.log(chatCompletion.choices);
completion = portkey.chat.completions.create( messages = [{ "role": 'user', "content": 'Say this is a test' }], model ='gpt-3.5-turbo')
Passing Trace ID or Metadata
You can choose to override the configuration in individual requests as well and send trace id or metadata along with each request.
constchatCompletion=awaitportkey.chat.completions.create({ messages: [{ role:'user', content:'Say this is a test' }], model:'gpt-3.5-turbo',}, { traceId:"39e2a60c-b47c-45d8", metadata: {"_user":"432erf6"}});console.log(chatCompletion.choices);
completion = portkey.with_options( trace_id ="TRACE_ID", metadata = {"_user": "USER_IDENTIFIER"}).chat.completions.create( messages = [{ "role": 'user', "content": 'Say this is a test' }], model ='gpt-3.5-turbo')
Async Usage
Portkey's Python SDK supports Async usage - just use AsyncPortkey instead of Portkey with await:
import asynciofrom portkey_ai import AsyncPortkeyportkey =AsyncPortkey( api_key="PORTKEY_API_KEY", virtual_key="VIRTUAL_KEY")asyncdefmain(): chat_completion =await portkey.chat.completions.create( messages=[{'role': 'user', 'content': 'Say this is a test'}], model='gpt-4' )print(chat_completion)asyncio.run(main())
Parameters
Following are the parameter keys that you can add while creating the Portkey client.
Keeping in tune with the most popular language conventions, we use:
camelCase for Javascript keys
snake_case for Python keys
hyphenated-keys for the headers
Parameter
Type
Key
API Key
Your Portkey account's API Key.
stringrequired
apiKey
Virtual Key
The virtual key created from Portkey's vault for a specific provider
Trace ID
An ID you can pass to refer to 1 or more requests later on. Generated automatically for every request, if not sent.
string
traceID
Metadata
Any metadata to attach to the requests. These can be filtered later on in the analytics and log dashboards
Can contain _prompt, _user, _organisation, or _environment that are special metadata types in Portkey.
You can also send any other keys as part of this object.
object
metadata
Cache Force Refresh
Force refresh the cache for your request by making a new call and storing that value.
boolean
cacheForceRefresh
Custom Host
Route to locally or privately hosted model by configuring the API URL with custom host
string
customHost
Forward Headers
Forward sensitive headers directly to your model's API without any processing from Portkey.
array of string
forwardHeaders
Parameter
Type
Key
API Key
Your Portkey account's API Key.
stringrequired
api_key
Virtual Key
The virtual key created from Portkey's vault for a specific provider
Trace ID
An ID you can pass to refer to 1 or more requests later on. Generated automatically for every request, if not sent.
string
trace_id
Metadata
Any metadata to attach to the requests. These can be filtered later on in the analytics and log dashboards
Can contain _prompt, _user, _organisation, or _environment that are special metadata types in Portkey.
You can also send any other keys as part of this object.
object
metadata
Cache Force Refresh
Force refresh the cache for your request by making a new call and storing that value.
boolean
cache_force_refresh
Custom Host
Route to locally or privately hosted model by configuring the API URL with custom host
string
custom_host
Forward Headers
Forward sensitive headers directly to your model's API without any processing from Portkey.
array of string
forward_headers
Parameter
Type
Header Key
API Key
Your Portkey account's API Key.
stringrequired
x-portkey-api-key
Virtual Key
The virtual key created from Portkey's vault for a specific provider
Trace ID
An ID you can pass to refer to 1 or more requests later on. Generated automatically for every request, if not sent.
string
x-portkey-trace-id
Metadata
Any metadata to attach to the requests. These can be filtered later on in the analytics and log dashboards
Can contain _prompt, _user, _organisation, or _environment that are special metadata types in Portkey.
You can also send any other keys as part of this object.
string
x-portkey-metadata
Cache Force Refresh
Force refresh the cache for your request by making a new call and storing that value.
boolean
x-portkey-cache-force-refresh
Custom Host
Route to locally or privately hosted model by configuring the API URL with custom host
string
x-portkey-custom-host
Forward Headers
Forward sensitive headers directly to your model's API without any processing from Portkey.