December '23 Migration
Date: 8th Dec, 2023
This December, we’re pushing out some exciting new updates to Portkey’s SDKs, APIs, and Configs.
Portkey’s SDKs are upped to major version 1.0 bringing parity with the new OpenAI SDK structure and adding Portkey production features to it. We are also bringing native Langchain & Llamaindex integrations inside the SDK. This is a Breaking Change that Requires Migration.
Portkey’s APIsare upgraded with new endpoints, making it simpler to do /chat/completions
and /completions
calls and adding Portkey’s production functionalities to them.
This is a Breaking Change that Requires Migration.
Configs are upgraded to version 2.0, bringing nested gateway strategies with granular handling. For Configs saved in the Portkey dashboard, this is NOT a Breaking Change and we will Auto Migrate your old Configs. For Configs directly defined at the time of making a call, through the old SDKs or old APIS, they will fail on the new APIs & SDKs and require migration.
Compatibility & Deprecation List
List | Compatibility | Deprecation Date |
---|---|---|
API (Old)/v1/proxy /v1/complete /v1/chatComplete /v1/embed /v1/prompts/ID/generate | SDK (Old) SDK (New) Configs (Old) Configs (New) | Q2 ‘24 |
API (New) /v1 /v1/completions /v1/chat/completions /v1/embeddings /v1/prompts/ID/completions | SDK (Old) SDK (New) Configs (Old) Configs (New) | - |
SDK Version < 1 (Old) | API (Old) API (New) Configs (Old) Configs (new) | Q2 ‘24 |
SDK Version = 1 (New) | API (Old) API (New) Configs (Old) Configs (new) | - |
Configs 1.0 (Old) | API (Old) API (New) SDK (Old) SDK (new) === Configs saved through the Portkey UI will be auto migrated. | Q2 ‘24 |
Configs 2.0 (New) | API (Old) API (New) SDK (Old) SDK (new) | - |
We recommend upgrading to these new versions promptly to take full advantage of their capabilities. While your existing code will continue to work until the deprecation date around Q2 ‘24, transitioning now ensures you stay ahead of the curve and avoid any future service interruptions. Follow along with this guide!
Major Version Release of the SDK
Here’s What’s New:
- More extensible SDK that can be used with many more LLM providers
- Out-of-the-box support for streaming
- Completely follows OpenAI’s SDK signature reducing your technical debt
- Native support for Langchain & Llamaindex within the SDK (Python)
- Support for the Portkey Feedback endpoint
- Support for Portkey Prompt Templates
- Older SDK versions to be deprecated soon
Here’s What’s Changed:
FROM
import portkey
from portkey import Config, LLMOptions
portkey.config = Config(
mode="single",
llms=LLMOptions(provider="openai", api_key="OPENAI_API_KEY")
)
response = portkey.ChatCompletions.create(
model="gpt-4",
messages=[
{"role": "user","content": "Hello World!"}
]
)
TO
from portkey_ai import Portkey
portkey = Portkey(
api_key="PORTKEY_API_KEY",
Authorization="OPENAI_KEY"
)
response = portkey.chat.completions.create(
messages=[{'role': 'user', 'content': 'Say this is a test'}],
model='gpt-3.5-turbo'
)
print(response)
Installing the New SDK,
pip install -U portkey-ai
SDK
All-New APIs
Here’s What’s New:
- Introduced 3 new routes
/chat/completions
,/completions
, and/embeddings
- Simplified the headers:
x-portkey-mode
header is deprecated and replaced withx-portkey-provider
- Which takes values:
openai
,anyscale
,cohere,
palm
,azure-openai
, and more.
- Which takes values:
- New header
x-portkey-virtual-key
is introduced.
/complete
and/chatComplete
endpoints to be deprecated soon- Prompts endpoint
/prompts/$PROMPT_ID/generate
is upgraded to/prompts/$PROMPT_ID/completions
and the old route will be deprecated soon- We now support updating the model params on-the-fly (i.e. changing temperature etc at the time of making a call)
- Prompt response object on the
/completions
route is now fully OpenAI compliant
- New
/gateway
endpoint that lets you make calls to third-party LLM providers easily
Here’s What’s Changed
FROM
from openai import OpenAI
client = OpenAI(
api_key="OPENAI_API_KEY", # defaults to os.environ.get("OPENAI_API_KEY")
base_url="https://api.portkey.ai/v1/proxy",
default_headers= {
"x-portkey-api-key": "PORTKEY_API_KEY",
"x-portkey-mode": "proxy openai",
"Content-Type": "application/json"
}
)
chat_complete = client.chat.completions.create(
model="gpt-4",
messages=[{"role": "user", "content": "Say this is a test"}],
)
print(chat_complete.choices[0].message.content)
TO
# pip install -U portkey-ai
from openai import OpenAI
from portkey_ai import PORTKEY_GATEWAY_URL, createHeaders
client = OpenAI(
api_key="OPENAI_API_KEY", # defaults to os.environ.get("OPENAI_API_KEY")
base_url=PORTKEY_GATEWAY_URL,
default_headers=createHeaders(
provider="openai",
api_key="PORTKEY_API_KEY" # defaults to os.environ.get("PORTKEY_API_KEY")
)
)
chat_complete = client.chat.completions.create(
model="gpt-4",
messages=[{"role": "user", "content": "Say this is a test"}],
)
print(chat_complete.choices[0].message.content)
Chat
Completions
Gateway for Other API Endpoints
Simlarly, for Prompts
FROM
curl https://api.portkey.ai/v1/prompts/$PROMPT_ID/generate \
-H 'x-portkey-api-key: $PORTKEY_API_KEY' \
-H 'Content-Type: application/json' \
-d '{"variables": {"variable_a": "", "variable_b": ""}}'
TO
curl https://api.portkey.ai/v1/prompts/$PROMPT_ID/completions \
-H 'x-portkey-api-key: $PORTKEY_API_KEY' \
-H 'Content-Type: application/json' \
-d '{"variables": {"variable_a": "", "variable_b": ""}}'
Prompts
Configs 2.0
Here’s What’s New
- New concept of
strategy
instead of standalonemode
. You can now build bespoke gateway strategies and nest them in a single config. - You can also trigger a specific strategy on specific error codes.
- New concept of
targets
that replaceoptions
in the previous Config - If you are adding
virtual_key
to the target array, you no longer need to addprovider
,Portkey will pick up the Provider directly from the Virtual Key! - For Azure, only now pass the
virtual_key
- it takes care of all other Azure params like Deployment name, API version etc.
The Configs UI on Portkey app will autocomplete Configs ONLY in the new format now. All your existing Configs are auto migrated.
Here’s What’s Changed
FROM
{
"mode": "single",
"options": [
{
"provider": "openai",
"virtual_key": "open-4110dd",
}
]
}
TO
{
"strategy": {
"mode":"single"
},
"targets": [
{
"virtual_key": "open-4110dd"
}
]
}
Gateway Config Object
Support
Shoot ANY questions or queries you have on the migration to the Portkey team on our Discordand we will try to get back to you ASAP.
Was this page helpful?