AI SDKs and Frameworks

Keep your SDK's model calls and route them through Archestra for usage logs, spending limits, and Guardrails. The examples below use OpenAI Chat Completions.

Requirements

Configure an OpenAI provider key and a standard virtual key mapped to it.

Set OPENAI_API_KEY in your application's server environment to the virtual key. Replace http://localhost:9000/v1/openai with your reachable Archestra endpoint. A container's localhost refers to that container; use its host-network address, such as Docker Desktop's host.docker.internal, when Archestra runs on the host.

Session IDs for Guardrails

Use one stable session ID per conversation, including retries and follow-up turns.

Set CONVERSATION_ID to a conversation or thread ID your application stores. Reuse it for follow-up turns and retries; create a different ID for each new conversation. These examples assume the packages are installed in your server application.

Keep the client and key server-side, and scope the session headers to that conversation. A process-wide client with a fixed ID mixes unrelated conversations' policy state.

Vercel AI SDK

Install ai and @ai-sdk/openai, then configure the OpenAI provider with your proxy URL.

import { createOpenAI } from "@ai-sdk/openai";
import { generateText } from "ai";

const openai = createOpenAI({
  baseURL: "http://localhost:9000/v1/openai",
  apiKey: process.env.OPENAI_API_KEY,
  headers: { "X-Appa-Session-ID": process.env.CONVERSATION_ID! },
});

const result = await generateText({
  model: openai.chat("gpt-4o"),
  prompt: "Reply with connection verified.",
});
console.log(result.text);

Use .chat() to select Chat Completions explicitly. Keep the key and client in server-side code.

Pydantic AI

Install pydantic-ai, then pass a custom client to OpenAIProvider.

import os
from openai import AsyncOpenAI
from pydantic_ai import Agent
from pydantic_ai.models.openai import OpenAIChatModel
from pydantic_ai.providers.openai import OpenAIProvider

client = AsyncOpenAI(
    base_url="http://localhost:9000/v1/openai",
    api_key=os.environ["OPENAI_API_KEY"],
    default_headers={"X-Appa-Session-ID": os.environ["CONVERSATION_ID"]},
)
model = OpenAIChatModel("gpt-4o", provider=OpenAIProvider(openai_client=client))
agent = Agent(model)
result = agent.run_sync("Reply with connection verified.")
print(result.output)

OpenAIChatModel selects Chat Completions. Use OpenAIResponsesModel with the same provider for Responses.

Mastra

Install @mastra/core and configure the agent's custom model endpoint.

import { Agent } from "@mastra/core/agent";

const agent = new Agent({
  id: "connection-check",
  name: "Connection Check",
  instructions: "Answer the user's request.",
  model: {
    id: "openai/gpt-4o",
    url: "http://localhost:9000/v1/openai",
    apiKey: process.env.OPENAI_API_KEY,
    headers: { "X-Appa-Session-ID": process.env.CONVERSATION_ID! },
  },
});

const result = await agent.generate("Reply with connection verified.");
console.log(result.text);

Use the base URL, without /chat/completions. The openai/ prefix identifies the provider.

Verify the Connection

Find the test request in Logs → LLM Proxy with the expected model, virtual key, and timestamp.

For tool access, connect your SDK's MCP client to an MCP Gateway as well.