Supported LLM Providers

Overview

Archestra Platform acts as a security proxy between your AI applications and LLM providers. It currently supports the following LLM providers.

OpenAI-Compatible Model Router

The model router exposes one OpenAI-compatible interface for models across configured providers.

Supported Model Router APIs

• Responses API (/responses) for text requests across model-router-compatible providers
• Chat Completions API (/chat/completions) for text chat requests across model-router-compatible providers
• Models API (/models) for provider-qualified chat and embedding model IDs
• Embeddings API (/embeddings) for embedding models across supported providers

Embedding models use the same provider-qualified IDs as chat models (for example openai:text-embedding-3-small or gemini:gemini-embedding-001). Anthropic, Bedrock, Cohere, and GitHub Copilot have no compatible embeddings API and return 501 Not Implemented.

GitHub Copilot Through the Model Router

GitHub Copilot is routable, with one difference from every other provider: it serves each model over a single API. The router reads which one from the model and sends the request there. Codex and GPT-5.x models go to the Responses API; the rest go to Chat Completions.

Requesting a Responses-only model on /chat/completions returns 400 Bad Request naming the endpoint to use instead.

A Copilot key is tied to one GitHub account, so it is routable only through your own personal virtual key. That key can hold your other providers too, so one router endpoint reaches all of them. See GitHub Copilot.

Model Router Connection Details

• Base URL: http://localhost:9000/v1/model-router/{llm-proxy-id}
• Authentication: Pass either a mapped virtual API key or an LLM OAuth client access token in the Authorization header as Bearer <key>. Use virtual keys for generic LLM clients and OAuth client access tokens for backend services that can perform OAuth client credentials. See Authentication.

List Models

Call GET /v1/model-router/{llm-proxy-id}/models to list OpenAI-compatible model objects. Model IDs are returned as <provider>:<model-id> and only include providers mapped to the virtual key or LLM OAuth client used for the request. The list includes chat models and embedding models. Models restricted to teams are omitted unless the request is attributed to a user in one of those teams. See Authentication for configuration details.

Model Resolution

Use provider-qualified model IDs from /models for deterministic routing, for example openai:gpt-5.4, anthropic:claude-opus-4-6-20250918, groq:llama-3.1-8b-instant, or bedrock:amazon.nova-pro-v1:0.

The prefix before : is the provider. The value after : is the provider's native model ID, so provider model IDs can still contain slashes or colons.

The /models response includes model-router-compatible text models for the providers mapped on the virtual key. Providers that use native request formats, including Anthropic, Bedrock, Gemini, and Cohere, are translated between OpenAI request/response formats and provider-native formats before forwarding.

Model Router translation forwards inline non-text content where the provider's native format supports it: Gemini (base64 data URL images, audio, and files), Anthropic (base64 data URL images and PDF files, plus http(s) image URLs), Cohere (images via base64 data URI or web URL in user messages), and Bedrock (base64 data URL images). Anthropic also forwards images returned inside tool results. Content the provider format cannot represent is dropped — for example http(s) image URLs to Gemini (its fileData accepts only Files API or gs:// URIs), audio to Anthropic, and non-text content in Gemini and Cohere tool results.

OpenAI

Supported OpenAI APIs

• Chat Completions API (/chat/completions)
• Responses API (/responses)
• Embeddings API (/embeddings)

OpenAI Connection Details

• Base URL: http://localhost:9000/v1/openai/{profile-id}
• Authentication: Pass your OpenAI API key in the Authorization header as Bearer <your-api-key>

Important Notes

• Use Responses API for new clients: OpenAI recommends /responses for new integrations. Chat Completions remains supported for existing clients.
• Streaming: OpenAI streaming responses require your cloud provider's load balancer to support long-lived connections. See Cloud Provider Configuration for more details.

ChatGPT Subscription (Codex)

Reuse a ChatGPT/Codex subscription for chat instead of a metered API key. Add an OpenAI provider key, pick the ChatGPT Subscription tab, and use Sign in with ChatGPT to connect the account that holds your subscription.

First turn on device code authorization for the account, in ChatGPT → Settings → Security → Enable device code authorization for Codex. It is off by default, and ChatGPT blocks the approval step until you enable it.

These keys are per-user and personal-only: each person connects their own ChatGPT account. Requests are billed to the subscription. An agent set up with a subscription key always runs on the chatting user's own subscription — never someone else's. Users without a connected account get a sign-in prompt in chat.

Anthropic

Supported Anthropic APIs

• Messages API (/messages)

Anthropic Connection Details

• Base URL: http://localhost:9000/v1/anthropic/{profile-id}
• Authentication: Pass your Anthropic API key in the x-api-key header
• Messages path: POST /v1/anthropic/{profile-id}/v1/messages

Anthropic on Microsoft Foundry

Claude models deployed in Microsoft Foundry use the Anthropic Messages API at https://<resource>.services.ai.azure.com/anthropic. Set ARCHESTRA_ANTHROPIC_BASE_URL to that /anthropic base URL. For keyless Microsoft Entra ID authentication, also set ARCHESTRA_ANTHROPIC_AZURE_FOUNDRY_ENTRA_ID_ENABLED=true; Archestra sends a bearer token scoped to https://ai.azure.com/.default.

Claude Foundry deployments must exist in Azure before requests will work. Use the deployed Claude model name in the Anthropic model field. Microsoft lists extra Claude prerequisites: a paid eligible Azure subscription, a supported region such as East US2 or Sweden Central, Azure Marketplace access for partner models, permission to subscribe to model offerings, and Contributor or Owner role on the resource group.

Azure requires Anthropic deployment metadata when creating Claude deployments: industry, organizationName, and countryCode. In Azure CLI this may require an ARM REST deployment call with properties.modelProviderData.

See Microsoft's Claude on Foundry guide for the Azure endpoint and authentication details.

Workload Identity Federation (keyless)

Archestra can authenticate to the Anthropic API without a static API key using Workload Identity Federation: it exchanges a short-lived OIDC identity token from your identity provider (Kubernetes, AWS, GCP, Entra ID, GitHub Actions, and others) for an Anthropic access token and sends it as Authorization: Bearer upstream. Tokens are cached and refreshed automatically before expiry.

Configure a federation issuer, service account, and federation rule in the Claude Console (Settings → Workload identity), then set the ARCHESTRA_ANTHROPIC_* WIF environment variables — see Environment Variables in the deployment docs. When configured, Archestra creates an "Anthropic Workload Identity Federation" system key automatically and syncs the available Claude models; users can also create Anthropic provider keys without entering an API key.

Note: the SDK-standard ANTHROPIC_API_KEY / ANTHROPIC_AUTH_TOKEN environment variables take precedence over federation if present in the backend environment, matching Anthropic's documented credential precedence.

Google Gemini

Archestra supports both the Google AI Studio (Gemini Developer API) and Vertex AI implementations of the Gemini API.

Supported Gemini APIs

• Generate Content API (:generateContent)
• Stream Generate Content API (:streamGenerateContent)
• Embeddings API (/embeddings) - OpenAI-compatible

Gemini Connection Details

• Base URL: http://localhost:9000/v1/gemini/{profile-id}/v1beta
• Authentication:
  • Google AI Studio (default): Pass your Gemini API key in the x-goog-api-key header
  • Vertex AI: No API key required from clients - uses server-side Application Default Credentials (ADC)

Using Vertex AI

To use Vertex AI instead of Google AI Studio, configure these environment variables:

GKE with Workload Identity (Recommended)

For GKE deployments, we recommend using Workload Identity which provides secure, keyless authentication. This eliminates the need for service account JSON key files.

Setup steps:

1. Create a GCP service account with Vertex AI permissions:

   gcloud iam service-accounts create archestra-vertex-ai \
     --display-name="Archestra Vertex AI"
   
   gcloud projects add-iam-policy-binding PROJECT_ID \
     --member="serviceAccount:archestra-vertex-ai@PROJECT_ID.iam.gserviceaccount.com" \
     --role="roles/aiplatform.user"
   

2. Bind the GCP service account to the Kubernetes service account:

   gcloud iam service-accounts add-iam-policy-binding \
     archestra-vertex-ai@PROJECT_ID.iam.gserviceaccount.com \
     --role="roles/iam.workloadIdentityUser" \
     --member="serviceAccount:PROJECT_ID.svc.id.goog[NAMESPACE/KSA_NAME]"
   

   Replace NAMESPACE with your Helm release namespace and KSA_NAME with the Kubernetes service account name (defaults to archestra-platform).

3. Configure Helm values to annotate the service account:

archestra:
  orchestrator:
    kubernetes:
      serviceAccount:
        annotations:
          iam.gke.io/gcp-service-account: archestra-vertex-ai@PROJECT_ID.iam.gserviceaccount.com
  env:
    ARCHESTRA_GEMINI_VERTEX_AI_ENABLED: "true"
    ARCHESTRA_GEMINI_VERTEX_AI_PROJECT: "PROJECT_ID"
    ARCHESTRA_GEMINI_VERTEX_AI_LOCATION: "us-central1"

With this configuration, Application Default Credentials (ADC) will automatically use the bound GCP service account—no credentials file needed.

Other Environments

For non-GKE environments, Vertex AI supports several authentication methods through Application Default Credentials (ADC):

• Service account key file: Set ARCHESTRA_GEMINI_VERTEX_AI_CREDENTIALS_FILE to the path of a service account JSON key file
• Local development: Use gcloud auth application-default login to authenticate with your user account
• Cloud environments: Attached service accounts on Compute Engine, Cloud Run, and Cloud Functions are automatically detected
• AWS/Azure: Use workload identity federation to authenticate without service account keys

See the Vertex AI authentication guide for detailed setup instructions for each environment.

Cerebras

Cerebras provides fast inference for open-source AI models through an OpenAI-compatible API.

Supported Cerebras APIs

• Chat Completions API (/chat/completions)

Cerebras Connection Details

• Base URL: http://localhost:9000/v1/cerebras/{profile-id}
• Authentication: Pass your Cerebras API key in the Authorization header as Bearer <your-api-key>

Cohere

Cohere provides LLMs through an API with safety guardrails, function calling, and both synchronous and streaming responses.

Supported Cohere APIs

• Chat API (/chat)
• Streaming

Cohere Connection Details

• Base URL: http://localhost:9000/v1/cohere/{profile-id}
• Authentication: Pass your Cohere API key in the Authorization header as Bearer <your-api-key>

Environment Variables

Important Notes

• API Key format: Obtain your API key from the Cohere Dashboard

Groq

Groq provides low-latency inference for popular open-source models through an OpenAI-compatible API.

Supported Groq APIs

• Chat Completions API (/chat/completions) - OpenAI-compatible

Groq Connection Details

• Base URL: http://localhost:9000/v1/groq/{profile-id}
• Authentication: Pass your Groq API key in the Authorization header as Bearer <your-api-key>

Environment Variables

Getting an API Key

You can generate an API key from the Groq Console.

Popular Models

• llama-3.3-70b-versatile
• llama-3.1-8b-instant
• gemma2-9b-it

Important Notes

• OpenAI-compatible API: Groq uses the OpenAI Chat Completions request/response format, which makes it a good fit for existing OpenAI client libraries.
• Base URL includes /openai/v1: When configuring a custom Groq endpoint, ensure the base URL points to the OpenAI-compatible API root (for example, https://api.groq.com/openai/v1).

OpenRouter

OpenRouter provides access to many models - including free ones - via a single OpenAI-compatible API, with optional attribution headers for ranking and analytics.

Supported OpenRouter APIs

• Chat Completions API (/chat/completions) - OpenAI-compatible
• Embeddings API (/embeddings) for Knowledge Base embeddings

OpenRouter Connection Details

• Base URL: http://localhost:9000/v1/openrouter/{profile-id}
• Authentication: Pass your OpenRouter API key in the Authorization header as Bearer <your-api-key>

Environment Variables

Getting an API Key

You can generate an API key from the OpenRouter dashboard.

Popular Models

• openrouter/auto - OpenRouter's Auto Router; picks the best model per request, billed at that model's rate.
• openrouter/free - OpenRouter's Free Models Router; see below.
• ~-prefixed ids such as ~anthropic/claude-sonnet-latest are OpenRouter "latest" aliases that always redirect to the newest model in a family. They sync and behave like ordinary models, and are shown with a "Latest" badge in the picker.

Free Models

OpenRouter exposes :free model variants that cost nothing. An OpenRouter API key is still required to use them, but OpenRouter doesn't charge for requests that route to free models. Model providers may use the data from free model requests to improve their models, so it may be not suitable for sensitive data.

Free Models Router (openrouter/free) is OpenRouter's built-in router that picks a free model per request, filtering for the features the request needs (tool calling, structured outputs, image input).

When an OpenRouter key is added to an organization that has no default model configured, Archestra sets the Free Models Router as the organization default, giving a zero-cost starting point. An explicitly chosen default is never overridden.

Dynamic-pricing routers (openrouter/auto) report no fixed per-token price, so the pricing is dynamic.

Mistral AI

Mistral AI provides open and commercial AI models through an OpenAI-compatible API.

Supported Mistral APIs

• Chat Completions API (/chat/completions)
• Embeddings API (/embeddings)

Mistral Connection Details

• Base URL: http://localhost:9000/v1/mistral/{profile-id}
• Authentication: Pass your Mistral API key in the Authorization header as Bearer <your-api-key>

Getting an API Key

You can get an API key from the Mistral AI Console.

Perplexity AI

Perplexity AI provides AI-powered search and answer engines with real-time web search capabilities through an OpenAI-compatible API.

Supported Perplexity APIs

• Chat Completions API (/chat/completions) — the sonar model family
• Agent API (/responses) — vendor-prefixed models such as anthropic/claude-opus-5 and perplexity/glm-5.2

One API key works for both. The model you pick selects the API.

Perplexity Connection Details

• Base URL: http://localhost:9000/v1/perplexity/{profile-id}
• Authentication: Pass your Perplexity API key in the Authorization header as Bearer <your-api-key>

Environment Variables

Getting an API Key

You can get an API key from the Perplexity Settings.

Important Notes

• Tool calling is per model: The sonar models take no external tools. They perform internal web searches and return the results. The Agent API models accept tools, so agents can run their usual workflows on them.
• Search results: sonar responses may include search_results and citations fields containing web search results used to generate the answer.
• Models: Popular models include sonar-pro and sonar-deep-research for search, and anthropic/claude-opus-5 for tool-using agents.

vLLM

vLLM is an inference and serving engine for LLMs. Use it for self-hosted deployments to run open-source models on your own infrastructure.

Supported vLLM APIs

• Chat Completions API (/chat/completions) - OpenAI-compatible
• Embeddings API (/embeddings) - OpenAI-compatible

vLLM Connection Details

• Base URL: http://localhost:9000/v1/vllm/{profile-id}
• Authentication: API key is optional. Pass in Authorization header as Bearer <your-api-key> if your vLLM deployment requires auth.

Setup

1. Go to Model Providers and add a new key with provider vLLM
2. Set the Base URL to your vLLM server (e.g., http://your-vllm-host:8000/v1)
3. API key can be left blank for most self-hosted deployments

The base URL can also be set globally via the ARCHESTRA_VLLM_BASE_URL environment variable. Per-key base URLs in the UI take precedence.

Environment Variables

Important Notes

• Configure base URL to enable vLLM: The vLLM provider is only available when ARCHESTRA_VLLM_BASE_URL is set or a per-key base URL is configured in the UI. Without either, vLLM won't appear as an option.
• Auto-seeding needs the base URL: Setting ARCHESTRA_CHAT_VLLM_API_KEY alone does not create a vLLM key at startup. ARCHESTRA_VLLM_BASE_URL must also be set, otherwise the provider is skipped (a key without a base URL would silently route to the public OpenAI endpoint).
• No API key required for most deployments: Unlike cloud providers, self-hosted vLLM typically doesn't require authentication. When adding a vLLM key in the platform, the API key field is marked as optional.

Ollama

Ollama is a local LLM runner for open-source large language models on your machine. Use it for local development, testing, and privacy-conscious deployments.

Ollama serves two APIs, and Archestra has a transport for each. Pick the transport when you add the key:

• Native — Ollama's own /api/chat endpoint. Use it to control Ollama's generation parameters — num_ctx, num_predict, top_k, repeat_penalty, thinking, and the rest. The /v1 endpoint silently drops those options.
• OpenAI-compatible — Ollama's /v1 endpoint. The simplest choice for standard chat, and the only transport with embeddings support.

Both transports talk to the same Ollama server. You can add either or both.

Supported Ollama APIs

• Chat API (/api/chat) - Native transport. Discovery endpoints (/api/tags, /api/show, /api/ps) are proxied through unmodified.
• Chat Completions API (/chat/completions) - OpenAI-compatible transport
• Embeddings API (/embeddings) - OpenAI-compatible transport only

Ollama Connection Details

• Base URL (Native): http://localhost:9000/v1/ollama-native/{profile-id}
• Base URL (OpenAI-compatible): http://localhost:9000/v1/ollama/{profile-id}
• Authentication: API key is optional. Pass it in the Authorization header as Bearer <your-api-key> if your Ollama deployment requires auth (e.g., Ollama Cloud).

Setup

1. Go to Model Providers and add a new key with provider Ollama
2. Choose a Transport: Native (the default) or OpenAI-compatible
3. Optionally set the Base URL if your Ollama server runs on a non-default host/port
4. Leave the API key blank for self-hosted Ollama

Model Parameters

Open LLM → Models, edit a native-transport model, and set any generation parameter (num_ctx, num_predict, temperature, top_p, top_k, repeat_penalty, stop, thinking) under Model parameters. Archestra sends the values you set on every chat turn; leave a field empty to inherit Ollama's own default. The section only appears for the Native transport — /v1 discards these options.

Context Window

Ollama often runs a model with a smaller context window than the model architecturally supports. Archestra resolves the effective window — a num_ctx configured under Model Parameters, else a num_ctx baked into the Modelfile, else the model's architectural context length — and displays and enforces it on the Models page and in the chat context ring.

A server-wide cap set through OLLAMA_CONTEXT_LENGTH is not reported by Ollama's model API and cannot be detected. If you run a capped server, set num_ctx on the model — a request-level value takes precedence.

Agent Suitability

Ollama reports each model's exact parameter count. Archestra marks a model Limited for complex tasks when that count is 8,000,000,000 or lower. The threshold applies to the reported count, not the name — models sold as "8B" usually report slightly more (Llama 3.1 8B reports about 8.03 billion), so they stay unmarked.

The marker shows on the model in the picker. It also shows next to the composer when the agent in that chat brings tools. A 4B model, for example, often calls those tools unreliably over a multi-step task — switch to a larger model for tool-heavy work.

The marker is advice, not a quality verdict. Models are treated as suitable unless something says otherwise, and no provider other than Ollama reports a parameter count today — so no model outside Ollama carries the marker. Each Ollama server is judged separately: the same tag can name different builds on two servers, and each key's marker reflects what its own server reports.

Environment Variables

Important Notes

• Enabled by default: both transports point at http://localhost:11434 out of the box — the OpenAI-compatible one under /v1, the native one at the server root.
• Model availability: models must be pulled first using ollama pull <model-name>.
• Not available through the model router: the router has no translation for /api/chat, so ollama-native: model IDs are not routable. Use the ollama: prefix through the router, or call the native base URL directly.
• Output tokens: Ollama publishes no output cap, so when a model's output ceiling is unknown the per-turn output budget falls back to the model's context window instead of the 8192-token default. Applies to both transports.
• Thinking (Native transport): on or off, not an effort scale. Inherit sends nothing and lets Ollama decide. Turning thinking Off on a model that reasons anyway can surface its reasoning as the visible answer.
• Running Archestra in Docker: when Archestra runs in a container and Ollama runs on the host, localhost resolves to the container itself. Use http://host.docker.internal:11434/v1 (OpenAI-compatible) or http://host.docker.internal:11434 (Native) as the Base URL instead. The platform detects this case and suggests the change when a localhost connection fails.

Zhipu AI

Zhipu AI (Z.ai) is a Chinese AI company offering the GLM (General Language Model) series of large language models. It provides both free and commercial models for Chinese and English language tasks.

Supported Zhipu AI APIs

• Chat Completions API (/chat/completions) - OpenAI-compatible
• Embeddings API (/embeddings) - OpenAI-compatible

Zhipu AI Connection Details

• Base URL: http://localhost:9000/v1/zhipuai/{profile-id}
• Authentication: Pass your Zhipu AI API key in the Authorization header as Bearer <your-api-key>

Environment Variables

Popular Models

• GLM-4.5-Flash (Free tier) - Fast inference model with good performance
• GLM-4.5 - Balanced model for general use
• GLM-4.5-Air - Lightweight model optimized for speed
• GLM-4.6 - Enhanced version with improved capabilities
• GLM-4.7 - Latest model with advanced features

Important Notes

• OpenAI-compatible API: Zhipu AI's API follows the OpenAI Chat Completions format, making it easy to switch between providers
• API Key format: Obtain your API key from the Zhipu AI Platform
• Free tier available: The GLM-4.5-Flash model is available on the free tier for testing and development
• Chinese language support: GLM models excel at Chinese language understanding and generation, while maintaining strong English capabilities

xAI (Grok)

xAI is Elon Musk's AI company offering the Grok series of large language models with real-time information access.

Supported xAI APIs

• Chat Completions API (/chat/completions) - OpenAI-compatible

xAI Connection Details

• Base URL: http://localhost:9000/v1/xai/{profile-id}
• Authentication: Pass your xAI API key in Authorization header as Bearer <your-api-key>

Environment Variables

Getting an API Key

You can generate an API key from the xAI Console.

X Premium (SuperGrok) Subscription

Reuse an X Premium (SuperGrok) subscription for chat instead of a metered API key. Add an xAI provider key, pick the X Premium (SuperGrok) tab, and use Sign in with X to connect the account that holds your subscription.

These keys are per-user and personal-only: each person connects their own X account. Requests are billed to the subscription. An agent set up with a subscription key always runs on the chatting user's own subscription — never someone else's. Users without a connected account get a sign-in prompt in chat.

The model list and inference requests use xAI's dedicated Grok CLI session proxy, not the metered api.x.ai API-key surface. Session requests carry the account identity returned by the device login.

Subscription keys only talk to the configured xAI subscription endpoint. A per-key base URL override is rejected.

Subscription sign-in is unavailable when Bring Your Own Secrets uses a read-only external Vault, because Archestra cannot save or rotate OAuth credentials there. Use a Vault-backed xAI API key instead, or switch to managed secret storage.

Popular Models

• grok-2-latest - Latest Grok model with enhanced capabilities
• grok-2-mini - Lightweight variant optimized for speed
• grok-beta - Beta version with experimental features

Important Notes

• OpenAI-compatible API: xAI's API follows the OpenAI Chat Completions format, making it easy to switch between providers
• Real-time information: Grok models have access to real-time information from X (Twitter) for up-to-date responses
• API Key format: Obtain your API key from the xAI Console
• Rate limits: Be mindful of xAI's rate limits when implementing high-volume applications

MiniMax

MiniMax is a Chinese AI company offering large language models. It provides the MiniMax-M2 series with chain-of-thought reasoning and support for text, images, and multi-turn conversations.

Supported MiniMax APIs

• Chat Completions API (/chat/completions) - OpenAI-compatible

MiniMax Connection Details

• Base URL: http://localhost:9000/v1/minimax/{profile-id}
• Authentication: Pass your MiniMax API key in the Authorization header as Bearer <your-api-key>

Environment Variables

Available Models

• MiniMax-M2 - Base model with strong reasoning capabilities ($0.3/$1.2 per M tokens)
• MiniMax-M2.1 - Enhanced model with improved performance ($0.3/$1.2 per M tokens)
• MiniMax-M2.1-lightning - Fast inference variant of M2.1 ($0.6/$2.4 per M tokens)
• MiniMax-M2.5 - Latest model with enhanced capabilities ($0.3/$1.2 per M tokens)
• MiniMax-M2.5-highspeed - Fast inference variant of M2.5 ($0.6/$2.4 per M tokens)

Important Notes

• OpenAI-compatible API (text-only): MiniMax's API follows the OpenAI Chat Completions format for easy integration. The integration uses text-only messages (no image or multimodal content support).
• Reasoning metadata: MiniMax models support extended thinking through the reasoning_details field in responses, which contains the model's reasoning process as structured data (not as <think> tags in the message content).
• API Key: Obtain your API key from the MiniMax Platform
• No /models endpoint: MiniMax does not provide a models listing API. Available models are hardcoded in the platform configuration
• Chinese and English support: MiniMax models excel at both Chinese and English language tasks

Kimi (Moonshot AI)

Kimi is Moonshot AI's family of large language models, including the Kimi K2 series and the moonshot-v1 long-context models, with strong reasoning, tool use, and long-context capabilities.

Supported Kimi APIs

• Chat Completions API (/chat/completions) - OpenAI-compatible

Kimi Connection Details

• Base URL: http://localhost:9000/v1/kimi/{profile-id}
• Authentication: Pass your Kimi API key in the Authorization header as Bearer <your-api-key>

Environment Variables

Getting an API Key

Obtain your API key from the Moonshot AI Platform.

Popular Models

• kimi-k2-0711-preview - Kimi K2 flagship model with strong reasoning and tool use
• kimi-latest - Rolling alias tracking the newest Kimi model
• moonshot-v1-128k - Long-context (128K) model

Important Notes

• OpenAI-compatible API: Kimi's API follows the OpenAI Chat Completions format, making it easy to switch between providers
• /models endpoint: Kimi exposes a models listing API, so available models are synced automatically from your account
• International endpoint: The default base URL uses Moonshot's international endpoint (api.moonshot.ai); the China endpoint (api.moonshot.cn) can be set via ARCHESTRA_KIMI_BASE_URL

GitHub Copilot

GitHub Copilot exposes the models included with a user's Copilot subscription (GPT, Claude, Gemini, and others, depending on plan) through an OpenAI-compatible API. Unlike other providers, Copilot has no static API keys: access is tied to an individual GitHub account.

Supported GitHub Copilot APIs

• Chat Completions API (/chat/completions) - OpenAI-compatible
• Responses API (/responses) - OpenAI-compatible
• Models API (/models) - lists the models the account can use

Copilot serves each model over one of the two generative APIs. The Codex and GPT-5.x models are served over the Responses API only; the rest are served over Chat Completions only. Calling a model on the wrong API returns an error, so pick the API that matches the model.

The Models API tells you which one to use. Each entry lists its API in supported_endpoints.

GitHub Copilot Connection Details

• Base URL: http://localhost:9000/v1/github-copilot/{profile-id}
• Authentication: Pass your GitHub OAuth token (the credential below) in the Authorization header as Bearer <token>

Copilot models are also reachable through the model router as github-copilot:<model-id>. See GitHub Copilot Through the Model Router.

Authentication

A GitHub Copilot provider key stores a long-lived GitHub OAuth token (gho_/ghu_…) for an account with an active Copilot subscription — not a Copilot API key, which does not exist. Archestra exchanges that token for a short-lived Copilot bearer on every request (cached and refreshed automatically), so clients only ever present the GitHub token.

Obtain the token in either way:

• Sign in with GitHub: when adding a GitHub Copilot key, use the "Sign in with GitHub" button. It runs GitHub's OAuth device flow — you approve a one-time code at github.com/login/device, and Archestra stores the resulting token.
• Reuse an existing token: the official Copilot CLI / VS Code store one in ~/.config/github-copilot/apps.json (the oauth_token value); paste it into the API key field. The /connection setup script for the Copilot CLI reuses or obtains this token automatically.

Environment Variables

Important Notes

• No static API keys: access is per-user via a GitHub OAuth token; model availability follows that account's Copilot subscription tier.
• Per-user only: because the token is tied to one GitHub account, Copilot keys are personal scope only — they can't be shared via team/org scope or wrapped in a team- or org-scoped virtual key. Each user connects their own account. Your own personal virtual key may map Copilot alongside other providers, which is what makes it routable through the model router. When someone uses an agent with a Copilot model but hasn't connected yet, Archestra resolves their key (never the agent owner's) and prompts them to connect: an inline "Connect GitHub Copilot" card in chat, or a message with a Settings link in Slack/Teams. Email and scheduled runs fail with an actionable message.
• Generative models only: the /models listing covers every model reachable through /chat/completions or /responses. Copilot also serves an Anthropic /v1/messages shim and embedding models, which Archestra does not route to.
• GitHub Enterprise: point the base, token-exchange, and device-auth URLs at your GHE host. Organizations with their own GitHub App can override the client id.

Microsoft 365 Copilot

Microsoft 365 Copilot answers prompts grounded in the user's Microsoft 365 tenant data (mail, SharePoint, Teams) and the web. Archestra connects to it through the Microsoft 365 Copilot Chat API (Microsoft Graph, beta). Like GitHub Copilot, there are no static API keys: access is tied to an individual Microsoft work account with a Microsoft 365 Copilot license.

Supported Microsoft 365 Copilot APIs

• Chat API (/copilot/conversations/{id}/chat) - synchronous answers
• Chat streaming API (/copilot/conversations/{id}/chatOverStream) - streamed answers

Archestra exposes both through its standard OpenAI-compatible /chat/completions proxy surface. Streaming requests (the built-in chat always streams) use chatOverStream; non-streaming requests use chat. The single model is microsoft-365-copilot — the Chat API has no model selection.

Microsoft 365 Copilot Connection Details

• Base URL: http://localhost:9000/v1/microsoft-365-copilot/{profile-id}
• Authentication: Pass the stored Entra refresh token (the credential below) in the Authorization header as Bearer <token>

Prerequisites: Entra App Registration

The sign-in flow needs an Entra ID app registration owned by your organization:

1. Register an application in the Microsoft Entra admin center. The device flow runs as a public client — skip the client secret and redirect URI.
2. Enable Allow public client flows (Authentication → Advanced settings). Without it, sign-in fails after the code is entered.
3. Add these delegated Microsoft Graph permissions and grant admin consent: Sites.Read.All, Mail.Read, People.Read.All, OnlineMeetingTranscript.Read.All, Chat.Read, ChannelMessage.Read.All, ExternalItem.Read.All. The Chat API requires all seven — one per data source Copilot searches.
4. Set ARCHESTRA_MICROSOFT_365_COPILOT_CLIENT_ID to the Application (client) ID.
5. For a single-tenant registration, also set ARCHESTRA_MICROSOFT_365_COPILOT_TENANT_ID to your tenant ID. The default (organizations) only works for multi-tenant registrations.

With a multi-tenant registration, users from another organization can sign in once their own tenant admin consents to the app.

Authentication

A Microsoft 365 Copilot provider key stores a long-lived Entra refresh token for an account with a Microsoft 365 Copilot license. Archestra redeems it for a short-lived Graph access token on every request (cached and refreshed automatically). Entra rotates refresh tokens; Archestra persists the rotated token back to the key.

To connect, use the Sign in with Microsoft button when adding a Microsoft 365 Copilot key. It runs Entra's OAuth device flow — you approve a one-time code on Microsoft's device sign-in page, and Archestra stores the resulting refresh token.

Environment Variables

Important Notes

• Preview API: the Chat API is a Microsoft Graph beta endpoint. Microsoft does not support it for production use and may change it without notice.
• License required: each user needs a Microsoft 365 Copilot add-on license, assigned in their own tenant. The seat license covers all Chat API usage. A missing license surfaces on the first chat request.
• Work accounts only: the Chat API supports delegated work or school accounts.
• Per-user only: keys are personal scope only, same as GitHub Copilot. Each user connects their own Microsoft account; an inline "Connect Microsoft 365 Copilot" card appears in chat when a key is missing. Every request runs as the signed-in user — Copilot only sees data that user can already access.
• Text-only, no tools: the Chat API returns text answers only. It cannot run tools or Copilot actions such as creating files, sending emails, or scheduling meetings. In Archestra chat, an agent with tools runs without them on this model — a notice above the composer says so. Proxy requests that declare tools are rejected with a clear error.
• Conversational answers only: prompts that trigger long-running work can hit Microsoft's gateway timeout. Keep requests to questions and answers.
• Estimated usage: the Chat API reports no token counts, so usage and cost figures are tokenizer estimates.
• Stateless mapping: each request creates a fresh Copilot conversation; prior turns ride along as context. If a streaming response has no recognizable text, Archestra retries through the synchronous endpoint in a second conversation, so one request can appear as two conversations in Microsoft 365 activity.
• Conversation cleanup: the Copilot conversation API currently documents no delete operation. If a chat request fails after its conversation is created, the abandoned conversation may remain visible in Microsoft 365 activity.

Archestra

Use another Archestra instance as an upstream provider. One Archestra routes its traffic through a second Archestra, which applies its own policies before reaching the real model. The upstream's model router is OpenAI-compatible, so this provider follows the OpenAI chat-completions path and can reach every provider that instance has configured.

Supported Archestra APIs

• Chat Completions API (/chat/completions) - OpenAI-compatible

Archestra Connection Details

• Base URL: the upstream Archestra's model router, for example https://your-archestra/v1/model-router/<llm-proxy-id>.
• Authentication: pass a virtual API key (arch_...) minted from that LLM Proxy in the Authorization header as Bearer <key>.

Setup

1. On the upstream Archestra, go to LLM Proxies and create an LLM Proxy, then create a virtual API key for it.
2. On this Archestra, go to Model Providers and add a new key with provider Archestra.
3. Set the Base URL to the upstream proxy's model router (for example https://your-archestra/v1/model-router/<llm-proxy-id>).
4. Paste the virtual API key from the upstream LLM Proxy.

Archestra fetches the model list from the upstream's {base-url}/models endpoint, so the picker shows exactly the models that proxy exposes. Model IDs are provider-qualified, for example openai:gpt-5.4.

Environment Variables

Important Notes

• Base URL is required: the upstream endpoint has no default, so a per-key base URL is always needed. Without one, the provider cannot resolve an upstream and requests would fall back to the public OpenAI endpoint.
• Models come from the upstream: the model list mirrors whatever the upstream LLM Proxy exposes, so it stays in sync as that instance changes.

Amazon Bedrock

Supported Bedrock APIs

• Converse API (/converse) (AWS Docs)
• Converse Stream API (/converse-stream) (AWS Docs)
• InvokeModel API (/model/{model-id}/invoke and /model/{model-id}/invoke-with-response-stream) for Anthropic models (AWS Docs). This is the API the Anthropic SDK's Bedrock client uses — point Claude Code at Archestra with CLAUDE_CODE_USE_BEDROCK=1 and ANTHROPIC_BEDROCK_BASE_URL=http://localhost:9000/v1/bedrock/{profile-id}.
• OpenAI-compatible API (Mantle) - ⚠️ Not yet supported (AWS Docs)

Bedrock Connection Details

• Base URL: http://localhost:9000/v1/bedrock/{profile-id}
• Authentication: Bearer API key or AWS IAM (see below)

Region

Each Bedrock key carries an AWS region. Amazon enables models per region, so the region decides which models the key can use — pick the one where your models are turned on.

A key can also point at a custom endpoint instead, for a VPC or PrivateLink setup. Archestra reads the region back out of that endpoint, and falls back to us-east-1 when the endpoint carries no region.

Authentication Methods

Bedrock supports two authentication methods:

API Key (default) — Pass your Bedrock API key via the UI or ARCHESTRA_CHAT_BEDROCK_API_KEY env var.

AWS IAM — Use the AWS credential chain (IRSA, instance profiles, environment variables) instead of API keys. When enabled, Archestra authenticates to Bedrock using SigV4 signing. No API key is needed — Bedrock appears as a system-configured provider automatically.

IAM Authentication Setup (IRSA)

To use IAM authentication on EKS with IRSA:

1. Create an IAM role with AmazonBedrockFullAccess or a scoped policy (see below)

2. Create an OIDC provider for your EKS cluster

3. Configure the IAM role's trust policy to allow the Archestra service account:

   {
     "Effect": "Allow",
     "Principal": {
       "Federated": "arn:aws:iam::<ACCOUNT_ID>:oidc-provider/oidc.eks.<REGION>.amazonaws.com/id/<OIDC_ID>"
     },
     "Action": "sts:AssumeRoleWithWebIdentity",
     "Condition": {
       "StringEquals": {
         "oidc.eks.<REGION>.amazonaws.com/id/<OIDC_ID>:sub": "system:serviceaccount:archestra:archestra-platform"
       }
     }
   }
   

4. Annotate the Archestra service account:

   kubectl annotate sa archestra-platform -n archestra \
     eks.amazonaws.com/role-arn=arn:aws:iam::<ACCOUNT_ID>:role/<ROLE_NAME>
   

5. Set the environment variables below and restart the deployment

Minimum IAM Policy

Archestra calls the Bedrock Converse API, and the InvokeModel API for clients that use it (Claude Code, for example). The IAM role needs these actions:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "bedrock:Converse",
        "bedrock:ConverseStream",
        "bedrock:InvokeModel",
        "bedrock:InvokeModelWithResponseStream"
      ],
      "Resource": [
        "arn:aws:bedrock:*:<ACCOUNT_ID>:inference-profile/us.anthropic.*",
        "arn:aws:bedrock:*::foundation-model/anthropic.*"
      ]
    },
    {
      "Effect": "Allow",
      "Action": [
        "bedrock:ListInferenceProfiles",
        "bedrock:ListFoundationModels"
      ],
      "Resource": "*"
    }
  ]
}

Use * for the region in resource ARNs — cross-region inference profiles (us. prefix) can route requests to any US region.

The two list actions populate the model picker. ListInferenceProfiles returns cross-region and application inference profiles. ListFoundationModels adds on-demand models that have no inference profile. Without it, those models are not offered.

Environment Variables

Common (both auth methods)

API Key auth

IAM auth (IRSA / instance profiles)

When IAM auth is enabled, Archestra uses the AWS credential chain — IRSA on EKS, EC2 instance profiles, or AWS_ACCESS_KEY_ID/AWS_SECRET_ACCESS_KEY env vars. No API key is needed.

ARCHESTRA_BEDROCK_BASE_URL

Required to enable the Bedrock provider. The URL format follows AWS regional endpoints:

https://bedrock-runtime.{region}.amazonaws.com

Model Discovery

Archestra uses the Bedrock ListInferenceProfiles API to discover available models. This means only models that have inference profiles configured in your AWS account will appear — ensuring the model picker only shows models you can actually use.

Filtering Models by Provider

By default, Archestra returns all active inference profiles from your AWS account. Use ARCHESTRA_BEDROCK_ALLOWED_PROVIDERS to limit which providers appear in the model picker.

The filter matches the provider segment of the inference profile ID (the part after the region prefix). For example, the profile us.anthropic.claude-sonnet-4-6 has provider anthropic.

# Only Anthropic and Amazon models
ARCHESTRA_BEDROCK_ALLOWED_PROVIDERS=anthropic,amazon

# Only Anthropic models
ARCHESTRA_BEDROCK_ALLOWED_PROVIDERS=anthropic

# All providers (default)
ARCHESTRA_BEDROCK_ALLOWED_PROVIDERS=

Common provider prefixes: anthropic, amazon, meta, mistral, deepseek, cohere, writer, stability, twelvelabs.

Filtering Models by Inference Region

Use ARCHESTRA_BEDROCK_ALLOWED_INFERENCE_REGIONS to limit which inference
regions appear in the model picker.

The filter matches the region prefix of the inference profile ID (the first
segment before the provider). For example, the profile
us.anthropic.claude-sonnet-4-6 has region prefix us.

# Only US and global profiles
ARCHESTRA_BEDROCK_ALLOWED_INFERENCE_REGIONS=us,global

# Only EU profiles
ARCHESTRA_BEDROCK_ALLOWED_INFERENCE_REGIONS=eu

# All regions (default)
ARCHESTRA_BEDROCK_ALLOWED_INFERENCE_REGIONS=

Known region prefixes: us, eu, ap, global.

Azure AI Foundry

Azure AI Foundry (formerly Azure OpenAI) provides access to OpenAI models through Microsoft Azure, with an OpenAI-compatible API.

Supported Azure AI Foundry APIs

• Chat Completions (streaming and non-streaming)
• Responses API (streaming and non-streaming)
• Embeddings API (/embeddings) - OpenAI-compatible

Azure AI Foundry Connection Details

• Base URL: http://localhost:9000/v1/azure/{profile-id}
• API key authentication: Pass your Azure API key in the Authorization header as Bearer <your-api-key>
• Keyless authentication: Set ARCHESTRA_AZURE_OPENAI_ENTRA_ID_ENABLED=true and assign the workload identity, managed identity, service principal, or local Azure CLI user an Azure role that can invoke the deployed model.

Azure AI Foundry Environment Variables

Setting ARCHESTRA_CHAT_AZURE_OPENAI_API_KEY alone does not create an Azure key at startup; ARCHESTRA_AZURE_OPENAI_BASE_URL must also be set (Azure has no usable default endpoint), otherwise the provider is skipped.

Getting an Azure API Key

You can generate an API key from the Azure Portal under your Azure OpenAI resource.

Keyless Authentication with Microsoft Entra ID

To use Azure OpenAI without storing an API key, set:

ARCHESTRA_AZURE_OPENAI_ENTRA_ID_ENABLED=true

Then create an Azure provider key in Archestra with no API key value and set its Base URL to one of the Azure resource endpoints below.

https://<resource-name>.openai.azure.com/openai

For Foundry v1, use:

https://<resource-name>.services.ai.azure.com/openai/v1

Archestra uses Azure Identity DefaultAzureCredential. Deployment URLs use the https://cognitiveservices.azure.com/.default token scope. Foundry v1 URLs use https://ai.azure.com/.default. Assign the workload identity, managed identity, service principal, or local Azure CLI user a role that can invoke the Azure resource.

See the Azure OpenAI keyless example for a minimal local script that uses the same authentication flow.
See Microsoft's Foundry Models Entra ID guide and Foundry Models endpoint guide for the Azure endpoint formats and token scopes.

AKS with Microsoft Entra Workload ID

For AKS deployments, use Microsoft Entra Workload ID with a user-assigned managed identity. Microsoft documents that Azure Identity DefaultAzureCredential uses the workload identity environment injected into the pod.

Enable OIDC issuer and workload identity on the AKS cluster, create a federated identity credential for the Archestra Kubernetes service account, and grant the managed identity the inference role required by the resource: Cognitive Services OpenAI User for Azure OpenAI deployment URLs, or Cognitive Services User for Foundry Models. The service account subject must match the namespace and service account name used by the Helm release:

az aks update \
  --resource-group "$AKS_RESOURCE_GROUP" \
  --name "$AKS_CLUSTER_NAME" \
  --enable-oidc-issuer \
  --enable-workload-identity

export AKS_OIDC_ISSUER="$(az aks show \
  --resource-group "$AKS_RESOURCE_GROUP" \
  --name "$AKS_CLUSTER_NAME" \
  --query oidcIssuerProfile.issuerUrl \
  --output tsv)"

az identity federated-credential create \
  --resource-group "$IDENTITY_RESOURCE_GROUP" \
  --identity-name "$USER_ASSIGNED_IDENTITY_NAME" \
  --name archestra-platform \
  --issuer "$AKS_OIDC_ISSUER" \
  --subject "system:serviceaccount:$NAMESPACE:$SERVICE_ACCOUNT_NAME" \
  --audience api://AzureADTokenExchange

Then annotate the Helm service account and add the pod label required by the AKS workload identity webhook:

archestra:
  orchestrator:
    kubernetes:
      serviceAccount:
        name: archestra-platform
        annotations:
          azure.workload.identity/client-id: "<user-assigned-managed-identity-client-id>"
  podLabels:
    azure.workload.identity/use: "true"
  env:
    ARCHESTRA_AZURE_OPENAI_ENTRA_ID_ENABLED: "true"

See Microsoft's AKS Workload ID deployment guide for the full cluster, service account, and federated credential setup.

Base URL Format

For Azure OpenAI resources, use the shared resource-level OpenAI URL:

https://<resource-name>.openai.azure.com/openai

Archestra discovers deployments from /openai/deployments and routes each request to the deployment named in the request model field.
Do not configure a deployment-specific URL such as https://<resource-name>.openai.azure.com/openai/deployments/<deployment-name>.
If your Foundry project has its own OpenAI endpoint, use the same resource-level format with the project hostname:

https://<project-name>.openai.azure.com/openai

For Microsoft Foundry v1, use the OpenAI-compatible API root:

https://<resource-name>.services.ai.azure.com/openai/v1

The same formats apply when configuring a Base URL in the API key settings UI. Base URL is used for deployment discovery and as the default runtime endpoint.

If deployment discovery and runtime inference use different Azure OpenAI endpoints, set the provider key's optional Inference URL to the runtime endpoint:

https://<runtime-resource-name>.openai.azure.com/openai

Archestra will still discover deployments from Base URL, then send chat, reranking, embedding, LLM Proxy, OAuth client, and virtual key traffic to Inference URL.

Deployment Discovery and RBAC

• For Entra ID configurations, Archestra first tries Azure deployment discovery. If the inference endpoint cannot list deployments, Archestra uses Azure management APIs to find the Cognitive Services account and list its deployments.
• Some Foundry project endpoints are backed by a parent Azure AI Services account, for example /providers/Microsoft.CognitiveServices/accounts/<account-name>/projects/<project-name>. Archestra resolves the project to its parent account before listing deployments.
• For Azure OpenAI resource URLs, Archestra does not fall back to the available model catalog because that catalog includes undeployed models.
• For built-in Azure RBAC, assign Cognitive Services OpenAI User at the backing Azure AI Services resource when possible. Use the full ARM resource scope, for example /subscriptions/<subscription-id>/resourceGroups/<resource-group>/providers/Microsoft.CognitiveServices/accounts/<resource-name>. For the narrowest access, use a custom role with Microsoft.Resources/subscriptions/read, Microsoft.Resources/subscriptions/resources/read, Microsoft.CognitiveServices/accounts/read, and Microsoft.CognitiveServices/accounts/deployments/read.

Routing Notes

• API Version: Azure OpenAI resource URLs use ARCHESTRA_AZURE_OPENAI_API_VERSION for Chat Completions and model discovery. Azure /responses requests use ARCHESTRA_AZURE_OPENAI_RESPONSES_API_VERSION. Foundry v1 URLs do not use either query parameter.
• Microsoft Entra ID: When ARCHESTRA_AZURE_OPENAI_ENTRA_ID_ENABLED=true, Azure provider keys can omit the API key value and Archestra sends Authorization: Bearer <token> to Azure OpenAI instead of api-key.
• Grok on Azure: Grok models sold directly by Azure use the Foundry v1 OpenAI-compatible Chat Completions API. The model must be deployed in the Azure resource before Archestra can route to it.
• Claude on Azure: Claude models on Microsoft Foundry use Anthropic's Messages API shape, not the OpenAI-compatible Azure route. Configure the Anthropic provider section above.
• Multiple Deployments: Azure OpenAI is the main provider that exposes multiple deployment names behind one resource-level credential. One Azure provider key should represent the Azure resource or Foundry v1 endpoint, not an individual deployment. After model sync, select the deployment by model name.
• Responses API model field: For Azure /responses requests, send the deployment name in the model field. Archestra will route the request to Azure's /openai/responses endpoint while preserving the configured deployment URL for discovery and management.
• OpenAI-compatible API: Azure AI Foundry supports both Chat Completions and Responses-style request flows through Archestra.