# OpenAI-compatible Servers

> Connect vLLM, llama.cpp, LM Studio, or another OpenAI-compatible server

Connect vLLM, llama.cpp, LM Studio, SGLang, or any other server with the OpenAI `/v1` API. Add it as the **OpenAI-compatible** provider. Archestra uses its Chat Completions and Embeddings endpoints.

## Connecting a Server

Start your model server before adding it. Archestra must be able to reach its URL; `localhost` inside a container refers to that container.

1. Go to **Model Providers → Add API Key** and select **OpenAI-compatible**.
2. Set **Base URL** to the server's OpenAI API root, for example `http://model-server:8000/v1`.
3. Enter an API key if your server requires one. Otherwise leave it blank.
4. Click **Test & Create**.

The key appears in the provider table, and the models returned by your server appear under **Models**. A server that lists several models needs only one key entry. Add separate entries for servers on different URLs.

Proxy clients use `https://<archestra-host>/v1/vllm`. Through the [Unified Model API](https://archestra.ai/docs/llm-proxy/unified-model), use model IDs prefixed with `vllm:`.

## Deployment Configuration

Set [`ARCHESTRA_VLLM_BASE_URL`](https://archestra.ai/docs/admin/deployment/configuration#ARCHESTRA_VLLM_BASE_URL) to configure a default endpoint. A Base URL set on a provider key takes precedence. [`ARCHESTRA_CHAT_VLLM_API_KEY`](https://archestra.ai/docs/admin/deployment/configuration#ARCHESTRA_CHAT_VLLM_API_KEY) supplies its default credential when needed.

The API key variable alone does not create a provider key at startup: the Base URL variable must also be set. If connection testing fails, check that the backend can reach the server and that `/models` is available under the configured API root.

## Prompt Caching

OpenAI-compatible servers manage their own prompt or prefix caches, and support depends on the server and model. The `/v1/vllm` endpoint currently does not forward explicit `prompt_cache_key`, `prompt_cache_retention`, `prompt_cache_options`, or content-block `prompt_cache_breakpoint` fields; the native OpenAI endpoint has separate request validation.

Archestra does not configure the server's cache, insert cache breakpoints, or guarantee cache hits. Check your server's documentation for automatic prefix caching and the usage fields it reports.
