# Google Gemini

> Connect Gemini using Google AI Studio or Vertex AI.

Use Gemini through Google AI Studio with an API key, or through Vertex AI with no key. On GKE, Archestra signs in to Vertex AI with Workload Identity.

## Adding a Provider

Go to **Model Providers → Add API Key**, select **Google Gemini**, and configure the credentials described below. Click **Test & Create**. A successful test adds the key and makes its models available under **Models**.

Archestra supports both the [Google AI Studio](https://ai.google.dev/) (Gemini Developer API) and [Vertex AI](https://cloud.google.com/vertex-ai) implementations of the Gemini API.

### Gemini Connection Details

- **Base URL**: `https://<archestra-host>/v1/gemini/v1beta`
- **Authentication**:
  - **Google AI Studio (default)**: Pass your Gemini API key in the `x-goog-api-key` header
  - **Vertex AI**: No API key required from clients - uses server-side [Application Default Credentials (ADC)](https://cloud.google.com/docs/authentication/application-default-credentials)

### Using Vertex AI

To use Vertex AI instead of Google AI Studio, configure these environment variables:

| Variable                                      | Required | Description                            |
| --------------------------------------------- | -------- | -------------------------------------- |
| [`ARCHESTRA_GEMINI_VERTEX_AI_ENABLED`](https://archestra.ai/docs/admin/deployment/configuration#ARCHESTRA_GEMINI_VERTEX_AI_ENABLED)          | Yes      | Set to `true` to enable Vertex AI mode |
| [`ARCHESTRA_GEMINI_VERTEX_AI_PROJECT`](https://archestra.ai/docs/admin/deployment/configuration#ARCHESTRA_GEMINI_VERTEX_AI_PROJECT)          | Yes      | Your GCP project ID                    |
| [`ARCHESTRA_GEMINI_VERTEX_AI_LOCATION`](https://archestra.ai/docs/admin/deployment/configuration#ARCHESTRA_GEMINI_VERTEX_AI_LOCATION)         | No       | GCP region (default: `us-central1`)    |
| [`ARCHESTRA_GEMINI_VERTEX_AI_CREDENTIALS_FILE`](https://archestra.ai/docs/admin/deployment/configuration#ARCHESTRA_GEMINI_VERTEX_AI_CREDENTIALS_FILE) | No       | Path to service account JSON key file  |

Vertex AI mode also gives Knowledge access to Vertex's multimodal embedding model (`multimodalembedding@001`) — see [Image Embedding](https://archestra.ai/docs/knowledge/settings#image-embedding).

### Priority PayGo

When Archestra uses Vertex AI, request Priority PayGo by sending these headers to its Gemini `generateContent` or `streamGenerateContent` endpoint:

```http
X-Vertex-AI-LLM-Shared-Request-Type: priority
X-Vertex-AI-LLM-Request-Type: shared
```

Archestra forwards both values to Vertex. Omit `X-Vertex-AI-LLM-Request-Type: shared` to use available Provisioned Throughput before spilling over to Priority PayGo.

Check `usageMetadata.trafficType` in the response or streamed usage chunk: `ON_DEMAND_PRIORITY` confirms priority service; `ON_DEMAND` means standard service. Eligibility, pricing, and possible downgrades follow [Vertex AI's Priority PayGo rules](https://docs.cloud.google.com/vertex-ai/generative-ai/docs/priority-paygo).

### Missing Models and Nano Banana

Some Vertex AI models, including Nano Banana 2.1, require the global endpoint.

1. Set [`ARCHESTRA_GEMINI_VERTEX_AI_ALLOW_GLOBAL_ENDPOINT=true`](https://archestra.ai/docs/admin/deployment/configuration#ARCHESTRA_GEMINI_VERTEX_AI_ALLOW_GLOBAL_ENDPOINT) and restart Archestra.
2. Click **Refresh Models**, then search for `Nano Banana`.

Leave global access disabled if requests must stay in your configured region.

### GKE with Workload Identity (Recommended)

For GKE deployments, we recommend using [Workload Identity](https://cloud.google.com/kubernetes-engine/docs/how-to/workload-identity) which provides secure, keyless authentication. This eliminates the need for service account JSON key files.

**Setup steps:**

1. **Create a GCP service account** with Vertex AI permissions:

    ```bash
    gcloud iam service-accounts create archestra-vertex-ai \
      --display-name="Archestra Vertex AI"

    gcloud projects add-iam-policy-binding PROJECT_ID \
      --member="serviceAccount:archestra-vertex-ai@PROJECT_ID.iam.gserviceaccount.com" \
      --role="roles/aiplatform.user"
    ```

2. **Bind the GCP service account to the Kubernetes service account**:

    ```bash
    gcloud iam service-accounts add-iam-policy-binding \
      archestra-vertex-ai@PROJECT_ID.iam.gserviceaccount.com \
      --role="roles/iam.workloadIdentityUser" \
      --member="serviceAccount:PROJECT_ID.svc.id.goog[NAMESPACE/KSA_NAME]"
    ```

    Replace `NAMESPACE` with your Helm release namespace and `KSA_NAME` with the Kubernetes service account name (defaults to `archestra-platform`).

3. **Configure Helm values** to annotate the service account:

```yaml
archestra:
  orchestrator:
    kubernetes:
      serviceAccount:
        annotations:
          iam.gke.io/gcp-service-account: archestra-vertex-ai@PROJECT_ID.iam.gserviceaccount.com
  env:
    ARCHESTRA_GEMINI_VERTEX_AI_ENABLED: "true"
    ARCHESTRA_GEMINI_VERTEX_AI_PROJECT: "PROJECT_ID"
    ARCHESTRA_GEMINI_VERTEX_AI_LOCATION: "us-central1"
```

With this configuration, Application Default Credentials (ADC) will automatically use the bound GCP service account—no credentials file needed.

### Other Environments

For non-GKE environments, Vertex AI supports several authentication methods through [Application Default Credentials (ADC)](https://cloud.google.com/docs/authentication/application-default-credentials):

- **Service account key file**: Set [`ARCHESTRA_GEMINI_VERTEX_AI_CREDENTIALS_FILE`](https://archestra.ai/docs/admin/deployment/configuration#ARCHESTRA_GEMINI_VERTEX_AI_CREDENTIALS_FILE) to the path of a service account JSON key file
- **Local development**: Use `gcloud auth application-default login` to authenticate with your user account
- **Cloud environments**: Attached service accounts on Compute Engine, Cloud Run, and Cloud Functions are automatically detected
- **AWS/Azure**: Use workload identity federation to authenticate without service account keys

See the [Vertex AI authentication guide](https://cloud.google.com/vertex-ai/docs/authentication) for detailed setup instructions for each environment.

## Prompt Caching

Cache a long system prompt or tool list once, and every request that references it bills its input at Gemini's cached rate.

1. Create the cache through Archestra with your Google SDK's caches API, such as `client.caches.create(...)`. Use the same base URL and key as for generation.
2. Pass the returned cache name in `cachedContent` when calling `generateContent` or `streamGenerateContent`.

Archestra records provider-reported cached input tokens, including implicit cache hits. Cache lifetime, model eligibility, and storage charges follow [Gemini’s caching rules](https://ai.google.dev/gemini-api/docs/caching).

The [Unified Model API](https://archestra.ai/docs/llm-proxy/unified-model) also accepts `cachedContent` alongside Chat Completions or Responses input when targeting Gemini. Reference a cache created for that Gemini model, and follow Google's restrictions on supplying system instructions or tools alongside cached content.

Anthropic-style `cache_control` markers and OpenAI-specific cache options cannot be translated to Gemini; the router rejects them with a 400 error instead of silently ignoring them.
