PostgreSQL
PostgreSQL holds all of Archestra's state, so its backups and failover are your instance's backups and failover. Use a managed database in production: the Helm chart's bundled database runs one replica with no standby.
Choose a Database
Archestra needs PostgreSQL 15 or newer; use 17 or 18. It also needs the vector (pgvector 0.8 or newer) and pg_ extensions, which every major cloud's managed PostgreSQL offers.
- Archestra's migrations create both extensions. The migration user must be allowed to, or a database administrator must create them first.
- Older pgvector versions can return fewer knowledge base results when a search filters by document.
- Archestra pools its own connections and sends every query to the primary. Connect it to the primary directly; a read replica adds no capacity.
Cloud Recommendations
Pick the instance for your size, turn on a standby in a second zone, and keep point-in-time recovery on.
Amazon RDS for PostgreSQL fits every size. Aurora PostgreSQL also works and grows its storage on its own; it offers memory-optimized classes only.
| Size | RDS Instance | Aurora Instance | RDS Storage |
|---|---|---|---|
| Small | db.m7g.large | db.r7g.large | 100 GiB gp3 |
| Medium | db.m7g.xlarge | db.r7g.xlarge | 250 GiB gp3 |
| Large | db.m7g.2xlarge, or db.r7g.2xlarge for a large knowledge base | db.r7g.2xlarge | 500 GiB gp3 |
- High availability: create a Multi-AZ deployment, or add an Aurora replica in another Availability Zone.
- Backups: automated backups give point-in-time recovery. Keep at least seven days.
- Extensions: the master user can create
vectorandpg_. Pick an engine version that ships pgvector 0.8 or newer.trgm - Storage: turn on storage autoscaling. A gp3 volume under 400 GiB is fixed at 3,000 IOPS; grow it to 400 GiB to provision more.
- Connections: the default
max_follows instance memory and covers each size.connections - Avoid: burstable
db.t4gclasses. They slow down when CPU credits run out.
Cloud SQL for PostgreSQL fits every size. Enterprise Plus edition has shorter maintenance downtime and keeps point-in-time recovery longer. AlloyDB for PostgreSQL also works, with the same extensions.
| Size | Cloud SQL Enterprise | Cloud SQL Enterprise Plus | Storage |
|---|---|---|---|
| Small | db-custom-2-8192 (2 vCPU, 8 GB) | db-perf-optimized-N-2 (2 vCPU, 16 GB) | 100 GB SSD |
| Medium | db-custom-4-16384 (4 vCPU, 16 GB) | db-perf-optimized-N-4 (4 vCPU, 32 GB) | 250 GB SSD |
| Large | db-custom-8-32768 (8 vCPU, 32 GB) | db-perf-optimized-N-8 (8 vCPU, 64 GB) | 500 GB SSD |
- High availability: turn on regional availability. The standby runs in another zone.
- Backups: turn on automated backups and point-in-time recovery.
- Extensions: the
postgresuser can createvectorandpg_.trgm - Storage: keep automatic storage increases on. Cloud SQL sets IOPS and throughput from the machine type and disk size, so grow the disk to raise them.
- Connections: set the
max_flag to the size's Max Connections. The memory-based default can be lower.connections - Network: connect over private IP from the cluster's VPC.
Azure Database for PostgreSQL flexible server fits every size. Use the General Purpose tier, and Memory Optimized for a large knowledge base.
| Size | Compute | Storage |
|---|---|---|
| Small | General Purpose Standard_ | 128 GiB Premium SSD |
| Medium | General Purpose Standard_ | 256 GiB Premium SSD |
| Large | General Purpose Standard_, or Memory Optimized Standard_ | 512 GiB Premium SSD |
- Extensions: add
VECTORandPG_to theTRGM azure.extensionsserver parameter before you install Archestra. Without them, the first migration fails. - High availability: turn on zone-redundant high availability.
- Backups: point-in-time recovery is automatic; keep at least seven days. Geo-redundant backup can only be chosen when you create the server.
- Storage: turn on storage autogrow.
- Connections: connect on port 5432. The fixed
max_of each General Purpose and Memory Optimized size covers the matching Archestra size.connections - Avoid: the Burstable tier. It slows down when CPU credits run out.
Connect a Database
On AWS and Google Cloud, Archestra's pods sign in to the database with their cloud identity, so no database password exists to leak or rotate. On every cloud, keep the database on a private network with no public endpoint, connect as a user that owns only the Archestra database, and verify TLS.
Use IAM database authentication: each new connection signs a 15-minute token with the pod's IAM role. Set it up with Amazon RDS IAM Authentication.
- Network: set Public access to No, put the instance in private subnets, and allow port 5432 in its security group only from the cluster.
- TLS: IAM authentication always verifies the server certificate. The platform image ships the RDS CA bundle.
Run the Cloud SQL Auth Proxy beside Archestra. The chart adds it to the web, worker, and migration pods. It reaches the instance over private IP, encrypts the connection, and signs in as a Google service account through Workload Identity.
Before you start, give the instance a private IP in the cluster's VPC, turn on its cloudsql.iam_authentication flag, and turn on Workload Identity in the GKE cluster. AlloyDB does not work with this proxy; connect it with a password.
-
Create a Google service account that can connect to Cloud SQL and sign in to it:
bashgcloud iam service-accounts create archestra-db --project=PROJECT_ID for role in roles/cloudsql.client roles/cloudsql.instanceUser; do gcloud projects add-iam-policy-binding PROJECT_ID \ --member="serviceAccount:archestra-db@PROJECT_ID.iam.gserviceaccount.com" \ --role="$role" done -
Let Archestra's Kubernetes ServiceAccount use it. In the installation example, that is
archestra-platformin thearchestranamespace:bashgcloud iam service-accounts add-iam-policy-binding \ archestra-db@PROJECT_ID.iam.gserviceaccount.com \ --role="roles/iam.workloadIdentityUser" \ --member="serviceAccount:PROJECT_ID.svc.id.goog[archestra/archestra-platform]" -
Add the service account as a database user:
bashgcloud sql users create archestra-db@PROJECT_ID.iam \ --instance=INSTANCE --type=cloud_iam_service_account -
Connect to the
archestradatabase as thepostgresuser, create the extensions, and grant the new user access:sqlCREATE EXTENSION IF NOT EXISTS vector; CREATE EXTENSION IF NOT EXISTS pg_trgm; GRANT ALL PRIVILEGES ON DATABASE archestra TO "archestra-db@PROJECT_ID.iam"; GRANT ALL ON SCHEMA public TO "archestra-db@PROJECT_ID.iam";If Archestra's tables already exist under another user, such as
archestra, also runGRANT archestra TO "archestra-db@PROJECT_ID.iam";so migrations can still change them. -
Turn on the proxy and point Archestra at it:
yamlpostgresql: enabled: false external_database_url: "postgresql://archestra-db%40PROJECT_ID.iam@127.0.0.1:5432/archestra?sslmode=disable" archestra: database: cloudSqlProxy: enabled: true instanceConnectionName: PROJECT_ID:REGION:INSTANCE orchestrator: kubernetes: serviceAccount: annotations: iam.gke.io/gcp-service-account: archestra-db@PROJECT_ID.iam.gserviceaccount.comThe URL names the IAM user and holds no secret.
sslmode=disablecovers only the hop inside the pod; the proxy encrypts the connection to Cloud SQL. -
Run the Helm upgrade. It worked when the migration Job completes and the web pods turn ready.
- Migration Job cannot connect: run
kubectl logs <pod> -c cloud-sql-proxyon its pod. A 403 means the service account lacksroles/cloudsql.clientor the Workload Identity binding. Cloud SQL IAM service account authentication failed: the service account lacksroles/cloudsql.instanceUser, the database user is missing, or thecloudsql.iam_authenticationflag is off.permission denied for schema public: run the grants as thepostgresuser.
Archestra signs in to Azure Database for PostgreSQL flexible server with a password; it does not support Microsoft Entra ID sign-in. Keep the password in a Kubernetes Secret, and keep the server off the internet:
- Network: create the server with private access (VNet integration), or turn off public access and add a private endpoint in the cluster's VNet.
- User: as the server admin, create the extensions in the
archestradatabase, then create a dedicated user that owns that database. Do not connect Archestra as the server admin. - TLS: add
sslmode=verify-fullto the URL. Azure's certificates chain to roots Archestra already trusts, so no CA bundle is needed.
Then follow Connect with a Password.
Connect with a Password
Use a password for Azure, AlloyDB, and any other PostgreSQL. The application and the migration Job read the same URL.
-
In the Helm release namespace (
archestrain the installation example), create a Kubernetes Secret namedarchestra-databasewith the complete connection URL in itsurlkey. -
Reference it in the shared application settings:
yamlpostgresql: enabled: false archestra: envWithValueFrom: - name: ARCHESTRA_DATABASE_URL valueFrom: secretKeyRef: name: archestra-database key: urlThe migration Job inherits this Secret reference. The chart runs migrations before rolling out an upgrade. Keep
archestra.migrationJob.enabledenabled unless your pipeline applies migrations separately. -
Apply the Helm upgrade and confirm the migration Job succeeds before checking the application rollout.
Verify TLS Certificates
To verify the database server certificate, mount your CA bundle in every database client and include ?sslmode=verify-full&sslrootcert=/etc/ssl/db/ca.pem in the database URL:
archestra:
extraVolumes:
- name: db-ca
secret:
secretName: db-ca-bundle
extraVolumeMounts:
- name: db-ca
mountPath: /etc/ssl/db
readOnly: true
The chart applies these mounts to web pods, worker pods, and the migration Job. Use a Secret or ConfigMap for the CA; the migration Job skips persistent-volume claims.
Amazon RDS IAM Authentication
On Amazon RDS or Aurora PostgreSQL, enable IAM database authentication so each new connection in the query pool, cache pool, LISTEN/NOTIFY client, and migration Job signs a fresh 15-minute IAM token instead of using a static password.
1. Prepare the PostgreSQL Database and IAM Role
Enable IAM database authentication on your RDS or Aurora PostgreSQL instance, then connect as the master user to configure the archestra application database, required extensions, and IAM database role.
-
New database: Switch to the
archestradatabase first sovectorandpg_are installed in the application database, then create the IAM-enabled role and grant database and schema privileges:trgm sql\c archestra CREATE EXTENSION IF NOT EXISTS vector; CREATE EXTENSION IF NOT EXISTS pg_trgm; CREATE USER archestra_iam WITH LOGIN; GRANT rds_iam TO archestra_iam; GRANT ALL PRIVILEGES ON DATABASE archestra TO archestra_iam; GRANT ALL ON SCHEMA public TO archestra_iam; -
Existing Archestra database: Grant
rds_directly to the existing database role that owns Archestra's tables and sequences (for example,iam archestra) and keep that username inARCHESTRA_and theDATABASE_ URL rds-db:connectpolicy so table ownership and migration DDL (ALTER TABLE) privileges remain intact:sql\c archestra CREATE EXTENSION IF NOT EXISTS vector; CREATE EXTENSION IF NOT EXISTS pg_trgm; GRANT rds_iam TO archestra;If you must switch an existing database from its original owner role (
archestra) to a separatearchestra_role, also transfer object ownership and privileges in theiam archestradatabase so both application DML and migration DDL succeed:sql\c archestra CREATE USER archestra_iam WITH LOGIN; GRANT rds_iam TO archestra_iam; GRANT ALL PRIVILEGES ON DATABASE archestra TO archestra_iam; GRANT ALL ON SCHEMA public TO archestra_iam; GRANT archestra TO archestra_iam; REASSIGN OWNED BY archestra TO archestra_iam; GRANT ALL PRIVILEGES ON ALL TABLES IN SCHEMA public TO archestra_iam; GRANT ALL PRIVILEGES ON ALL SEQUENCES IN SCHEMA public TO archestra_iam; GRANT ALL PRIVILEGES ON ALL FUNCTIONS IN SCHEMA public TO archestra_iam;
Attach an IAM policy to the pod's IAM role allowing rds-db:connect for the RDS resource ID (DbiResourceId) and database username:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": "rds-db:connect",
"Resource": "arn:aws:rds-db:eu-west-1:123456789012:dbuser:db-ABCDEFGHIJKLMNOPQRSTUVWX/archestra_iam"
}
]
}
2. Configure the Helm Chart on Amazon EKS
On Amazon EKS, set archestra.database.iamAuth.enabled: true and archestra.database.iamAuth.roleArn (for IRSA) or bind the role with EKS Pod Identity:
postgresql:
enabled: false
archestra:
database:
iamAuth:
enabled: true
# Annotates the platform ServiceAccount with eks.amazonaws.com/role-arn (omit when using EKS Pod Identity):
roleArn: arn:aws:iam::123456789012:role/archestra-rds-iam-role
# Optional override when the endpoint hostname does not embed the AWS region
# (TLS hostname verification checks the URL hostname against the server certificate,
# so the URL must use an endpoint name covered by the certificate):
# awsRegion: eu-west-1
env:
ARCHESTRA_DATABASE_URL: postgresql://archestra_iam@mydb.123456789012.eu-west-1.rds.amazonaws.com:5432/archestra?sslmode=verify-full
The platform image includes the AWS RDS global CA bundle at /etc/ssl/certs/cloud-database-ca-bundle.pem and verifies the server certificate and URL hostname on every connection (sslmode=no-verify is rejected under IAM auth); use an endpoint hostname covered by the server certificate (such as the RDS instance, cluster, or RDS Proxy endpoint, or a custom certificate covering your alias).
When you change the pod's cloud identity, such as roleArn or an iam.gke.io/gcp-service-account annotation, the migration Job runs before Helm updates the platform ServiceAccount. As a result, the Job still uses the old identity on that upgrade. To avoid this, set archestra.migrationJob.serviceAccount.create: true, and the Job uses its own ServiceAccount with the current annotations. It has the platform ServiceAccount's name plus -migrate. Before you set it, allow that account name in the role's trust policy, or in the Workload Identity binding on GKE.
3. Non-AWS Kubernetes Clusters (GKE, AKS, OpenShift, on-prem)
When running outside AWS with network connectivity to RDS on port 5432, you can authenticate using either OIDC federation or Kubernetes Secrets:
-
OIDC federation (recommended): Register your cluster's OIDC issuer in AWS IAM and enable
injectWebIdentityTokenVolumeso the chart mounts a projectedsts.amazonaws.comServiceAccount token volume and setsAWS_andROLE_ ARN AWS_on web, worker, and migration Job pods:WEB_ IDENTITY_ TOKEN_ FILE yamlpostgresql: enabled: false archestra: database: iamAuth: enabled: true roleArn: arn:aws:iam::123456789012:role/archestra-rds-iam-role injectWebIdentityTokenVolume: true env: ARCHESTRA_DATABASE_URL: postgresql://archestra_iam@mydb.123456789012.eu-west-1.rds.amazonaws.com:5432/archestra?sslmode=verify-full -
Secret-backed AWS credentials: Supply
AWS_,ACCESS_ KEY_ ID AWS_, and optionalSECRET_ ACCESS_ KEY AWS_throughSESSION_ TOKEN archestra.envFromSecrets(orarchestra.env, where the chart automatically storesAWS_andSECRET_ ACCESS_ KEY AWS_in a Kubernetes Secret).SESSION_ TOKEN
Connection Capacity
Set archestra.database.connectionBudget to the connections Archestra may use, and the chart divides it among pods. Each pod gets the budget divided by the peak pod count. During a rolling update, each Deployment can run its full new revision, an old revision that is still terminating, and the surge (25% by default, rounded up). With the web HPA maximum at 5 and 2 workers, that is 12 web and 5 worker pods, so 17 peak pods.
Each pod keeps 12 of its share for caching and notifications and uses the rest as its query pool. A share under 13 leaves no query connection, and the chart rejects that budget.
- Set PostgreSQL
max_about 25% above the budget. Each pod opens a few connections the budget does not count, and migrations and monitoring need their own. Pick a Size gives a budget for each size.connections - A fixed
archestra.database.poolMaxorARCHESTRA_overrides budget sizing. Finish one rollout before starting another.DATABASE_ POOL_ MAX - Archestra does not need a connection pooler. Behind PgBouncer in transaction mode, point
ARCHESTRA_at a direct connection.CHAT_ ACTIVE_ RUN_ NOTIFY_ DATABASE_ URL