AdminObservability

Observability

Monitor your instance in your existing stack. For individual requests and conversations inside Archestra, use Logs.

NeedSet up
Request rates, latency, costs, guardrails, and infrastructure healthPrometheus metrics.
Follow a model request or tool call across servicesOpenTelemetry traces and backend logs.
Administrative changes sent to a SIEMAudit log export.
Frontend sessions and page timingsReal User Monitoring.

Grafana Dashboards

Archestra publishes eight prebuilt Grafana dashboards. Core operational dashboards require Prometheus only.

DashboardKey VisualizationsData Sources
LLM Proxy (archestra-genai-observability)Proxy and unified model API outcomes, upstream and streaming latency, token rates, and prompt-cache usagePrometheus (optional Tempo for exemplars)
Costs & LimitsMetered and subscription spend, prompt-cache savings, limit rule evaluations, storage errors, and recent exhausted allowancesPrometheus
Agent RuntimeFleet-wide run lifecycle, provisioning latency, steering, completion delivery, boundary refusals, task health, and optional Kubernetes pod resourcesPrometheus (optional Kubernetes Prometheus with cAdvisor and kube-state-metrics)
Guardrails / OpenAPPAEmbedded OpenAPPA hook dispatches and latency by hook and decision, runtime boundary refusals and engine operationsPrometheus
MCP Monitoring (archestra-mcp-monitoring)MCP Gateway tool call rates, error counts, execution duration, payload sizes, and self-hosted server deployment statusPrometheus (optional Tempo for exemplars)
Knowledge Base Operations (archestra-rag-knowledge-base)Connector syncs, document ingestion, OCR, embedding batches, search latency, quote verification, and permission sync gapsPrometheus
Platform Health (archestra-application-metrics)HTTP throughput, Node.js memory and event loop lag, database connection pool, background task queue, and PostgreSQL healthPrometheus
Session Investigation (archestra-agent-sessions)Per-session timeline, LLM calls, MCP tool invocations, trace waterfalls, and correlated logs across Agent Runtime and external client sessionsTempo; optional Loki

Import or update the dashboards with a Grafana service account that can write dashboards and folders and read/query the selected data sources. Basic authentication also works with GRAFANA_USER and GRAFANA_PASS:

bash
GRAFANA_URL=https://grafana.example.com GRAFANA_TOKEN=glsa_xxx \
  bash <(curl -sL https://raw.githubusercontent.com/archestra-ai/archestra/main/platform/dev/grafana/install-dashboards.sh)

The installer is idempotent and upserts every dashboard into the Archestra folder (archestra-dashboards) using stable UIDs—including archestra-genai-observability for LLM Proxy, archestra-application-metrics for Platform Health, and archestra-agent-sessions for Session Investigation—so existing dashboard links stay intact. For external databases on Platform Health, pass --postgres-provider otel (or cloudsql, azure; defaults to helm for the Bitnami postgres_exporter sidecar). To link upstream latency spikes on LLM Proxy and MCP Monitoring to Tempo traces, enable Prometheus exemplars with --enable-feature=exemplar-storage.

To preselect data sources during import, pass their Grafana UIDs:

bash
GRAFANA_URL=https://grafana.example.com GRAFANA_TOKEN=glsa_xxx \
  ./install-dashboards.sh --metrics-datasource prometheus \
  --traces-datasource tempo --logs-datasource loki

--kubernetes-datasource selects a separate Prometheus source for optional Kubernetes rows and defaults to the metrics UID.

Data Sources and Metric Semantics

Model and agent filters on LLM Proxy affect upstream usage; logical request outcomes have their own filters. Costs & Limits has separate cost and limit filters, with Cost breakdown selecting the grouping dimension. Scrape job selects a Prometheus target group; leave All selected unless you need one group of processes.

Each dashboard scopes its optional data sources and aggregates multi-replica metrics according to how Archestra exports them:

  • Session timelines vs. runtime fleet health: Session Investigation inspects individual conversations—working for both Agent Runtime runs and external client sessions that propagate session context via X-Archestra-Session-Id or X-Appa-Session-ID—and adds trace waterfalls and span logs when Tempo and Loki are configured. Agent Runtime tracks fleet, workload, and pod health rather than a per-session event timeline.
  • Optional Kubernetes row on Agent Runtime: The Kubernetes resource panels require cAdvisor and kube-state-metrics. Scrape kubelet /metrics/cadvisor and the kube-state-metrics endpoint into that source, preserving cluster, namespace, pod, and container labels. Verify kube_pod_status_phase and container_memory_working_set_bytes return runtime pod series before expanding the row. The panels use an independent Kubernetes Prometheus data source variable and Cluster, Namespace, and Runtime pod name regex filters so you can target your runtime pods without editing queries.
  • Historical limit detail vs. live headroom: On Costs & Limits, llm_limit_last_exceeded_timestamp_seconds, llm_limit_last_exceeded_threshold, and llm_limit_last_exceeded_usage report process-local observations from the last 24 hours at the time a check was exhausted, not live budget headroom.
  • Replica-wide health gauges: Fleet health reported identically by every replica (such as agent_runtime_health_tasks, agent_runtime_health_age_seconds) aggregate with max across replicas, never sum.
  • Guardrails and OpenAPPA metrics: Hook dispatches and engine operations come from the same Archestra /metrics endpoint and use the dashboard's Metrics and Scrape job selections. See the metric reference for their distinct counting boundaries.

Real User Monitoring (RUM)

Frontend monitoring requires an Enterprise license.

  1. Set the collector endpoint in your environment:
    bash
    ARCHESTRA_RUM_EXPORTER_OTLP_ENDPOINT=http://otel-collector:4318
    
  2. If your collector requires authentication, set ARCHESTRA_RUM_EXPORTER_OTLP_AUTH_BEARER, or both ARCHESTRA_RUM_EXPORTER_OTLP_AUTH_USERNAME and ARCHESTRA_RUM_EXPORTER_OTLP_AUTH_PASSWORD.
  3. Restart the backend to initialize the telemetry pipeline.

Browser events route securely through the Archestra backend and arrive at your collector under the service name Archestra Web. Events never contain prompt text, messages, email addresses, or secrets.

Explore