LLM Metrics
Track LLM request latency, token usage, and billed spend across providers and models. See Labels for request and agent dimensions.
Metrics
| Metric | Type | Extra Labels | Measures |
|---|---|---|---|
llm_request_duration_seconds | Histogram | status_code | LLM request duration |
llm_tokens_total | Counter | type (input, output) | Tokens used. input excludes prompt-cache tokens. |
llm_cache_tokens_total | Counter | cache_type (read, write) | Prompt-cache tokens read from or written to the provider cache |
llm_token_usage | Histogram | Input plus output tokens per request | |
llm_cost_total | Counter | billing_mode, auth_method | Estimated list-price cost in USD. Needs model pricing. |
llm_cache_cost_total | Counter | Cost in USD of prompt-cache reads and writes | |
llm_cache_savings_total | Counter | USD saved by cache reads at the discounted price | |
llm_time_to_first_token_seconds | Histogram | Streaming: from the upstream call to the provider's first chunk | |
llm_time_to_first_byte_seconds | Histogram | Streaming: from request receipt to the first byte sent to the client | |
llm_tokens_per_second | Histogram | Output token throughput | |
llm_blocked_tools_total | Counter | Tool calls blocked by guardrails | |
agent_runs_total | Counter | external_agent_id | Unique runs, counted by the X-Archestra-Run-Id header |
llm_active_users | Gauge | window (24h, 7d) only | Distinct users with at least one LLM request in the window |
billing_mode is metered for per-token billing and subscription for flat-rate subscription credentials. Billed spend is sum(llm_cost_total{billing_mode="metered"}). auth_method is provider_key, virtual_key, passthrough_virtual_key, jwks, oauth_client_credentials, oauth_user, internal, or unknown.
llm_time_to_first_byte_seconds minus llm_time_to_first_token_seconds is the time Archestra spends before the upstream call: authentication and guardrails.
Every replica reports the same llm_active_users value, so aggregate it with max(), not sum(). ARCHESTRA_METRICS_ACTIVE_USERS_REFRESH_INTERVAL_MS sets the refresh interval; 0 turns the metric off.
LLM and MCP metrics carry trace exemplars, so a Grafana panel can link a data point to its trace. See Exemplars.