Auto Routing

Send auto as the model name, and Archestra picks a fast, normal, or smart model for each agent session so that average daily spend stays near your target. Easy sessions run on the cheap model, hard ones on the smart model. The app code does not change.

Set It Up

Auto routing works on the Model Router only. Native provider routes answer auto with a 400 error.

1. Add a Jev API key under Model Providers. Jev scores each new session.
2. Go to LLM → Auto Routing, pick the fast, normal, and smart models, and set a daily budget in USD.
3. Turn auto routing on.
4. Send auto as the model from an app that uses a virtual key.

The three models can come from different providers. auto appears in a key's model list only when the key can reach all three.

How Sessions Are Routed

A session is routed once, on its first request, and keeps that model for its whole life.

• Session identity: the session id if the client sends one. Otherwise the virtual key plus the first user message, so identical first messages under one key share a model.
• Budget: the daily budget is an average target, not a cap. Spend on a given day can go above or below it. Hard caps come from limits.
• Plans: a background job builds a routing plan from past sessions. Changing the settings triggers a new plan within about a minute, and every organization is re-planned at least daily. Each plan is versioned.

Learning From Past Sessions

A new plan needs about 1000 finished sessions. Past traffic can supply them instead of waiting for auto traffic.

1. Find past sessions. Runs on its own when you first enable auto routing, or from the Overview page. It reads the last 35 days of proxy logs and counts the sessions a plan can learn from, by API and client. It costs nothing.
2. Score them. You start this. Archestra sends the opening prompt of a sample of those sessions to the Jev scorer, up to the spending limit you set (calls already in flight can take it a few calls over), then fits a plan.

Only sessions with a client-supplied session id that began inside the window are used. Sessions whose content was not logged or is end-to-end encrypted are skipped. After a scorer change the page offers to score the same sessions again.

When It Falls Back to the Normal Model

Archestra uses the normal model when it cannot make a better choice:

• Cold start: fewer than the minimum number of finished sessions exist. See ARCHESTRA_AUTO_ROUTING_MIN_SESSIONS.
• No plan yet, or the plan is stale because the three models or their prices changed.
• Scorer error or timeout. See ARCHESTRA_AUTO_ROUTING_SCORER_TIMEOUT_MS.
• No user text in the request to score.

The session keeps that fallback until it ends.

Savings

The page shows estimated savings against the smart model: what the same tokens would have cost on it. Treat the number as an estimate.

Long-Context Prices

Some models charge more for long prompts. On a model's settings, add a threshold and four long-context prices. Prompts above the threshold use them. This applies to all traffic of that model, not only auto routing.

What to Know

• Routing decisions are kept for 90 days. After that, the same session is routed again as a new one.
• Every request still goes through your existing cost limits and guardrails.
• Without a Jev key for the organization, every session uses the normal model.