#general
hello, currently experimenting archestra for our organization as centralized gateway for agent, mcp and llm proxies.
We are using Vertex AI and wondering how to make archestra to list Sonnet 5 in model choice?
because using our mounted Vertex SA, archestra only shows all Gemini model.
Using the same SA and testing it locally using curl, it does respond correctly so i believe it should not be the settings from model garden or access issue.
--impersonate-service-account="$SA")
curl -i -X POST \
"<https://aiplatform.googleapis.com/v1/projects/company/locations/global/publishers/anthropic/models/claude-sonnet-5:rawPredict>" \
-H "Authorization: Bearer $TOKEN" \
-H "Content-Type: application/json" \
-d '{
"anthropic_version": "vertex-2023-10-16",
"max_tokens": 32,
"messages": [
{
"role": "user",
"content": "Reply only with OK"
}
]
}'
WARNING: This command is using service account impersonation. All API calls will be executed as [platform-eng-vertex-ai-service@company.iam.gserviceaccount.com].
HTTP/2 200
.
.
.
{"model":"claude-sonnet-5","id":"msg_vrtx_011Cek8kRJKHbBqjfLRmyT9H","type":"message","role":"assistant","content":[{"type":"text","text":"OK"}],"stop_reason":"end_turn","stop_sequence":null,"stop_details":null,"usage":{"input_tokens":13,"cache_creation_input_tokens":0,"cache_read_input_tokens":0,"cache_creation":{"ephemeral_5m_input_tokens":0,"ephemeral_1h_input_tokens":0},"output_tokens":4,"output_tokens_details":{"thinking_tokens":0}}}%```👀1
7 repliesRead-only live mirror of Archestra.AI Slack
👋Join the discussion withAI enthusiasts!hello, currently experimenting archestra for our organization as centralized gateway for agent, mcp and llm proxies.
We are using Vertex AI and wondering how to make archestra to list Sonnet 5 in model choice?
because using our mounted Vertex SA, archestra only shows all Gemini model.
Using the same SA and testing it locally using curl, it does respond correctly so i believe it should not be the settings from model garden or access issue.
--impersonate-service-account="$SA")
curl -i -X POST \
"<https://aiplatform.googleapis.com/v1/projects/company/locations/global/publishers/anthropic/models/claude-sonnet-5:rawPredict>" \
-H "Authorization: Bearer $TOKEN" \
-H "Content-Type: application/json" \
-d '{
"anthropic_version": "vertex-2023-10-16",
"max_tokens": 32,
"messages": [
{
"role": "user",
"content": "Reply only with OK"
}
]
}'
WARNING: This command is using service account impersonation. All API calls will be executed as [platform-eng-vertex-ai-service@company.iam.gserviceaccount.com].
HTTP/2 200
.
.
.
{"model":"claude-sonnet-5","id":"msg_vrtx_011Cek8kRJKHbBqjfLRmyT9H","type":"message","role":"assistant","content":[{"type":"text","text":"OK"}],"stop_reason":"end_turn","stop_sequence":null,"stop_details":null,"usage":{"input_tokens":13,"cache_creation_input_tokens":0,"cache_read_input_tokens":0,"cache_creation":{"ephemeral_5m_input_tokens":0,"ephemeral_1h_input_tokens":0},"output_tokens":4,"output_tokens_details":{"thinking_tokens":0}}}%```👀1
7 replies
👀1
sorry for late reply, we have upgraded to 1.3.59 and still no sonnet model detected using same service account
any way to debug why the model not showed up?
hi there 👋 and if you hit GET /models (or the applicable equivalent) against Vertex’s API, with the same service account, outside of Archestra, it does indeed show up?
I know with Vertex you need to enable each specific available model in their UI.
also there is some specific config that needs to be set specific to consuming Anthropic models via Vertex
in particular
ARCHESTRA_ANTHROPIC_VERTEX_AI_ENABLED, ARCHESTRA_ANTHROPIC_VERTEX_AI_PROJECT (and optionally ARCHESTRA_ANTHROPIC_VERTEX_AI_LOCATION) - have you configured these after upgrading? (docs)oh wait a minute, I just realized that config has "
ANTHROPIC" now. completely skipped that, my badokay let me try that
this one resolved, thanks