Vadim Larin — April 21, 2026

Hi team! I'd like to discuss one idea, I don't know how much it is relevant for you now at all, but to me it looks like a pretty natural continuation of what you already have in guardrails. It is about something like policy dry run / what-if for tool policies: so it would be possible to run tool call scenarios through current or proposed policies and see what would happen with them and for what reason. I see two modes here: 1. synthetic scenarios - more for new clients, I don't know how your sales are organized, but potentially it could be useful there. 2. running on historical cases - for already existing clients, so before changing a policy they could see how it would affect their real scenarios. It seems to me that in a light version this should fit pretty well into the current architecture. If it's interesting, I'd be glad to discuss it more and take implementation on myself.

9 replies

Matvey Kukuy (archestra team) — April 21, 2026

@user @user @user join the discussion 😉

Ildar Iskhakov (archestra team) — April 21, 2026

Nice idea! Do you mean dry runs without LLMs? Just determined sequences of tool calls? How do you think it should look like from the UI/UX point

Ildar Iskhakov (archestra team) — April 21, 2026

BTW we had something slightly similar with evals, see www.youtube.com/watch?v=o7L6_MyOCt0 and github.com/archestra-ai/archestra/tree/evals

Innokentii Konstantinov (archestra team) — April 21, 2026

We already store existing conversations, so dry-running policies against them should work without LLM?

Vadim Larin — April 21, 2026

Actually at first I was thinking with LLMs, because I was looking at it from product side, like a user can have a question, will Archestra stop the agent because of policy or not (I don't know what is in users heads of course, but I was starting from the worst case). If not orienting on that, then of course it can be done without LLMs too. I didn't see evals before, it is really very similar, I will look into it, thanks. About how it could look in UI/UX, I would like to go deeper and think more, including looking at how similar mechanics are done in existing projects, like MS what if and so on.

Innokentii Konstantinov (archestra team) — April 22, 2026

Once you have an initial design, feel free to open an issue with the proposal! I'm also will be happy to have a call with you to discuss the design 🙂

Vadim Larin — April 22, 2026

Yes, happy to sync on a call before I open the issue. If that works for you, could you send me your calendly in dm?

Vadim Larin — April 22, 2026

@user I decided to open the issue first to save time 🙂

Innokentii Konstantinov (archestra team) — April 23, 2026

Thanks @user! Send my calendly to the DM, let's book something next week?