On the first request in a conversation, we pick the model/provider based on the request, and then keep that route sticky for the rest of that conversation. So we’re not bouncing a large context between providers every turn and constantly blowing away the cache. If that provider fails and we have to fail over, then yeah, the next provider has to prefill the context again. There’s no real way around that.

If you need to handle a separate task mid-conversation, the cleaner pattern with Smart Groups is to spin up a sub-agent with its own context. That request gets evaluated separately, and then that route stays sticky for that task too.

Also, smart routing is only one part of Relay. You don’t actually need to use dynamic routing at all to get value from the rest of the gateway, but it is a cool offering.