You can't really solve this with sessions. Even when you use chatgpt in the web console the entire history of the chat is submitted each time you generate a new response. The history is the only way these models know what you've been talking about. The history _is_ the session. I suppose there will be ways around this eventually, though. For this particular application it might suffice for slightly large datasets to just send a sample.

I think the approach here is kind of dumb, though. It works significantly better to send the AI meta-data and let it write queries (with a limit on result count) iteratively to answer your questions.