Isn't this what the Assistants API is meant for? Honestly not sure, I haven't used it before but their documentation seems to suggest you can set up the assistant to already have this context, then just send API commands to it without said context.