This is great, congrats on open-sourcing. The (now $20/user/mo we've been told) Slack AI is really not all that good in my experience, so it shouldn't be hard for you to do better by just spending a little extra $ in API credits.

That said, the Slack API limits third party apps from accessing chat history of private messages, so any implementations like this are artificially limited. In addition, if you're hosting something for your company to use, it would realistically need to take into account user permissions to make sure the AI isn't sending them data they shouldn't have access to. That could be tricky to implement correctly.