I'm in the process of building one too. Handing off SQL queries to LLMs feels like a no-brainer.
I went pretty simple, used OpenAI agent sdk and built a couple of tools like “run_query” with read only connection. Initially I also had a tool for getting the join path from A to B, but the context I wrote out was sufficient.
I think main challenge with this agent is how to keep the context up to date.