I did similar experiments with GPT4 and Soulver[1], though I "tuned" it by teaching it Soulver interactively before continuing to prompt. It can then be used in the same way. My primary goal was to add basic calculation capabilities to GPT, that are: a) guaranteed to halt (all Soulver sheets are functions) and b) readable by both GPT and the user (a Python program may be too dev-oriented for normal people).
It worked quite well. To well almost: I started a meta-conversation where I asked another GPT4 instance to come up with conversations SoulverGPT could have with a user where the addition of solving is beneficial. This worked, and eventually even found a bug in Soulver - essentially fuzzing the language.