IanCal said it all. But for alternative approaches that also use LLM (with miniKanren) you can check https://arxiv.org/abs/1809.02840
Later edit: you have a nice way to generate unstructured text, and now you want to go and bolt a structured representation on top. So now you have to do all the hard work by hand, again, to write the structured representation. That sounds like a regression.
I'll have to make time to read your paper, thanks for linking it.