Wolfram Alpha was (at least initially) famously bad* at actually parsing natural language. For math, it was much easier to enter raw Mathematica expression into Wolfram Alpha than structuring the question in natural language. LLMs should give it a boost, by simplifying the kind of the natural language input into the limited forms parsable by Wolfram Alpha.

*: And I say this as a paying customer of Mathematica on my computer and Wolfram Alpha on my phone.