what would a proper tool look like? I really want to write such a tool. But I can't formalize the product.
Although you can write your own 3D renderer inside Desmos...
Then think about what subdomain of mathematics you want to write software for. (9th grade polynomials, graphing, and algebra? Addition? Times tables?)
I'd also think about who you expect to use the software. (Just you? Teachers? Students?)
I was able to pick up that software and start making animations during a tour I got of the studio. The tools were designed to allow linking objects, paths, and animations in a very intuitive way.
See if you can find a demo of some of those weather animation user interfaces as there is probably good inspiration there.