In my experience doing lots of computational science, the analytical/symbolic part of the problem solving is typically done separately, as it really needs to be done once, and (mostly) by hand so that you understand the problem and solution better.
Then, once you've gotten the problem and solution framed the way you like, you start implementing the numerical solution of your problem.
There are a few exceptions, particularly when things do get hairy. For example one of my colleagues has created a radial basis function finite difference solver[1]. In some of the modules, it manipulates the RBFs using SymPy and then compiles that to C which is then executed by Cython as part of the solution process.