Math has more aspects than just logical deduction via mechanical rules. Math also has an aesthetic aspect that guides people to find elegant, powerful solutions within the space defined by the mechanical rules. There may be many paths of deduction from point A to point B, which are all mechanically equally valid. But from the human point of view, they have different value. Some will be simple and easy to understand; others will rely on ideas from one or another realm of math, making them friendly to people who understand those ideas. Some will suggest, to the human brain, analogies to other problems. The mathematical validity of the argument is judged according to whether it correctly follows the mechanical rules, but all other aspects are judged by aesthetics and intuition and ultimately by how the solution is received and utilized by other mathematicians.

If the only aspect of mathematics that you bring into programming is logical deduction by mechanical rules, then I doubt it will help, except for rare cases where you prove or disprove the correctness of code. If, on the other hand, you bring over the aesthetic concern, the drive to make painfully difficult ideas more beautiful (ergonomic) for human brains, then it will help you make your code simpler, clearer, and easier for others to work with.

Is this really a common thing? How can you try to implement something without first having had thought of the solution?

It's common, and as you can imagine, it doesn't lead to good outcomes. When people start by coding first, it's so much work they tend to stop at their first solution, no matter how ugly it is. When people start by solving the abstract problem first (at a whiteboard, say) they look at their first solution and think, "I bet I can make this simpler so it's easier to code." The difficulty of coding motivates a bad solution if you start with code and a good solution if you write the code last.