Philosophy of mathematics is just one part of the connection between philosophy and mathematics. Many historical philosophers have been heavily influenced by mathematical thinking.

To give just a few examples from European history of philosophy: Spinoza took the geometric method (basically the axiomatic method) from Euclid. Kant is heavily drawing on geometry and the discovery of non-Euclidean geometry has been suggested as disproving him. If you look into Hegel's Encyclopedia of the Sciences, you will find a lengthy discussion of calculus.

In current analytic philosophy there are many other areas of overlap and cross-polination, for example in decision theory and metaphysics.