1) Science does not necessarily involve experimentation. 2) String theory has attempted many times at experimental verification. 3) Math are more experimental than most people think.
That is not to say, however, that all important work must be scientific. There is a certain value to exploratory work, which can guide the direction of experiments in such a way as "if we could get experiment X then we could turn hypothesis Y into a testable hypothesis." In my opinion that is important work. You can see this in fields like psychology and such where the systems are just too complex to be easily testable (even medicine to some degree). But if we don't keep pursuing we will never get there.
EG: We already consider "Dark Energy" a sort of made-up assumption to justify a model which, after recent observations was considered falsified. So, instead of focusing on others we decided to keep using it, we created a not-falsifiable nor testable assumption (dark energy indeed) and kept going with it, even if it violates everything Popper teach us.
Honestly it seems to me that, especially in physics, too many people are just too much mentally involved in their own theory that they have an hard time just dropping it when it doesn't seem to work anymore.
Most argue that the discovery of the neutrino saved the fundamental conservation laws, but we should never lose sight of the fact that we still don't detect them in a very large percentage of events we record (we say simply that they are there, but interact weakly).
One important measure of whether a hypothesis is likely going to be accepted as true in the future is that as we build better and more sensitive detectors we detect more outcomes favorable to the hypothesis. This is the way smart money bets when data are still hard to come by (see 'climate change').
> This was a non-scientific hypothesis at the time, but it told scientists the direction they needed to pursue in order to get to the point where the hypothesis could become scientific.
"What is Philosophy of Physics?"
Not by definition, but by one specific definition -- from the popular epistemology by Popper (who also coined the relevant term). Others (Feyerabend for one) have pointed out that actual science (in practice) doesn't work that way.