Physics is actually still pretty hard without the maths: if you imagine you have a new phenomena (e.g. a whole bunch of seemingly bizarre "elementary" particles a la hadrons before quarks) characterising them in a way that they can be modelled let alone modelling them is very hard.