Edit: Please stop the down votes, just an electrical engineer here, with one basic course in Probability and Stat. :)
Another way of thinking about it (described in Wasserman's book) is that statistics is the inverse problem of probability.
Probability theory asks: given a process, what does its data look like? Statistics asks: given data, what process might have generated it?
Excellent summary, thank you.
Making inferences and predictions from data, in the presence of uncertainty.
Analysis of the properties of procedures for doing the above.
If you want examples that avoid the feel of just "curve fitting" (assume you mean something like "inferring parameters given noisy observations of them") -- maybe look at models involving latent variables. Bayesian statistics has quite a few interesting examples.
Probability is at the heart of the project: frequencies that summarize reoccurring data. Instead of storing a reoccurring pattern multiple times, we just store it once and record how often it has occurred.
On the basic materials level, density functional theory is the current gold standard, and it's extremely statistics heavy.
At the systems and architecture levels, you may be right, though.
I think you can agree now that your original observation of statistics as "glorified curve fitting" as a bit naive.
Basically, you count things and compare that to how many things you think you should have counted given your assumptions.