"Haven't been happier"?

If I say "it's all downhill from here" does that mean things are getting better or worse?