That's not necessarily the case. Another possibility is that when it seems like time is passing more rapidly, it's because the events occurring have a lot of similarity to previously experienced events—so the brain compresses them. It fits the data in the same way: more things are novel when you're younger, less can be compressed. And, it fits exceptions to this rule that occur later in life (in my own experience anyway).
To get around it, I think having less abstract experiences would be a good approach. If you're thinking in a highly abstract manner, maybe all your mornings might be identical and can be compressed into the unit 'morning.' In particular, I mean mindfulness, through meditation. Or you could just throw yourself into something new.