You go to sleep every night. When you wake up the next day, how do you know you're the same person as yesterday? What does that mean?
Now consider if you didn't have memories of the past. What would make you "you"? The body would be the same but your amnesia would change things, eh?
Now what if you knew next week you would lose your memories for sure, but this week you can go on an amazing vacation. Does it matter where you go? It matters to your current self but to your future self.
Now think about sacrificing pleasure now so you can live a longer life. Or feel good about your life at 80.
Now think about the afterlife. What if you are reincarnated with no memories of the past? We already kind of established that "reincarnated you" wouldn't care about current you. But will current you care about reincarnated you?
Consider this: if a "future you" doesn't remember being "current you" how are they really different than a random person with a similar body who also has amnesia?
In short, long term, is it all about the memories?
What is the self preservation instinct about?
Finally the big one... if there were no conscious observers at all, in what sense would anything exist? That is, if I described to you a universe that you can never detect or deduce must exist from any observations in this one, how would saying "that universe EXISTS be different than saying it DOESN'T EXIST? It seems the word itself requires concious observers. Thus, the concept of consciousness and existence may be two sides of the same amazing conundrum.