Seneca cared about using one's time wisely in order to maximise the meaningful accomplishments and noble conduct of one's life. Whether time is illusory or real is irrelevant to his philosophy. One can only accomplish so much in life, and much of life is wasted because of poor choices of how to spend one's time. I've arguably wasted some of mine by writing this.

But this is assuming that having the most memories at the end is the goal of life. But your memories are also gone when you die.

The only thing that remains after you die are memories in others' minds about you. The bigger your achievements the longer the memories about you will last in the collective mind. Maybe that's a better life goal.