And if I'm wrong, I apologize and would gladly hear the correct answer.
For these reasons "time slows down for you when you move at relativistic speeds" is really a bad way to think of it (and not how the math of the theory actually works). It's much more proper to say things in the form "To <observer a> it appeared as if the clock of <observer b> was <sped up|slowed down>". E.g.:
To the traveler it appeared as if the clock of Alpha Centauri was sped up. To Alpha Centauri it appeared as if the clock of the traveler was slowed down.
So if you travel to Centauri from Earth, the fastest you get can get there is 5 years as seen from Earth but to the actual spaceship, it could perceive any arbitrarily small change in time.