How hard/easy is it to switch servers and retain @accountname references (i.e. your username)?

When faced with a big list of servers to choose between, my hesitation is that I'll pick one now and realize later that another one is better. For example, I may someday want to host GNU social myself, but if I do so will I essentially be starting over (even if I can retain my data, will my connections still retain a relationship to the new server?).

A nice little diagram and/or technical details page would be much appreciated :)