It's not just adjustment, there's a decent amount of evidence it's just bad for your health in general, apart from the issues when coming back to earth. Stuff like your circulation also gets messed up.
[1] - https://journals.physiology.org/doi/full/10.1152/japplphysio...