I was the CTO of my startup and was a senior architect looking across multiple domains in my previous job. Now, I'm more a specialist, looking at one single thing while it's expected that someone has the responsibility to solve a different problem.
To me, that's the biggest career impact. In our startup, everyone did everything. Need to hotfix something in production in the morning and then head off to a investor pitch in the afternoon? Come back and rework on our marketing message? Yes. Yes and Yes.
Now, I'm just a dev. Not the marketing guy. Not the finance guy. That's someone else's responsibility. Am I being paid less? Not really.. But am I having lesser responsibilities? Definitely. Is that a career downturn or not? I don't know, but I do see others who joined the same company from a more successful startup being "higher up" when it comes to responsibilities and job titles.