Most people are mediocre in their early years and get filtered out, even though a good percentage of them had the underlying talent and skill, and just never got lucky. Most of the ones who succeed once aren't going to succeed again, not because they're actually getting dumber but because non-success is the norm in this world.
Wreckage by success can happen, but it usually requires some odious externality that the person is unable to handle. You see this in the arts and in the literary world, because authors start getting invited to the parties that become necessary to make connections (for reviews, resources, and future publication) and then turn into coke zombies who can't write anymore. You don't see it as much with math professors, because there is no incentive pulling them into that scene--with the latter, it's just regular regression to the mean.