Many people become conservative as they get older, so maybe he is just comparing himself to the norm.
Winston Churchill once said, "If a man isn't a liberal in his twenties, has has no heart. And, if he isn't a conservative in his forties, he has no brain."