https://github.com/facebookincubator/create-react-app#conver...
Usually, you'd want to do this if you want do implement one of the "limitations" cited or if you had specific things to configure that had been abstracted away.
For example, the release notes mention webpack 2 changed their configuration. If you hadn't ejected yet there's nothing to do because they took care of it on the back end, if you had ejected you have to update your webpack config.
In my experience, setting it all up and getting dependencies is one of the pain points with react, so it's nice to get some sane defaults and get to writing your app. But it is nice to take the time to do everything manually at least once to see what's under the hood.