I've been working with es6 for two years now and never even heard of this happening to someone. Given the huge number of people who use babeljs, if there were to be a bug in it, it'd be reported and fixed rather quickly.

Yes, transpiling es6 -> es5 is adding another layer of complexity to your codebase, another thing that could potentially go wrong, but the reality is that the development gains we get from being able to use es6 are far more important than a small chance of a problem. It works well enough that you don't even think about it after leaving webpack in watch mode.