> Perhaps if ES4 landed, fewer people would need complex build tools like Babel, Webpack and Typescript.
I'd pose it the other way: if stuff like Babel existed at the time, folks might have actually written es4 because they could do so without breaking backwards compatibility. And most of the value in a static type system is the ability to flag errors before running the program, so I don't think there's anything you can add to the browser that would have obviated an offline tool for that.
Frankly, I think we'd be better off if we started getting a lot more selective about what gets added to browsers. Adding more and more features to browser's js implementations is silly when most folks are using ahead of time build steps to get access to those features without the browser's help. And continuing to add things is a great way to ensure that they'll keep right on using those tools, so they don't have to worry (as much) about who supports what. And it just makes the barrier of entry for building browsers higher and higher.
There's more utility when native browser support can do something a compiler can't. Things like reducing code size for really widely used functionality, enabling technologies that can't be bootsraped through developer tools (e.g. WebRTC). I'd like to see additional growth of the web platform be focused on areas that can't adequately be addressed by separate tools.