Browsers should compete on performance. As long as dart2js outputs efficient JavaScript (current benchmarks: https://www.dartlang.org/performance) so that Dart code runs well on browsers without the VM, I think it's fine if some browsers run Dart more efficiently than others. Consider Dart VM as an accelerated way to run Dart code which otherwise runs fine in other browsers.

To be very clear, Dart runs across the modern web when you compile it to JavaScript, so it's not a language that works only in one browser.