Sure, JavaScript is fast enough for a lot of things.
But it's also too slow for many things that reasonable people would like to do on the web.
See Mozilla's pdf.js: it makes perfect sense to have a PDF viewer written in JavaScript (and not a blob of insecure C++ code in the form of the plugin).
pdf.js would not be possible at all "long, long time ago", even though JS was adequate for your use cases at that time.
pdf.js is barely possible today - it works for simple documents but its performance on more complex documents is still so much worse than what C++ plugin gives that many people go back to plugins.
Performance is one of the most important competitive aspects of software. Given 2 products with similar features, people go for the faster, which is a big reason why Chrome was so successful.
We're still far away from making JS speed covering all the important use cases.
Until JavaScript can decode mp4 in JavaScript (at speeds not much slower than optimized C++), JavaScript is not fast enough.