The main reason why Scala is slow is because the compiler is poorly written. Take a look at the articles from Paul Phillips, one of the Typesafe cofounders who left the company in frustration, he's probably the person who knows scalac the best and he says the code base is basically hopeless.
Note that 1/3rd of Martin's presentation is about Dotty, and that's basically where he spends most of his time these days, he hardly contributes to Scala any more [1].