Think of it the other way around: Dart was created for the sake of Flutter.
You seem to think that the framework designers at Google just decided to pick a niche language called "Dart" to write Flutter in when they could have picked Javascript to write Flutter in.
No.
They decided to make the design of Flutter as a framework so clean and pure that they would have to work backwards to figure out what language it would need to be written in and then they created Dart.
You're not supposed to use Dart for any purpose other than Flutter -- that's not a drawback to learning Dart -- that's the whole point. An entire team at Google literally created a language to make it easier for you as a developer to express yourself in this domain, and you think that's a drawback? That's like thinking being given a ladder is a drawback when you want to get up on the roof.
Sure, you could just assemble a big pile of rubble and scramble up it to get on the roof, and it is true that you might be able to later use the rubble for other purposes (whereas a ladder is built for a specific purpose) -- but that does not make the ladder the more limiting tool.
If there is one repeated comment I hear about Flutter it is "wow, it seems like such superior technology, I wonder why it isn't more widely used".
To which the answer I keep hearing is "it is only superior for doing the thing it is built for -- it isn't superior for other things."
Those who would give that answer consider Javascript, on the other hand, an (inferior) tool not only for this problem, but also for other problems, so it is the more generalizable tool and therefore it makes more sense to use Javascript.
The argument seems to be that at least if you're using the same crappy tool for everything then you can claim the efficiency of never having to change your tool.
"Only a fool would use the best tool for the job", is the implication of those who disparage Dart.