It's going to depend a lot on how the library is designed and how much of its functionality I need.
I am very skeptical about adopting a library because of all the reasons you listed but I am quite happy to adopt one when I see it's been stable and designed well.
What I'm looking for is a well tested library with a clean interface that doesn't look impenetrable when I look at its source. If the source looks a bit like something I might write, then at least I feel confident I can figure out an issue on my own when I hit a wall.
It's a judgement call, it just depends on where you draw the line. You're not writing ALL your code with just vanilla Typescript.
I use Ruby, I don't write all my code in vanilla Ruby.
But I definitely agree that blindly pulling in a library just because it looks like it makes your problem simpler RIGHT NOW, is a bad instinct.