It's worth some concern.

As the copay attack shows, that everyone uses these libraries doesn't let us rest assured. There are virtually zero eyeballs on the code of transitive dependencies because you would have to extract tarballs to read code. And attacks can be extremely targeted. The copay attack was only discovered because of a deprecation warning in the attacker's code that someone reported.