It's different based on each dependency and where you're at in the project lifecycle. If you're running as fast as you can to get off the runway before you crash, you have more of an incentive to bring it in and punt the problem down the road. But after that point passes it should be reevaluated. You should know every dependency in your code and why it's there.
For me, I find that it's a function of overall value to your code, number of issues you've had with it, overall size, ability to understand the code relative to what it's doing, and project health. Again, the calculus is different for everybody, but the important aspect is that you're consciously thinking about the utility of the dependency.
An ORM is hard to write, is often hard to read, but provides extreme value, and hopefully has a large community to quash issues and contribute to the project's health. To borrow an extreme example, left pad is not. Even if left pad is not in your core competency, that's not something that should be brought in because you can write your own. Even left pad may be subject to the event-stream style attack, and it being out of your control is risk you're bringing into your project.
Risk management is tricky, and you'll never get to 0 risk, but you should try to minimize it where you can reasonably do so. It's all about balance.