You're not alone. But you're also missing an important point. You often use libraries without fully understanding their code, don't you?

You have to. You usually don't have the time or luxury to dig in and understand everything when you have a specific task to complete. Unless you're refactoring some old code where you try and make sure that you understand everything relevant to that codebase. Or you can just wait until it breaks...

Things like automated testing and proper documentation try to help in this regard, but they're useless if they're not kept up-to-date with the code and they have their own set of problems.