Include more of your dependencies in the repo and build them aa part of the ordinary build process. Now a package manager does not need to get involved.
You don't want to depend on a third-party hosting the code, so you need to copy it, and pin it to a specific version. You might also need to patch it since you'll be using it and most likely will run into problems with it.
Using third-party code means taking ownership of all the problems one might encounter when trying to use it in your project, so you might as well just adapt it for it to work with your tools and processes.
If you use a modular system this is essentially just maintaining a fork.