Yes: not all code on GitHub is licensed in a way that lets you use it at all. People focus on GPL as if that were the tough case; but, in addition to code (like mine) under AGPL (which you need to not use in a product that exposes similar functionality to end users) there is code that is merely published under "shared source" licenses (so you can look, but not touch) and even literally code that is stolen and leaked from the internals of companies--including Microsoft!... this code often gets taken down later, but it isn't always noticed and either way: it is now part of Copilot :/--that, if you use this mechanism, could end up in your codebase.
If you publish the code anywhere, potentially. You could be (unknowingly) violating the original license if the code was copied verbatim from another source.
How much of a concern this is depends heavily on what the original source was.
And the problem with copilot is that you have no way of knowing. If it changes even a little bit of the code, it's basically ungoogleable but still potentially in violation.
Distributing binaries to third parties is enough to trigger a license violation. For internal corporate tools, it would be less of an issue as "distribution" hasn't happened.