I.e. that would be the appropriate thing to do if you're trying to prevent leakage of information i.e. enumeration of resources. But you should not return 401 for this still. A 404 is the appropriate response for pretending that "it's just not there" if you ask me. You can't return 404 when it's not there and a 403 when you have no access if enumeration is bad.
So for example, if you don't have access to say the settings of a repo you have access to, a 403 is OK. No use pretending with a 404, because we all know the settings are just a feature of Github.
However, pretending that a repo you don't have access to but exists isn't there with a 404 is appropriate because otherwise you could prove the existence of "superSecretRepo123" simply by guessing and getting a 403 instead of a 404.