1. Not all users are comfortable with trusting a third party in order to use my app. Some people actively mistrust and avoid megacorps like Google and FB. They should only have to trust you, not a chain of companies.
2. Writing code to integrate numerous external API calls is a comparable effort to just doing it all custom.
3. The UX of jumping to a third party dialog to log in and then back in to the original app is jarring. "What just happened?"
4. It introduces more potential points of failure with less control over being able to deal with such issues. If the third party services or APIs fail, or a user can't access those domains for some reason, tough luck.
5. Someone else owning your app's user records is troublesome. You still need to have your own user records for things like session state, roles and authorization. You have to keep your user records synchronized with theirs.
6. Users sometimes do not want to link their accounts on other services to your app - they prefer separate identities. When your Google account's avatar appears in an app that has nothing to do with Google it can be annoying, or even perceived as a privacy violation.
7. User authentication involves a standardized, conventional set of practices and code that are well known and not hard to implement.
8. If the third party service you put at the core of your architecture decided to shut you down or compete with you, or they shut down themselves, you'd be in big trouble.
9. Sooner or later you will pay for this service. The more successful your app is, the more you will pay. If you do it yourself, there's no cost beyond your regular hosting costs.
10. KISS - Keep It Simple, Silly. Don't add unnecessary dependencies and complexity.