Yes, that is a problem. If your app/site supports multiple login options you may have users who "lose" their account (I'm one of them).

However your solution... doesn't work. Apple requires apps that support social app to have sign in with apple. Google has similar requirements (because your app better be multi-platform in this day in age). Facebook is way popular. So that's three right there.