Google created an Android userspace that requires Google services to do anything meaningful. This violates everything Linux stands for.
In fact they would like to because a majority of their users install it. But there's a lot of legal strings attached to it.
You'll miss features like push notifications, Google accounts outside of the browser, an app store and various other features.
But there's no reason the open source community couldn't add these, a package repo/app store equivalent and push messaging would probably be the deal breakers for me as a phone where chat apps don't work isn't useful to me.
The binary blobs to get driver support are also a problem, but it's also true in desktop linux, see for example Nvidia's graphics card driver.
Legally though, Linus Torvalds gets to define what Linux is because he owns that name.