google doesn't like shipping GPL software to end users, and anyway, they had a lot of opinionated changes to make and plumb through, wakelocks and other things. No glibc, mininal other GPL stuff, it's not GNU/Linux, it's just Linux.
Display hardware for mobile SoCs often had no or poor x11 drivers during the early days of Android. x11 drivers are not easy to write, it's easier to do something else that has more specific requirements. Nowadays, I think you could probably do pretty well with glamor, but that wasn't a given back then. Limited capability mobile devices really don't need to run a distributed systems protocol that outputs to screens as a side effect, regardless of how cool distributed systems are.
Desktop apps running on a phone is not a good experience, and is unlikely to ever be a good experience. Applications need to be rebuilt for the context and if you're going to do that, x11 doesn't buy you much.
I'm not saying Android UI did a good job, but it made sense to start clean even if the results were poor.