This is not, as you think, simple. There are no margins in commoditized devices. If they all trend toward being identical and low cost, all partners will find the most exceedingly clever ways to differentiate from each other or boost margins. Software is a prime target.
You also speak of "letting" vendors ship crapware. Let? This is an open platform. Google only has control if vendors desire the Google goodies that cost a little bit of money. Also, how much leverage do they have exactly against a carrier like Verizon, which is their primary sales channel? Not as much as you would think. As this article points out, forking Android (especially to Samsung) is becoming an increasingly viable alternative to playing ball with Google, and the consequences would be pretty disastrous if they were to do that.
This hedge of doing their own hardware represents yet another challenge with your shockingly simple solution: that supply chain management in this market is really difficult, especially if you have some killer ideas in mind for new kinds of hardware (i.e., using materials other than plastic, new experimental displays, sensors, etc.) How quickly did Nexus 4's sell out? It's very possible that demand was strong, but the double-glass enclosure probably proved to be more of a headache than originally conceived to manufacture.
So it's not simple. This market is difficult, fraught with trust issues and risks in doing it all yourself.