Except we can't, because standards dictate stupid this. My example: GPS data gives altitude above the spheroid; KML requires altitude above the geoid. This means that even 2D GPS coordinates that were not taken at sea level cannot be used directly in KML and hence Google Maps.

Converting this requires projecting spheroid data into 3-space and reprojecting it onto the geoid (which is a complex geometrical shape). Neither of these steps is trivial. Of course it's easy if you've got proj4; but this kind of stuff just does not ship with Android.