Google already has done it: https://www.google.com/maps/@40.7407861,-73.987388,348a,35y,...
Check out https://www.google.com/maps/place/San+Francisco,+CA/@37.7623... and rotate the image. Check out the trees to see the seams of the different pictures used to create the textures/models.
http://i.imgur.com/qCEE7cB.jpg
(updated screenshot to be from the same angle you have)
I don't have any idea what they are using there, but they do pull in a lot of aerial photos from the U.S. government. Those photos are (largely?) taken from piloted planes though.
If this is done in an automated way they are on to something pretty great.
Also to do this, is really really hard/lots of work. So kudos for that.