I investigated making this recently, however the (free) Google Street View API currently only returns 640x640 images, not the entire equirectangular image [1] that would be needed to create a photosphere for a VR app. So one can make a cool application like OP's here, but creating a panoramic experience would require multiple API calls and your own fancy stitching algorithm.