That's not true as a blanket statement. If it weren't for gps metadata, photogrammetry[1] would be very difficult. I am able to make my own "google earth" type maps with a much higher degree of detail because I can fly my drone 100 feet over the land and take thousands of gps-stamped pics vs outdated google maps satellite imagery and then get them all stitched together to make a large GeoTIFF. All the processing is done on my PC using opensource WebODM[2], which can then be imported into opensource QGIS where accurate elevation data can be added. It also creates textured 3d models of the land and buildings which can then be imported into opensource 3d apps like Blender. It's been a huge boost.
I also like being able to show all of my various images on a map to see where they were taken.
[1] https://en.wikipedia.org/wiki/Photogrammetry
[2] WebODM quick 3D demo: https://www.youtube.com/watch?v=G3pMJR-L0Gk
I don't upload my photos anywhere public. I keep metadata on so I can sort them by location. Uploading photos to the public web is really what is invading your privacy.
Since the location information can easily be removed but is extremely difficult to add, the clear solution to me seems to be to strip it when not wanted, and only upload your photo to trusted services. Most legit services will strip the info anyway (facebook, slack for example).