The process was essentially to fill out a form and send them a new, unopened hard drive. What I got back a few weeks later was about 125GB of high resolution GeoTIFFs. Along with it were PDF's explaining how the data was rectified and cleaned up as well as information for how they encoded the data.
The process and interactions really impressed me with the quality of people involved in agencies like the USGS and NASA.. just like the people who provide API's in the tech industry, they're usually really in to what they do and interested in hearing how other people are making use.
The data is totally out there and you can get your hands on it without a) breaking terms of service or b) needing any sense of entitlement or self-righteous "ethics". I think the article hit the nail on the head. In that ignoring the data freely available reduces its viability. Projects like GeoNames, OpenStreetMap, the deactivated OpenAerialMap are all incredible sources of information that we can't afford to ignore in favor of the easy way out
They are extremely competent and sincerely want you to have good data. They are also hampered by the bureaucratic limits of any large organization. So it’s like working with a large, well-run business that’s hired a lot of the best people in its field and is working on good problems, yet is large enough that it can’t move to meet the exact needs of any one customer.
But on top of that there are political concerns. They have an institutional fear that a congressperson in a budget debate is going to stand up and say something like “And apparently we’re paying the Geological Survey $N million a year to run a web server for something called geotiffs that tell you how tall hills are!” That’s my impression from reading between the lines, anyway; no government employee I know has been indiscreet enough to deliberately hint at such a fear.
For example, the best interface to SRTM isn’t from the agencies that made it, it’s a single-page project from Derek Watkins at the NYT: http://dwtkns.com/srtm/
Working with NASA in particular feels like working with an industry leader that has a mysterious policy against advertising, or even going out of its way to help you find resources. (Individuals do, but not the organization, at least not anywhere near in proportion to the number and value of its resources.)
NOAA too: they have some amazing satellite imagery that’s public domain, but they simply do not have the budget to do anything but the most halfassed job of hosting, publicizing, and documenting it, because from a funding perspective that’s frivolous. They barely archive their images, because no one with budget control gets why a weather agency should save its input data. Look up “VIIRS granule” – that’s technically open data, but yikes.
The resources are there, and if you make the effort to figure it out, the people who manage them are pretty much all a delight to work with. But you have to deal with the damage created by a political culture that too often treats our civilian space and geospatial agencies as afterthoughts rather than as highly multiplied public goods.
This would be a hard argument to defend against as it directly hits the value derived from the service rather than the idea that the service itself exists. In either case, I couldn't agree with you more that the civilian agencies really get a raw deal (and I'm sure its only compounded by the mostly stagnant level of people graduating with STEM degrees[1])