Fate of Data.gov Revealed; US Gov Almost Completely Drops the Ball | Hacker News Reader

Fate of Data.gov Revealed; US Gov Almost Completely Drops the Ball

readwriteweb.com

56 pointshelwr16 comments

The dumb part here is really that cutting these problems INCREASES the cost of government. By a lot. Taking public data offline, or making it less accessible means it gets FOIAd instead. FOIA costs the government a half billion dollars a year.

Need an example of how the data.gov team did some great work?

Compare a search for "interchange fees" on the http://www.gpoaccess.gov/fr site, with federalregister.gov. Not only does it return the most relevant result in the #1 spot, but it's also beautiful and usable:

http://www.federalregister.gov/articles/2010/12/28/2010-3206...

The Federal Register is a critical function of how our government functions. My jaw dropped the first time I saw the new site, it's sad to compare with what we were forced to put up with before.

"And so, my fellow Americans, ask not what your country can do for you; ask what API you can do for your country."

--JFK 2.0

This is very unfortunate and demonstrates how easy it is to kill a relatively inexpensive program that has enormous potential upside when its constituency is diffuse and dividends from it may only be realized in the future.

So if I'm BigContractor X or Union Y, I have a few issues I really care about and have the willingness to spend money to influence legislators. Open data does not concern me.

But since the the beneficiaries of "open data" are widely dispersed, who lobbies on behalf of greater transparency? Sounds like a startup opportunity!

The Sunlight Foundation, OMBWatch, POGO, and a variety of trade associations for newspapers and investigative reporters lobby on behalf of transparency issues.

So after 2.5 years of doing jack shit, the forthcoming budget cut is the problem? I see.

Can any of the moderators who downvoted this tell us how much has been spent on data.gov over the past two years and how useful they think it is presently?

On HN the bulk of the downvoting is done by users with karma over 500, not moderators.

I downvoted this comment. I work in the government transparency space, and I can tell you I think these tools are incredibly useful. Overall, if you do some deep diving into what data.gov actually costs the answer is -- as always that it depends.

If you look at just the technology and infrastructure costs, Data.gov itself costs about a half million dollars a year to run. Layer on staff time and you're looking at roughly double that.

A week ago, I spoke with the Data.gov team, which now consists of 4 FTEs: an evangelist, a person whose job it is to get agencies to put their data on data.gov, a program manager (who basically manages the contractors, REISystems and Socrata), and 1/5th of Sonny Bhagowalia's time. Based on the work of this team, Data.gov has generated a significant ROI.

For example: when I was at the Sunlight Foundation, we held a contest called Apps for America to promote Data.gov which offered 25,000 to the developer who built the best app on top of data.gov data. One of the winners was an app called GovPulse.us which took the Federal Register and built a website around the data.

The Federal Register is the official journal of the united states. It's where all the public notices and stuff go -- so making that data more accessible is important, and making it so people can sort through it and link to it is also important. But ideological issues around transparency aside -- a few weeks after the contest concluded, the Government Printing Office issued an RFP looking for someone to build a new website for the Federal Register.

With some work, the fellows that built GovPulse.us (which is open source) were called in to help build FederalRegister.gov, at a fraction of the cost of what the government intended. Based on what the federal government usually spends on websites -- That single event of adopting open source software and repurposing it -- probably saved the government upwards of ten million dollars -- far more than what's spent on Data.gov.

Another company, BrightScope.com, is able to use data.gov data to build a platform for pension analysis. While their tax revenues are probably not paying for data.gov, they are a strong, growing business generating wealth for its shareholders and employees. When government open data has the opportunity to mature, it does this. Your weather reports, GPS data, and flight data usually come from government data sources which are beginning to become accessible via data.gov.

Startups ought to be outraged by this for this reason alone. The fact is, there are THOUSANDS of opportunities to build businesses off of government data. SEC Filings, TIGER maps, Census data, BLR Statistics -- there's a tremendous opportunity here for a program that's operating inside the margins of government expense and has likely paid for itself many times over already.

Cutting the program, or saying it doesn't work means you aren't paying attention to either current performance, or future potential. Both of which are just fine.

I should note though that none of that is why I downvoted you. I downvoted you because I thought the comment was not substantive,was inflammatory, and didn't bring any value to the conversation other than what came across as confrontational name-calling.

To me, it seems like it was likely an ideological complaint about the administration rather than demonstrative of the facts. Political ideology doesn't belong on HN.

Happy to upvote a comment that challenges those assumptions.

> Political ideology doesn't belong on HN.

If my comment contained political ideology, then so did the story itself.

Thanks for your comment. I can't say I agree with your assessment of its current performance. I routinely need government data and I'm still waiting for the first time I find it on data.gov. I remember the contest and I thought the entries I saw were useless. Sorry to be so blunt, probably I just have higher expectations than you. Partly those were conditioned on Obama campaign promises, and partly they're conditioned on how the internet normally works.

Why don't they publish the data in a quick and dirty no frills way, and let hackers like myself pretty it up? Data.gov has a lot of value, and I'd love to contribute some of my free time working with it.

Exactly. When data.gov first launched I was really excited because somehow my mental image of what it should be was a raw dump of all the datasets they had available. What a found was something that felt very sanitized and sparse compared to what we all know the government actually has. Making all that data available in any format would spur all kinds of new ideas and businesses.

Government is so good at managing this sort of thing. We should really ask our government to do more stuff because of its stellar track record! <sigh>

Reply on news.ycombinator.com