White House Open Data Unavailable | Hacker News Reader

White House Open Data Unavailable

open.whitehouse.gov

170 pointsrunesoerensen36 comments

All the data appear to still be available here: https://open.obamawhitehouse.archives.gov/

I think that might just be the usual revamping of the White House website for the new administration. I spot-checked a few datasets and it looked like they are still available at the link above.

Mods, a better title for this might be "White House Open Data Page Archived"? And the link changed to the above link?

Thanks for sharing this! The notion of archiving "Open Data" due to a change in government still seems odd considering this is factual, time-stamped data.

All digital properties go to a clean slate when the Presidency changes hands. There's nothing odd about this. It's similar to the physical aspects of the government, such as cabinet posts, that are directly under the Presidents purview. It being factual really has no bearing on this, although given that for data it just gets archived rather than completely purged it seems to be an incredibly reasonable treatment.

The title of this post, on the other hand, is, dare I say, an alternative fact.

Apparently they changed the software used to store the data, in the bottom on https://open.obamawhitehouse.archives.gov/ it says "powered by dkan", on https://open.whitehouse.gov/ it says "powered by socrata". Maybe it was not easily possible to migrate the data and/or it is planned to be done later.

Thanks, this is exactly what I suspected would happen.

Same thing as the last panic when they switched the web server and pages on major Obama policies vanished. Just an entirely new webapp moved in.

Defend your policies, whatever you stand for, but please don't panic over a website changing hands.

But does this also mean that there won't be any new open data shared by the new administration? If a revamped open data site doesn't ever land then this is still pretty disturbing.

I'm not sure - did any good really come of the Obama admin's open data? I don't think I saw any going to particularly good use myself at least.

While open data in general may be good, if it produces no real valuable results it may be more work than it's worth.

Related tweet: "Today Trump removed all open data (9GB) from the White House https://open.whitehouse.gov/browse but I grabbed it all Jan 20! Will distribute soon"

https://twitter.com/denormalize/status/831581871230193664

Why did trump do this? What was in the data? Could this be an error?

It isn't an error. If you click into one of the individual data sources (links available on the home page), you see messages like this:

> This page is being updated. It will post records of White House visitors on an ongoing basis, once they become available.

> This page is being updated. It will post records of White House staff salaries as the data becomes available for the Trump Administration.

No request sent from the browser

I'd argue that Trump probably doesn't have a clue this even exist, someone else dictated it's archival/removal.

If anyone wants to help out with these sorts of preservation projects https://www.reddit.com/r/Archiveteam/ and https://www.reddit.com/r/DataHoarder/ are good places to start.

#archiveteam on efnet irc

EDIT: Looks like Jason Scott is aware of the WH.gov dump and has offered to ingest at the Internet Archive: https://twitter.com/textfiles/status/831585297003458560

Is there more to this than a page of empty results? Looking at this from a troubleshooting perspective, there are a number of things that could be causing this page besides the data itself being gone.

I agree, it's something that should be looked into, but before proclaiming a purge, I'd want to know more about the situation.

I did consider using "No Longer Available" rather than "Purged" in the title, but all things considered I think there's overwhelming reason to believe this was intentional. Hopefully we'll soon have more information if someone looks further into this.

Some other things I'd like to know, partly from being unfamiliar with the data that would otherwise be here. How far back did the existing data go? Was there anything similar between the Bush and Obama administrations? It sounds like you might be familiar with the data that's usually there. Providing this additional context would be really helpful.

but all things considered I think there's overwhelming reason to believe this was intentional.

I understand where you're coming from. I also know that this kind of hyperbole plays right into the narrative that's being constructed to discredit anyone who raises concerns about the Trump administration.

You considered using a less inflammatory, more accurate version of your title, but decided to go the other way.

So you recklessly decided to post something inflammatory just because you think there's "overwhelming reason to believe" it--so overwhelming that you haven't cited it? Meanwhile "hopefully" we'll have more information "if someone looks further into" it?

When Trump does this the New York Times calls it a lie. He often says we "need to figure out what the hell is going on," just like your "if someone looks further into this."

If the Trump administration wanted to control the story they could've just posted information about the change on the webpage and redirect users to the Obama administration archive (which exists, as pointed out here https://news.ycombinator.com/item?id=13647041).

Instead they chose to break all links and API integrations with no warning or information.

The problem is you've just invented a "fake news" story, in practical terms. You took an event, invented a conspiratorial narrative, then presented it as unvarnished truth. The fact you qualified your title as speculation in the comments doesn't matter.

I'll tell you what I tell my very conservative family members: Your feelings or beliefs are not the same as facts. They are not equivalent in any way. Representing your emotions or opinions as anything other than your emotions or opinions is dishonest.

What's your goal? What do you want to accomplish? Raise awareness of what's going on in the Trump administration? Who's your audience? Do you want to reach anyone other than people who already share your concerns? Do you want to be written off by people who aren't already predisposed to agree with you?

Sure, they could have handled this differently. I also suspect that the people who are running IT at whitehouse.gov may not be used to doing the work that they're currently doing (how often do you change administrations? I know I rarely optimize once-every-four-year processes that aren't critical), or are doing it under time constraints that's preventing them from doing the job the best they can. Except under the most ideal organizations, I think most of us have been in situations where things have been rolled out, on purpose or by accident, that haven't been done the best way possible.

There are plenty of things for people who are concerned about the Trump administration to watch closely. To raise the alarm at all of them very much dilutes the power of the alarm for more serious issues.

In such a polarized climate, preaching to the choir isn't going to make things better. People need to be focused on reaching some kind of shared understanding, and that takes people paying attention to what they're saying and how they're saying it, as well as people being willing to listen and understand (even if they don't agree) what others are trying to say.

Not a single link on the homepage leads to more than some cliff-hanger/placeholder text.

Is there any reason to believe this won't just be moved to the Obama admin archive or something?

Seems like it could just be part of the website change process.

>Seems like it could just be part of the website change process.

It might be.

But don't worry, if that's the case I'm sure there will be a story right at the top of HN that says, "Whoops! Trump didn't delete anything". Because we hate fake news and are not hypocrites...right?

it is the top-voted comment right here before you even made your post.

I shouldn't have to read the comments to determine if the headline is a lie or not.

Indeed you shouldn't, I was responding to your suggestion that the HN community would ignore it if the article content or headline were determined to be inaccurate. (I _think_ that's what you were suggesting through the sarcasm?) The community already hadn't ignored it before you even made that suggestion. So the aspersions against HN's ethical or intellectual consistency were unwarranted.

It would actually be better for the White House datasets to be cataloged and managed via catalog.data.gov. They posted some speeches and videos back in 2014 but have been running their own show since then.

https://www.data.gov/metrics

Notice that White House is listed as having a total of 4 datasets.

Those pretty much have press releases and other specific videos.

I would like to get good answers on this question: What are the U.S. Government data retention policies and rules?

> As I understand it, the most applicable overarching law is the Federal Records Act. More information can be found on the Record Management FAQ. Additionally, I've found the NARA Management Guide. ...

Question posted at: http://opendata.stackexchange.com/questions/10446/cross-cutt...

From Jan 26th: http://web.archive.org/web/20170126072506/https://open.white...

Bulk of them seem to be reports to Congress regarding staff with the odd climate change adaptation task force.

We as a community need to invest into people who are willing to take the time and duplicate all data that is "public and open source" in truly independent repo's. Its simple as that, we know from history that as times change, government changes, political policies change, the truth becomes a victim.

um.. clickbait? misleading for certain.

Reply on news.ycombinator.com