13M Pages of Declassified CIA Documents Posted Online | Hacker News Reader

13M Pages of Declassified CIA Documents Posted Online

motherboard.vice.com

200 pointshappy-go-lucky45 comments

Tip: use the codeword "keyhole" to browse interesting satellite reconnaissance photos: https://www.cia.gov/library/readingroom/docs/CIA-RDP78B04560...

Ref: https://en.m.wikipedia.org/wiki/Keyhole,_Inc and https://en.m.wikipedia.org/wiki/Corona_(satellite)

> March 1964

> Approved for release 2001/8/21

The redacted parts in the documents have codes, e.g.:

(b)(1) (b)(3) NatSecAct

What are those codes refering to?

Are only names redacted?

For you who have trouble searching, here's a document I just stumbled on while browsing through the content: https://www.cia.gov/library/readingroom/docs/0006541525.pdf

It is about interrogation techniques used by USA (power drills were mentioned). Scary stuff.

I wonder, if you train a machine with content from major (all?) news sites, could it then, in theory, predict what was redacted to some degree?

I have an itch to scrape this content. Maybe I shouldn't.

They are codes for FOIA exemptions. http://www.foiadvocates.com/exemptions.html

Interesting searches of the cache include "LSD", "hacker", "Martin Luther King", and "psilocybin". After those searches, the resource was suddenly unavailable.

I haven't gotten the search to work at all, assuming that search bar is for the documents (the UX isn't great). I've been getting "CIA.gov is Temporarily Unavailable" each time.

That said, there are some interesting (and mildly terrifying) documents you can find from some general browsing of the categories.

It's federal IT. I'm going to guess overload due to extremely resource-inefficient implementation before I guess "uh oh, somebody's curious about MKULTRA, better pull the search".

I hope someone downloads the contents quickly, before another administration takes it offline or 'reclassifies' the information.

I wonder if that's actually the motivation behind opening it up now rather than weeks or months from now. There are apparently efforts underway at other federal and federally-funded bodies (e.g. EPA, NOAA, NASA, some federally-funded college departments) to get backups of raw data into safe places in case Congress or the Trump administration orders it taken offline or even deleted [1].

[1] https://www.washingtonpost.com/news/energy-environment/wp/20...

Funny that they only worry about that now that Republicans are taking over.

Reportedly, Obama ordered just such a deletion of records.

http://thehill.com/blogs/congress-blog/homeland-security/268...

Is there anything in this article that actually links the deletion to President Obama or explains why the deletion was ordered? Or even any evidence that this happened?

Edit to be clear: I'm not implying the Obama administration did not do things like this but this read more like an anecdote by someone with political leanings. We also have no information about why they were deleted or modified and don't know if the author had access to the full picture. Perhaps they were merged with another record, therefore improving the "haystack" etc.

The author made that connection: "the Obama administration was ordering they be wiped away."

I haven't seen any followup on this story. Liberal media seems to have ignored it entirely, conservative media reports it without further details.

The author, Philip Haney, has been a vocal critic, appearing on Fox and writing numerous articles, so there's been ample opportunity for the Obama administration to respond if there was indeed an explanation such as you suggest, but I haven't seen any response.

I'm sure the Obama administration is just far too busy with finally getting around to the FEMA camps and gun confiscation to answer every little conservative conspiracy claim.

Maybe he's too busy closing down Gitmo right?

You can't say he didn't try. But the Republican response was so over the top, portraying the prisoners as some sort of supervillains with superpowers, that it never happened. I don't know why the Republicans are so scared of these prisoners.

And sharing NSA data (obtained without a warrant) with the Justice Department?

https://www.eff.org/deeplinks/2017/01/obama-expands-surveill...

Can't blame that one on Republicans.

If you are actually interested in getting an understanding of what happened, you may want to read the declassified source document instead of reading the EFF's take on an incorrect portion of the NYT article.

Or you could share what you know that you think the NYT writer did not.

I'm far from an expert on these things, but is this a mischaracterization in your opinion?

https://www.theguardian.com/us-news/2016/feb/24/obama-guanta...

Just one example: "The day before the Archives speech, the Democratic-held Senate dealt Obama’s plans an even harsher blow. By a blowout bipartisan margin of 90-9, the Senate rejected financing the closure. A Senate leadership aide at the time, stunned by what she considered White House lassitude, explained why even people inclined to help Obama would vote against the measure: Obama had decreed Guantánamo be closed without presenting lawmakers with a specific plan they could defend to skeptical constituents."

To be fair though, there's a lot in that article to justify getting angry at both Democrats and Republicans. For example:

"As implied by the term, Graham spent the summer of 2009 looking beyond Guantánamo. His basic proposition to the White House, reported in detail in Charlie Savage’s 2015 book Power Wars, was to deliver the votes in the Senate for shuttering Guantánamo and opening a replacement facility. In exchange, Graham wanted a broader wartime detentions policy than Obama was willing to embrace. It would allow the US military to continue making captures far beyond the declared battlefields of Iraq and Afghanistan, sending them to the alternative site. When trial was an option, it would occur in the military commissions."

Dirty politics, absolutely unacceptable (to me).

These things are extremely complicated. I really dislike this simplistic recurring narrative (not you specifically, I'm speaking in general) that everything is the Republican's fault. I have no problem believing this is usually true, it is the implicit assertion that it is always true that bothers me.

Although, the Republicans now have a historic opportunity to show their true colors with no excuses....I am ready to be extremely disappointed.

Are you dismissing these claims (which were made by a long time DHS employee, not by me) simply because you support the politician involved?

Surely you aren't claiming to know more than he does.

There is no equivalency between deleting criminal justice records, which should and normally do remain private unless used in a court of law, and removing information that's been released the public sphere.

"After leaving my 15 year career at DHS, I can no longer be silent about the dangerous state of America’s counter-terror strategy, our leaders’ willingness to compromise the security of citizens for the ideological rigidity of political correctness—and, consequently, our vulnerability to devastating, mass-casualty attack."

Both are stories about deleting information that doesn't fit a political agenda.

There's an option between releasing and deleting information about suspected terrorists.

Unless you're saying the suspected terrorists were cleared? If so, what's your source?

> Both are stories about deleting information that doesn't fit a political agenda.

One set of information is public, one is private.

The information we're discussing is public - this recently released trove of CIA documents and NOAA & NASA climate data for instance. The DHS' internal database of terrorism suspects is private unless used specifically to prosecute individuals for crimes.

If your claim is that the DHS should publicly release data about terrorism suspects, that's another debate, although a spurious one. We've done that before, it was called McCarthyism, and it left an indelible negative mark on the US government. If DHS were to engage in such behavior it would likely be challenged and reversed on 6th amendment grounds.

My claim is that tracking terrorists is important, not that such information should be made public.

Information about terrorists should remain private, but it shouldn't be deleted.

In this, as in so many other ways of life, anybody saying 'political correctness' in seriousness can be ignored.

What would you call it when crimes are covered up because of the race of the offenders?

As, for example, in Cologne.

https://www.theguardian.com/world/2016/jan/06/tensions-rise-...

http://www.independent.co.uk/news/world/europe/cologne-polic...

Remember how loud Obama screamed when the CIA "lost" that torture report.

Oh wait. He was silent on that one.

Both parties love secrets. If the public knew what they were up to they would all be out on their asses next election cycle.

... How can someone work at the DHS for fifteen years in February 2016?

Would it even be legal to delete existing data sets? Surely it belongs to the US public, as it was collected using taxpayer money?

(A more likely scenario would be that the programs collecting data the new administration doesn't like gets the budget axe, creating a hole in the data series for posterity to wonder at. ("Say, ma, how come we have climate records from 1870-2017 and then again from 2021 onwards?")

How would one go about mirroring these documents?

Well sort of:

   CIA.gov is Temporarily Unavailable

   CIA.gov is temporarily unavailable. We apologize for the
   inconvenience. Please try again later.

   Posted: Aug 27, 2012 04:31 PM
   Last Updated: Aug 27, 2012 04:31 PM

That was the result of my search 'Nicaraguan contra'

Considering they went from about four computers accessing the db, to the Internet, I'm leaning more towards the reddit/HN/whatever-effect :)

Here's to hoping the "someone who got called in to fix it" makes a tremendous work in scaling it.

Federal IT. Betting it's chock-full of easy wins for a savvy dev, and those do tend to turn up with a GS number from time to time. They rarely seem to stick, though, which I gather is part of what 18F is trying in a small way to change.

Search another term, like 'keyhole' and it succeeds immediately. Switch back and forth and you can see it's deliberate.

Or just cached...

They should put it in a torrent

They likely won't, but would be awesome if someone here on HN did!

Torrents are only for criminals. /s (edit: double sarcasm in fact)

Since the landing page required everyone interested to enter a keyword to search, the back-end server can record all query and ip-addrs.

I doubt if WikiLeaks.org use this exact same method to study what document attract traffic|attention. WikiLeaks' new releases are not signed by PGP anymore, I take that as either a Warrant Canary or some kinds of warning.

Of course, if everything packed in a torrent, then there's no way to conduct such a study.

You can iterate through all the documents by going to https://www.cia.gov/library/readingroom/search/site, going through each year, and within each year, going through all the result pages.

link: https://www.cia.gov/library/readingroom/collection/crest-25-...

Search function not working - redirected url is "https://www.cia.gov/$(SERVE_NS_)/library/readingroom/collect... ?

CREST has been available to the public for a while now, though not online. ODCI/DI would be interesting to look into but I won't hold my breath finding anything outlining the bastardly of the early 60s. [0]

First terms I looked up, "Budget Hawkeye" for 1963-64. Found some details, hard numbers were redacted. NPIC ^grew^ a lot post '63 judging by the budget requests. Lots of files referencing "Brass Knob" in '63 (daily Cuban flyover). The other interesting details were NPIC monthly memos detailing the daily operations.

Some interesting finds

- David A. Phillips ('81) https://www.cia.gov/library/readingroom/document/cia-rdp90-0...

- Howard E. Hunt ('85) https://www.cia.gov/library/readingroom/document/cia-rdp90-0...

- Jim Garrison ('67) https://www.cia.gov/library/readingroom/document/cia-rdp90-0...

- Film 'Executive Action' ('73) https://www.cia.gov/library/readingroom/document/cia-rdp88-0...

[0] https://news.ycombinator.com/item?id=12324273

[1] https://www.cia.gov/library/readingroom/document/cia-rdp76-0...

The declassified studies in indigence are an interesting read - I remember one where they documented some of the kooks that wrote to the DCI.

The printing hack was pretty nice :)

Reply on news.ycombinator.com