Formerly Top Secret NSA Cryptologs From 1974-1997 | Hacker News Reader

Formerly Top Secret NSA Cryptologs From 1974-1997

nsa.gov

153 pointsthirsteh63 comments

There's an interesting blurb in the following (bottom of the fifth page in the PDF):

https://www.nsa.gov/public_info/_files/cryptologs/cryptolog_...

"In today's Age, the public has centered in on government as "the problem." Specifically, the focus is on the potential abuse of the Government's applications of this new information technology that will result in an invasion of personal privacy. For us, this is difficult to understand. We are "the government," and we have no interest in invading the personal privacy of U.S. citizens."

This attitude is similar to Bill Binney's (in that U.S. citizens are off-limits due to FISA)[1]. I presume he wasn't the only person within the NSA who felt like that…and I can't help but wonder what the internal dialog is like these days.

1: http://www.newyorker.com/reporting/2011/05/23/110523fa_fact_...

> We are "the government," and we have no interest in invading the personal privacy of U.S. citizens.

That is interesting. That may be true for the people who made that statement, though it's hard to guarantee it for anyone who has ever or will ever have access to NSA information. You don't have to believe in a nefarious Big Brother to be concerned about the perhaps inevitable potential for mistakes or abuse by some individuals behind "Government's applications of new information technology". History offers plenty of examples, after all.

The explanation is simple for me. They have brainwashed themselves. One of the most selective criteria for working there is patriotism and unquestioned loyalty to the country, this includes people who love and swear they support the Constitution.

So how do they end up sucking up tons of data from all over the internet, filtering and storing it (which includes private data of US citizens) -- easy, it is justified as fighting terrorism and protecting our country.

There is a story one needs to tell oneself continuously in order to maintain and support this brainwashing. These are stories that NSA tells itself (public is concerned but they have nothing to worry, we know we don't want to harm them, we are here to protect them).

Pretty sure if you asked those who conducted tortured at the CIA, they'd also tell you they are devout patriots and did what the did to protect the Country, the Constitution, the Flag and everything that stands behind it.

"One of the most selective criteria for working there is patriotism and unquestioned loyalty to the country, this includes people who love and swear they support the Constitution."

Did you work in HR at NSA/CIA or is this how you guess things work?

The basic criteria for getting a job at the NSA is public knowledge (https://www.nsa.gov/careers/jobs_search_apply/hirerequire.sh...):

The background investigation helps determine the applicant's honesty, trustworthiness, reliability, discretion, and unquestioned loyalty to the United States.

I have seen NSA's recruiting information a couple times in the past. Can you highlight the bit that says which criteria is the most selective?

Stop being obtuse.

unquestioned loyalty to the United States.

As soon as one stops questioning ones reasoning gets selective and biased.

What am I being obtuse about and why are you talking about selective and biased reasoning? The aspect of the comment I was responding to was "One of the most selective criteria." Do you know what the most selective hiring criteria is for TS/SCI work?

Instead of beating around the bush, why don't you tell us what agency you work for, what department and what is the most selective criteria to TS work.

I hate to ruin the conspiracy party but I do not work for the government.

Loyalty and blind obedience are not necessarily the same thing. I'm fond of Carl Schurz's aphorism: “My country, right or wrong; if right, to be kept right; and if wrong, to be set right.”

Potential abusers are not only individuals - even if the current government was mostly harmless, there is no guarantee that it will still be as nice fifteen years later : tyranny sometimes rises fast and with no warning. So even with a theoretical perfectly benevolent government, privacy restrictions are necessary to slow down the hypothetical rise of tyranny - be it individual tyranny or government-wide tyranny.

The description of the Director's Summer Program (DSP) for recruiting math undergrad interns sounds almost reminiscent of Ender's Game. These were some brilliant kids who achieved a lot in one summer.

> The students had to learn decades of classified cryptologic mathematics in two weeks, as well as a myriad of details about the four problems presented to them. During these two weeks, some learned to program for the first time. All were proficient programmers by the end of the summer.

> Incredibly, before they met us, two of our DSP students, juniors, had not been planning to go on to graduate school following their senior year. These two were performing exceptionally well in their current, demaning academic programs and, ironically, made the most direct contributions to the most significant results of the workshop. One went home from the DSP with a surge of confidence, applied to all the top graduate schools and is now in a Ph.D. program on a fellowship. The other wished to become an NSA employee, but we talked her out of joining us right away. She took all pure mathematics courses her senior year and is now in graduate school in a Ph.D. program on a fellowship.

It'd be fascinating to know what they're working on now.

(Vol. XX, No. 1 - 1st Issue 1994, #126 on the list)

> It'd be fascinating to know what they're working on now.

They're almost certainly not working with technology that's "ahead by 10 years", as their recruiters like to advertise: Their hardware is basically standard stuff shipped by Sun (... I guess that's Oracle now), running mostly Java.

I just love reading ████████ documents. The ████████ parts just keep me guessing

what's in them. I think that

████████████████████████████████████████████████

███████████████████████. Would that just make my day!

There is some humour to be had. From No 136:

"An Example of Intelligence Community Synergy"

[four blank pages]

https://www.nsa.gov/public_info/_files/cryptologs/cryptolog_...

This reminded me of that Pentagon doc in 2005 that was poorly sanitized: http://news.bbc.co.uk/2/hi/europe/4506517.stm

Let's not forget the Onion's take on it:

http://www.theonion.com/articles/cia-realizes-its-been-using...

"Top Secret Umbra" I haven't seen that code word since I worked at a communications monitoring site in Turkey in 1977. I never could have imagined seeing it on a document released to the public. Time does change things.

To save everyone some Google queries, "UMBRA is the highest-level compartment of the three compartments of Special Intelligence—the euphemism for COMINT. The lower level compartments are MORAY and SPOKE."

Wrong tense, "UMBRA was..."

Really? Just because these documents were unclassified, doesn't mean UMBRA is no longer in use.

According to Wikipedia's sources it is no longer being used. http://en.wikipedia.org/wiki/Sensitive_Compartmented_Informa...

Well, per http://www.dtic.mil/doctrine/dod_dictionary/, a code word is

    A word that has been assigned a classification and a 
    classified meaning to safeguard intentions and 
    information regarding a classified plan or operation.

That is to say, the meaning of a code word is generally classified. It's not just a convenient label; its purpose is to obfuscate even the general intent behind . . . whatever's going on under that umbrella.

If they think it's been compromised--that is, if they think someone has figured out UMBRA=COMINT--they'll generally change it. If it's made it all the way onto Wikipedia, they probably changed it long ago. And if it isn't redacted in declassified materials, they definitely changed it long ago.

Though it's always possible that they just don't care anymore. Sometimes programs persist under their code words long after what they're doing isn't classified anymore.

Well... half the pages are half-redacted. It's not like they're releasing much of the information that justified the classification anyway.

From the September 1978 article "NONSECRET ENCRYPTION (Public Key Cryptosystems)":

"We in the intelligence community have become accustomed to holding a monopoly on useful advanced cryptologic knowledge, so it is with surprise and apprehension that we have witnessed in recent years an increasing interest in cryptology on the part of American academicians."

I'd be interested to know if GCHQ let NSA know about Clifford Cocks' work on PKI. (Since he effectively invented a system in 1973)

Seeing redacted docs like this always makes me wonder - for brief blacked-out passages, couldn't you make measurements of the remaining letters/words on the line, their sizes and spacing, and algorithmically generate a few likely candidates for the blacked-out text?

You could at least estimate the length in characters of the blacked-out text. For a monospaced font this character count is trivial; for a proportionately-spaced font it'd be a little harder but you have lots of other non-censored characters to learn from.

See here for example: http://cryptome.org/cia-decrypt.htm

There was a released-but-redacted CIA memo saying, "An Egyptian Islamic Jihad (EIJ) operative told an XXXXXXXX service at the same time..." From analysis of the size and shape of the blob, the missing text could only be "Egyptian".

In fact, a monospace font turns out to be harder for this; with a proportional font, as here, there is more variability in total word length due to the different letter widths, and so a greater ability to reduce the number of possible matches.

Its kind of funny that the crossword puzzle in the September 1978 issue is redacted.

36 Down: Confirmed Russian spy in the State Department

Here's a 7-zipped archive of all the PDFs: http://dl.dropbox.com/u/94483242/nsa-cryptologs.7z (md5sum: 88b39bd611a88e5e0bc789fe493701ba)

Many thanks - the site is only showing a single entry for me, and endlessly timing out :|

November '81 has a cool 8 page article on the coming age of "powerful personal computers", with a good overview of the tech of the time. Soon everyone can have their own VAX or 370!

https://www.nsa.gov/public_info/_files/cryptologs/cryptolog_...

Too bad they redacted the recipes; I was looking for how to bake some Crypto Cookies.

http://en.wikipedia.org/wiki/Evercookie

I love some of the examples for people discussing routine "over classification" of government documents:

https://www.nsa.gov/public_info/_files/cryptologs/cryptolog_...

Page 33, Book Review "Rapid Development" by Steve McConnell. A "top secret" book review now sees the light of day!

edit: The introduction mentions some predecessor magazines targeted to specific groups. "Dragonseeds" to B group, "Keyword" to G group, "QRL" to language, "Command" to traffic analysis and special research. I wonder if anyone has FOIA'd these earlier publications?

I believe that they (government) are just in a business where it's better to overclasify 100 documents than underclasify one.

Think of it from web development perspective. Years ago SSL were used only for financial transactions, then for e-commerce transactions. Nowadays it's considered a good practice to use it anywhere you transfer any user data or session. Isn't that our industry's equivalent of their over-classification routine? I think they basically do the same what we do with SSL - they apply their security layer to all content produced by all their users. It's exactly what we do with our security layers in software development.

I enjoy this analog, and it works very well from a purely developer/intel analyst perspective.

It seems to me that the key difference here would be that no one is harmed by overuse of SSL, whereas over classification of information can have far-reaching negative effects. Failure by the intelligence community to realize such, or a systemic issue that incentivizes over classification, lead to our current situation where a FOIA is required to read a parking ticket.

> no one is harmed by overuse of SSL

Serving everything over SSL has removed HTTP's whole notion of "caching proxies." Now a website can be cached by your browser, or by the remote (i.e. through a CDN which they'll hand their X.509 cert to), but never by, say, your ISP.

And this is a shame, because HTTP's method idempotency semantics and Expire headers allowed intermediary caching to work perfectly--when something was set to expire from your local cache, it would also expire from any intermediary caches at the same time.

Sadly, some ISPs overreached and started modifying the content they proxied, at which point SSL-everything became the clear winner. Additionally, that kind of caching kind of screws things up when you serve any HTML that has been customized per-user on a generic cacheable endpoint (say "GET /timeline")--even though proper HATEOAS strongly indicates against this.

CDNs have really assumed the role of ISP-level caching - the good CDNs are co-located with the big ISPs anyway, so the effect is the same. IMO, it's a better solution because it allows the content-server much better control over the exact details of the caching and allows stuff like partial caches. The problem with ISP-level caching is just what you suggest: they screw it up.

There is a time cost using SSL. So, maybe not on the same level of impact as over classification of documents.. but still there!

My guess is that the cumulative time cost of SSL doesn't exceed millions of dollars. On the other hand, if you believe that over-classification undermines the democratic process to the extent that wars have consequently been fought against the USA's self interest, the cost of over-classification could easily wander into the trillions of dollars (and tens to hundreds of thousands of lives).

I think it's reasonable to neglect the time cost of SSL in this comparison.

Documents are classified at the highest classification of any single piece inside the document. In the cryptolog you linked to, there are two large redacted sections. It's not that the book review was classified, it was just published next to stuff that was classified. If you filed a FOIA request, they would have given you the book review with minimal fuss, but kept the redacted parts out of the FOIA. You can tell the information that is unclassified because it is marked (U). Classified info will be marked with different letters depending on the level of classification, and will likely be heavily redacted in anything released to the public.

Does anyone else see some of this as really similar to old 2600 or phracks? It's probably just the printing technology.

I'm still looking for the article where the NSA card 20 pizzas.

I have to say page 19 of the last publication is quite interesting. They talk about the Sentinel database filter on Sybase. https://www.nsa.gov/public_info/_files/cryptologs/cryptolog_...

I usually try to subvert the redaction on PDF files with a reasonable degree of success, but I suspect it would be a waste of time in this case :) Anyway, most interesting, both technically and socially. Had I been born in the US I think I'd have enjoyed working at the NSA.

Well it's a start, I guess...

Anyone stumbled upon Untangling the Web? It's a DOD "book" about web search, classified, remarkably interesting and nothing warranting being classified. I'm sure NSA has tons of actually interesting stuff they could make public

I'm curious what is in the redacted parts that still needs to be classified? Surely nothing from 1974 is still state-of-the-art today. Surely no covert operatives are still in danger from the 70's (though I guess it's possible).

For the specific purposes of redaction, it's not about whether something "needs" to be classified. They redact what is classified. The decision to unclassify lies elsewhere.

And the reason it hasn't all be declassified with a blanket order is no doubt simple bureaucratic conservatism. No one is going to get an award for "brilliant work in declassification", and the last thing any spook wants for her career is to be yelled at for declassifying something embarrassing.

1974 is only 40 years. The terms "career politician" and "career bureaucrat" apply everywhere; someone recruited, say, right out of school, could easily still be working. That's to say nothing of anyone those people may have recruited.

Same goes for technology. What was done 40 years ago may not directly apply, but it might give clues to what's around today.

Statistically, real threats are rare, but ambition and corruption are common. Overwhelmingly, the purpose of censorship is not the protection of national security, but the protection of individual careers. That's not ideology, but mathematics. Because there are very, very, few true national secrets, but a huge amounts of information that someone would like to bury for one reason or another.

Seth Finkelstein

http://grep.law.harvard.edu/article.pl?sid=03/12/16/0526234&...

I don't think any kind of careerist conspiracy is required to arrive at a culture of over-classification.

The habits of anyone working in any kind of role involving information security are so utterly obvious that they barely require discussion.

You make sure your office environment is secured, don't leave papers on your desk. Don't duplicate information more than necessary. Full disk encrypt everything. Never email documents without encryption. Don't use USB sticks without encryption. Know who you're talking to on the phone. Don't ever talk about incidents, jobs or the specifics of what you do.

Now think about people who reflexively do all this stuff and consider: a) how strong the urge to classify by default is and b) how much more work it takes to be 100% sure a document is safe for release.

You've just described a careerist culture, if not a conspirace, it seems to me.

What's the difference between reflexively classifying everything as highly as you can, and routinely covering up inefficiency and waste, and maybe a little graft on the side? Pretty much nothing.

Looks like the site became overloaded. The files can be found here: http://cryptome.org/2013/03/cryptologs/00-cryptolog-index.ht...

You can download a zip archive of all of the issues from here: https://www.copy.com/s/tVBQpAJYGxYSTKw/Readings/Cryptolog

It seems like all the good parts have been redacted...

Why are some parts redacted? If they are de-classifying and keeping the juices away then they might as well keep it a secret.

Not at all. If the original statement was something like, "Top Secret Agent Spongebob Squarepants determined that rot13 is not a good encryption scheme," then redacting the name of the top secret agent while releasing the rest of the statement makes perfect sense.

But a name cannot be paragraphs long.

Tell that to this man : https://en.wikipedia.org/wiki/Wolfe%2B585,_Senior

I love the NSA sponsored writing contest in this newsletter:

https://www.nsa.gov/public_info/_files/cryptologs/cryptolog_...

"Writing for a competition may bring about revolutionary changes!"

Reply on news.ycombinator.com