Room 641A | Hacker News Reader

Room 641A

en.wikipedia.org

333 pointshourislate70 comments

One day I was getting out of my car to carry groceries into my house. Our nice neighbor was passing on the sidewalk at that moment, saw the EFF hat I was wearing, and asked me about it. "Do you work for the EFF?" "No, but I throw some money at them when I can because I like them." "Oh. I like them, too. They helped me out a lot once." "Oh yeah? For what?" He laughed. "I'm Mark Klein. Go Google me when you're finished with your groceries." I knew his story from following it in the news, but didn't recognize his name. I about spit my teeth out when I found out who I was living next to.

Super nice guy. Has the only two golden retrievers I've ever been frightened of, and if I were him, I'd probably have some assertive watchdogs, too.

Cool story.

Off topic question... Do golden retrievers have a mean side to them? I’ve always liked them and I dont like scary dogs in general

I don't know if they were actually mean, or just trained to be noisy and act like they were. Either way, I know they'd raise holy hell at the window whenever I walked past it. They might've been thinking "hey Dad, it's so fun to bark at people! Look at how big I'm smiling!" for all I know.

They'll get that hair on you, and you'll never get it off of yourself. Better to leave them well alone.

Not normally, but temperament depends on a lot of things. I've read that socialization and temperament of the dog's parents are important.

Wait, socialization of the parents via some epigenetic effect? Fascinating.

Or more likely because very young dogs would grow some basic traits of character while influenced by the behavior that their parents exhibit in front of them.

For some reason, I assumed that the animals were not brought up with their parents.

Puppies are supposed to be separated from their parents no earlier than about 6-8 weeks old. Critical socialization time is the first 3 months of their life. (Or so. These numbers vary by region and amount of caring; I'd rather have the puppies spend more time with the mom than less.)

Interesting. TIL

Are you trying to point out the missing Oxford comma?

Haha no, did I misunderstand because of the absence of a comma?

I published most of Klein's docs at Wired the morning of a hearing on EFF's case vs. AT&T. The docs were under legal seal at the time but I got them from a source not bound by the seal. Still had a good chance AT&T would sue me (I was a freelancer).

Walking into that courtroom later that day and getting stares from AT&Ts lawyers was, for lack of a better term, special.

Also Mark Klein is an underappreciated American hero.

Mark Klein, the whistleblower for all of this, wrote a great book on his experience that I highly recommend: https://www.goodreads.com/book/show/6625940-wiring-up-the-bi...

Putting ethics and legality aside (if you can, and you shouldn't), it's a marvellous technical achievement to be able to take a beam splitter and analyse all the traffic. I assume they must do a lot of early filtering in hardware to select traffic of interest based on IP headers and then some deeper inspection, and then gradually move up to software on the filtered traffic. Do we think they start with dedicated ASICs, perhaps add FPGAs for second line filtering which would give them a degree of adaptability, then software for the rest? What were internet backbone speeds like in the early 2000s?

I work in an environment where we do super high speed deep packet inspection for commercial customers. We use FPGAs. If all you want to do is split and exfiltrate (send to other servers for analysis) the packets as they come over the wire and maybe send them into different buckets then an FPGA is more than enough. You only need to be as fast as the slower of the two devices you're between.

Oddly enough, the "fiber optic tap" shown in the article is an extremely mundane component that you can buy out of a catalog. There are two of them in the box, called fiber directional couplers. I used one when I just wanted to combine two colors of lasers for an R&D project.

Needless to say, the stuff that those fibers would go into and out of in that particular application, is way beyond my comprehension.

IIRC the Narus Insight device was an early attempt at deep packet inspection[1]

Also, TippingPoint had a pretty blazin' 40Gbps ASIC-based packet filter 10 years ago.

1: https://arstechnica.com/information-technology/2013/06/what-...

>it's a marvellous technical achievement

Not trying to argue for the sake of it here, but _is_ that a huge achievement? I assumed this was a simple device that splits the fiber beam, sending one part on it's way to AT&T and the other to some NSA database for further processing.

Well at the speed the data comes in, you don't have time/space to store it all on disk (and just filling up also doesn't seem to beneficial for them) and also can't just double all traffic for sending it to the NSA

So they have to filter a lot of data for interesting bits really fast before storing / sending it anywhere else

Interesting, thanks.

The thin blue line runs around the world. NSA is party to multinational intelligence sharing agreements. If they don’t delete the data to prevent discovery and disclosure in public trials, they are in violation of the contract. Any legal scholars have a rejoinder to this? How would you have approached this case to prevent outmaneuvering in these legalistic ways? It is a mockery of the rule of law that it is upheld and twisted in this way for some agreements but not in favor of the Constitution.

From linked article on Jewel v. NSA[1]:

On March 10, 2014, Judge White imposed a temporary restraining order, requiring the NSA and other parties to halt the destruction of evidence until a final resolution of the case. On June 5, 2014, the EFF filed a motion for an emergency hearing requesting that the court enforce this temporary restraining order after discovering that the government had continued the destruction of evidence. A motion filed by the government claimed that doing so would have severe consequences "including the possible suspension of the Section 702 program and potential loss of access to lawfully collected signals intelligence information on foreign intelligence targets".[2]

[1] https://en.wikipedia.org/wiki/Jewel_v._NSA

[2] https://www.eff.org/files/2014/06/05/govtemergencymotion.pdf

The NSA wasn't the first agency to not follow the laws as written. In America no one acts like they care about the rule of law unless it impacts their personal lives.

It's unfortunate that people take it for granted.

That said, I would rather live in a country surrounded by people who take it for granted, than live in a country where there is no rule of law.

Off-topic, but what kind of door lock is shown in this picture?

https://upload.wikimedia.org/wikipedia/commons/8/89/Room_641...

The lock on that door is largely circumvented by the fact that the hinges are mounted backwards. The door opens outward, so it could be trivial to lift the hinge pins and pull the door towards oneself.

Hopefully the hinges have security pins.

> lift the hinge pins and pull the door

That’s a bit of a myth. It won’t work on normal door hinges going back many decades. When the door is closed, the knuckles of the left and right sides of the hinge overlap, so the door won’t slide out even if the hinge pin is removed. You need to swing the door open at least 10-20 degrees to disengage the knuckles. (I wish I found a decent close-up picture of hinges on a closed door to include here.) The easiest way to see this is to walk over to the nearest door and look at the hinges.

I've had a residential condominium door that wouldn't open even if you removed the hinges completely. The door itself had bars on the hinge side, those bars went into holes in the frame when the door closed.

Looks like rivets are holding a panic bar or stronger bar.

Fun fact: Klein got actually interested in that room because he overheard a janitor that was complaining that he is not allowed to clean that room and that makes him feel he's not doing 100% his job. The initial response from some higher ups was that the room will become a new janitor closet but months went by and then these reinforce bars were installed overnight. That persuaded Klein to continue his research.

PS. The room still exists and continue to serve its purpose.

Similar: https://www.youtube.com/watch?v=kF4EUM8CwT4 "The Secret Behind the Huge, Windowless Skyscraper in NYC" by Half as Interesting

And: https://en.wikipedia.org/wiki/33_Thomas_Street ; https://news.ycombinator.com/item?id=21555954

The reason it doesn't have windows it because it's a telephone exchange. These are, almost universally, windowless shacks in the US. The cloak and dagger bits are just because it's in a convenient location for spying.

Telephone exchanges especially big ones are built for resilience and high security

A few years ago I had to do an update to some of our sun servers in CAPITAL (the really big international Exchange in Edinburgh) interesting experience cavernous rooms and quite spooky with all the various clicks and bleeps

Is it a known fact that AT&T is in bed with the NSA, or just a coincidence? Titanpointe and Room 641A are currently or formerly owned by AT&T.

> Is it a known fact that AT&T is in bed with the NSA, or just a coincidence

It's complicated.

Telcos that operate in the US are required to follow US laws and court orders and there are laws and orders that require them to submit data to the gov.

But they also don't seem to be trying to fight it: https://www.wired.com/2013/09/telcos-metada-orders/

I'd imagine if they fought it, a new law would magically appear in the books so it's hard to tell if they are being practical by not pursuing legal action that is bound to be overruled or just plain evil (the truth probably lies in the middle)

Basically all telecom. Highly important for three-letter agencies to have access, and they have basically unlimited resources, so they get access.

Don't communications companies have to follow some law stating that their infrastructure MUST be tappable by the US government?

CALEA, FISA, and PATRIOT allow for that together.

My uncle worked for AT&T and he said he was installing hardware to do wiretaps all the time. I don't know the exact specifics, but he would only go so far with the tap, someone later would come in and complete it, presumably from the NSA.

All PTT's and ex PTT's do its just how that industry is.

All telecoms are. It's no coincidence that telecom security is crap.

https://www.aftenposten.no/verden/i/Olkl/sources-we-were-pre...?

https://www.theguardian.com/business/2014/jun/06/vodafone-re...

Clickbait. It's not a 'secret'. Atlanta has 56 Marietta Street...same windowless tower. They're in every major communications hub.

Also see: https://en.wikipedia.org/wiki/_NSAKEY

The async nature of internet routing, bonding, traffic sharing cold vs hot potato, against the rumoured capabilities of these rooms has always bothered me. You would have to be in a position to decrypt or store both directions of the async flow, then correlate that flow between locations that could be for example an alternative landing station on the other side of the country. Or am I completely wrong and are both directions of flow somehow easy to gather or simply not important for the level of detail needed to spy? Eg. You only need to see the request for an A record, not the response, or you only need to see traffic going to target A have suspicion and you work back to access or sync traffic locations with ISPs?

That door has no handle, no sign of a swipe card reader. Any ideas how the people who can would get that door open?

My guess...some kind of reader hidden in the door that triggers a mechanical device that lifts the panic bar (another comment's comment) or otherwise disengages some such other locking mechanism.

My guess is that they just use noclip and walk straight though.

My assumption is that this door is an exit — perhaps an emergency exit with a push bar. The room is big, 1152 square feet according to the article, so it probably has a real entrance somewhere else which couldn’t be photographed.

What kind of data do they have access to, now that most internet traffic is encrypted?

- Data which is not encrypted

- Data which is encrypted with a key they have access to (or which is encrypted with insecure methods)

- Data for which they might find a key in the future (If not yet secured by perfect forward secrecy)

Various leaks and whistleblowers have revealed that encryption alone doesn't provide as much protection as people think. There's all sorts of ways around the problem.

State-sponsored attacks against crypto are targeted, they don't care about your online shopping cart contents secured by TLS. They go for VPN connections, IPSec tunnels, and Tor.

A classic example is the Dual_EC_DRGB (Dual Elliptic Curve Deterministic Random Bit Generator), which uses a public/private key pair to generate the random numbers. The NSA pinky swears that they destroyed the private key, and don't know what it is. Nobody in their right mind believed them. Yet, the NSA basically forced a bunch of VPN vendors such as Juniper to include it.

Now, fine, okay, in theory Dual_EC_DRGB is safe even if the private key is know to an adversary, unless ~40 bits of the internal state is leaked during the connection handshake, which seems terribly unlikely. That was NSA's argument for why everyone should trust them and their algorithm. Unfortunately, guess what... the Juniper ScreenOS had a "bug" in it that just so happened to leak a bit over 40 bits of the RNG state into the handshake packets. Accidentally, I'm sure. Ooops.

It's also a safe RNG if the private key is destroyed, and is not known to anybody. But unfortunately for Juniper, the Chinese government hackers broke into their source control and replaced the public key with their own, matching a private key they know. So for a while, all Juniper VPN connections were being spied on by the Chinese government instead of the US government. I'm not sure which is better.

I've read similar stories about Cisco, Citrix NetScaler, etc... They all have purposefully weak crypto, government mandated back doors, and so forth.

The various western governments' fears of Huawei being used by the Chinese government to hack them is absolutely warranted: this is exactly what they would do given the same opportunities!

> now that most internet traffic is encrypted

And the encryption is getting better. For forever DNS queries were plaintexted on the wire, but now with most browsers adding a DoH feature, those days of passively sniffing DNS lookups on the wire are over. You might be shifting your queries to a centralized provider, but it's miles better than letting the NSA lift logs from an ISP.

Metadata

Also I'd be suprise if they don't have a copy or at least acess to all of the maj root certs and websites. Iean sure maybe Facebook or Apple or AWS takes a stand against the govt. Saying "No you can't have our data" but I doubt Verisign or other root level CAs are so scrupulous.

But that would only allow them to sign new certs, CA’s don’t get the private keys only the public part to sign... Or did I misunderstand you?

That's correct. To sniff traffic without replacing the certificate with one of their own they would need the the private key which was used in a session. (That key might have been derrived from the server private key, but again nothing the CA has access to.)

I will just assume they have a direct feed from FB, Google and Twitter too. And possibly Apple.

I've always wondered how many of these rooms exist, but also their geographic location. Are they in every state? Every major city? What about a city like Minneapolis? Or is it just the major cities with the backbone connections?

I'm also surprised I haven't seen more pictures and stories of where these rooms actually located (or maybe I just haven't been paying attention)

Can't discuss many details but supposedly where most of the major backbone interchanges are. So you to your neighbor across the street. Probably not going to be picked up at the packet level. You to some random WordPress blog. More than likely they got that.

I remember an article with a title along the lines of "The Room that Copies the Internet", and I think it was in Rolling Stone. I've tried finding it a few times over the last couple years and have been unsuccessful. Does anybody know of the article to which I'm referring and where to find it?

Don't remember that but there's this:

https://www.wired.com/2007/05/spying-in-the-death-star-the-a...

Is Jewel v. NSA still ongoing? The last update was in 2017 but did not sound like a decision.

> After many years of litigation, on April 25, 2019, ruling from the Northern District of California for Jewel v. NSA concluded that the evidence presented by the plaintiff's experts was insufficient; "the Court confirms its earlier finding that Klein cannot establish the content, function, or purpose of the secure room at the AT&T site based on his own independent knowledge."

> The ruling noted, "Klein can only speculate about what data were actually processed and by whom in the secure room and how and for what purpose, as he was never involved in its operation." The Court further went on to discredit other experts called upon, citing their heavy reliance on the Klein declaration.

How Kafkaesque.

(2006) needs to be added to the title, unless there’s substantial new information

Why would a Wikipedia article require a date? Someone else just posted a Wikipedia article about an ancient Aztec city but I think it would be kind of ridiculous for the title to include (1325).

Point of adding the date to the title is to provide context where relevant as it relates to the present. Generally speaking, goal of HN is to cover substantial new information on a topic - not repeatedly rehash topics. As is, this Wikipedia is not a good fit for HN unless their’s specific substantial new information and the source of that information would be the best URL, not the Wikipedia page.

Weirdly written article, readS like an SCP.

The NSA has gone way beyond that now. Expand "room" to fortress, and then multiply by 8.

https://theintercept.com/2018/06/25/att-internet-nsa-spy-hub...

It’s a little hyperbolic, those are generally just datacenters. They’ve got lots of fiber running through them, sure, but most of the “blacked out windows” is being dramatized.

Structural reinforcements as mentioned for datacenters are also fairly normal, they’re often designed to survive the worst imaginable environmental conditions for the area they’re being built into.

No idea why you were downvoted, this is completely factual.

Reply on news.ycombinator.com