Hacker News Reader

Comment by p-e-w | Hacker News Reader

There’s no way a single datacenter costing a couple of billion dollars can store “all Internet traffic the NSA can access”, unless the traffic the NSA can access is a microscopic fraction of the total Internet traffic.

Think about it. The Internet runs on tens of thousands of massive datacenters. Thousands are being built as we speak. Obviously a single datacenter cannot hold an appreciable fraction of that.

BTW, the total budget of the NSA is less than the R&D budget of a FAANG company, so if you find yourself believing that they might have alien-level technology far beyond Google and AWS, you’re watching too much TV.

View on HN

https://en.wikipedia.org/wiki/Fairview_%28surveillance_progr...

https://en.wikipedia.org/wiki/Room_641A

Public information shows that the NSA has been active intercepting as much data as possible.

It doesn't require the budget of a FAANG to peek through a significant volume of internet data.

That’s not what the GP comment said though. They claimed that they are storing everything they can intercept, which is weapons grade horseshit.

People said/wrote in the past that what NSA and other intelligence agencies did was to gather data and store the meta data. The actually traffic got processed for the meta data, and small amount of the traffic got sorted out and also stored.

I would suspect that today they also process the traffic for llms and thus store a bit more of the traffic as weight and biases. All that can be done distributed and to different degrees based on how much access they got and under what operational conditions.

Regarding storage, a single data center using only slow but dense storage (magnetic tapes) can store far more data than a data center providing regular web services.

“They’re not as good at violating your civil liberties as you think they are” isn’t very comforting. They are still doing it at scale and i’m not too keen on opening the back door for them myself.

> The Internet runs on tens of thousands of massive datacenters.

"The Internet" would require 1000 times less servers if it wasn't running off Python scripts in Docker containers in VMs in a virtual overlay network. (I'm exaggerating these numbers only slightly.)

Sure. But a government agency has secretly rebuilt all the same infrastructure without Python, on a shoestring budget compared to what Big Tech is spending, yet it’s a lot more efficient than what they use, right?

Caveat: I have zero experience with USA government agencies. The spying tech from other countries I'm familiar with are beige box routers made by network engineering types with embedded firmware written in C inside.

I think spying on traffic is just a massively simpler task than generating content.

It doesn't need to store the payloads itself. It stores the metadata of connections and probably the fingerprints of the content passing through the Internet. "The headers of the whole Internet" could be physically stored in a single datacenter.

Reply on news.ycombinator.com