How does Black Lotus Labs global telemetry know which IP communicated with which other IP if they have control of neither endpoint? Who/what is keeping traffic logs?
If these guys can do it, remind me again how Tor is secure because nobody could possibly be able to follow packets from your machine, through the onion hops, to the exit node where the same packet is available unencrypted...
Tor does defeat this though. Rather than seeing the true destination of your traffic they see that of a Tor exit node.
Seeing a packet heading to a tor exit node and then a similarly sized packet heading onwards a fraction of a millisecond later is a pretty surefire way to spy on individual tor users.
If 1, 2, or possibly all 3 nodes are run by a malicious actor, deanonymization becomes easier. At one point 10% of nodes were run by a single malicious actor: https://therecord.media/a-mysterious-threat-actor-is-running...
IIRC There is an alternate method of connecting to an endpoint which uses a 3rd node as a rendezvous point which is meant to be better, but I forget the name of the process...
Of course, on the other end of the spectrum, the NSA has tapped into core internet links, is recording everything it possibly can, and is keeping it forever.
Backbone routers have no need to implement stateful TCP inspection or deal with the transport layer for TCP, dealing with IP is enough.
If we are generous and assume there a zettabyte of data a year that they want to store.
At consumer prices, you would have to pay $10B per year just buying hard drives yet alone the operational costs/redundancy.
The budget for all of the US intelligence services is ~$65B. I think if they wanted to actually do what you are describing it would be the single biggest intelligence expense they have and I don't see how you hide that.
i.e. they can prove I visited a bunch of cloudflare sites, but there are millions of those so who cares?
Hopefully this is made clear in Windstream's contract terms.
So to capture this, you at a minimum need to be logging every TCP connection's SRC IP and DST IP.
And they seem pretty confident in their worldwide map and fairly exact counts, so I would guess they must have probes covering most of the world doing this monitoring, and it likely isn't just 1-in-a-million sampling either...
This is common in datacenters for traffic and flow analysis for troubleshooting, capacity planning, and the occasional incident response.
More details: https://en.wikipedia.org/wiki/IP_Flow_Information_Export
You're supposed to be protected by the fact that you're going through multiple nodes before exiting TOR, and traffic should be mixed. Can you find some streams if you have most/all the nodes within your network and can analyze the traffic? Probably some, but the more traffic a node handles the harder it would be.
There is a simpler approach though, which is to just run exit nodes.[1]
1: https://en.wikipedia.org/wiki/Tor_(network)#Exit_node_eavesd...
I believe it does. I believe each destination IP you connect to uses a different exit node. And that'll even switch over time. And even if you connect to 2 destination IPs from the same exit node, I don't think there's a way for the exit node to know those 2 connections were from the same user.
both for safety and ending abusers.
It's likely they just do sampling (think netflow) to get some statistics over the data that's already transiting their network.