I don't believe it. Unless someone is running bots that watch netflix/porn all day, there is no way that they are consuming half of all traffic. Half of all posts? Sure. Half of all webpage requests? Ok. But half of internet traffic?
Our data show in the first half of 2021 bandwidth traffic was dominated by streaming video, accounting for 53.72% of overall traffic, with YouTube, Netflix, and Facebook video in the top three.
https://www.sandvine.com/hubfs/Sandvine_Redesign_2019/Downlo...
And looks like Sandvine is very credible as countries relies upon it to censor internet in their countries.
American Technology Is Used to Censor the Web From Algeria to Uzbekistan
https://www.bloomberg.com/news/articles/2020-10-08/sandvine-...
Sandvine's primary business is traffic shaping and bandwidth management for cellular networks, shipboard networks, and other places where you need to do intelligent QoS. They are the reason Google Maps still works well on your cellular connection while you pass a house using cellular broadband to download a torrent.
The fact that some people use it to censor is an abuse of technology, not the intention of it.
I'm guessing it's half of "connections", including DDoS attacks. But even then, I wonder how reliable their methodology is. Like are they including port scanning here, when a connection isn't even made?
Now the code is a hot mess, sure and I am partially to blame for that but that is kind of besides the point. We don’t need to serve any more than thousands of concurrent users which the website can handle but we have to basically ban Chinese traffic to stay online.
Maybe people at bigger companies already know this but it was a revelation to me how much it takes just to stay alive in production.
I anal and I definitely don’t know what they get out of crawling every single product detail page on our website multiple times a day. Nothing here changes that often. Maybe they have some bad/overzealous code? Are they looking to attack take over our servers to them attack others with our machines? If it is an attack, why use Chinese IP addresses? Why not use their bit farms? If it is legitimate search engine, why not respect robots.txt?
Because there is nothing, you can do about it anyway?
Even if you could proof, it is an attack, do you really would consider sueing some chinese IP adresses?
But I rather suspect, it is just bad crawler code.
[1] $9.45/GB at brightdata.com/proxy-types/residential-proxies
There are a handful of bots that mirror/archive multimedia content that is anonymously accessible. There is no way those bots have the storage capacity to mirror even a single pass of all the anonymous content.
What I have seen increasing exponentially is port scanning but that takes up almost no bandwidth. Even the broken scanners that in effect look like an amateur DDoS only utilize about 15kb/s using dozens of CIDR blocks at the same time. That does not even remotely hold a candle to streaming.
The link below [1] is talking about Netflix as a percentage of internet downstream traffic and this is only Netflix. There are now hundreds of streaming providers and according to Sandvine streaming accounts for 65% of internet traffic. [2] This does not include torrents and other file sharing.
Here [3] are some fun stats. One of them backs up the submission but it isn't clear if they mean requests or bandwidth. Given that Netflix or streaming alone is 65% of the bandwidth that would lead me to believe the issue of this thread is a lack of clarity around bandwidth vs requests. The wording on all of these sites is too Wibbly Wobbly.
Every day, the internet generates more than 2,183,908 tons of CO2 emissions.
Internet traffic statistics show that 51.8% of all traffic is generated by bots, while humans account for only 48.2%. I think they mean requests, not bandwidth.
[1] - https://www.makeuseof.com/tag/how-much-of-the-internets-band...
[2] - https://www.tubefilter.com/2023/01/20/sandvine-video-data-ba...
And I can certainly imagine bots fetching Netflix or Porn or whatever video for personal archival purposes (I use youtube-dl to protect a few videos I really like from the vicissitudes of Google myself).
When cost to broadcast for anyone on the net falls to 0 everything turns to shit.