Daily Usenet Feed Size Hits 300TB
newsdemon.com
newsdemon.com
I use both types. Text groups almost certainly fell off a cliff earlier this year when Google Groups shut off their spam posting gateway. http://www.eternal-september.org/ is a good free project for the text groups. I've been a newsdemon customer for years but they suck for text groups (their headers got messed up a few years back after a move from highwinds backbone to usenetexpress backbone).
It seems so fundamentally ill-suited to the task.
And if the answer has something to do with privacy or warnings from your ISP, it seems like VPNs would be the answer.
What am I missing?
No need to seed which reduces legal liability and bufferbloat issues on lines with anemic upload speeds. Most torrent clients penalize peers that don't contribute to the swarm. It's also really convenient to plug in automated downloads with a decent index and sabnzbd.
I agree with you and I also want to add that I've had problems with trackers penalizing me because i can't get my ratios up since hardly anybody is torrenting the same outdated 1980s mecha animes i am. I don't usually have troubles finding or downloading them because there's always at least one guy who has it on a seedbox running 24/7, but my ratio is guaranteed to be stuck at 0.0 forever.
I actually got autobanned from a private tracker once because i downloaded a bunch of comic books which were all miniscule in terms of size and also extremely niche. this series was being distributed as one issue per torrent instead of a big archive torrent, with each torrent being on the order of a few MB. which resulted in me having several dozen torrents with a 0.0 ratio and therefore looking like a leech.
Small trick. Pick one, popular, item that everyone wants now, pull it down and seed it. That will add to your ratio which can then be used to grab the obscure stuff that is hard to seed.
With the exception of the rare, well seeded torrent, a Usenet download will proceed at full speed while the torrent will trickle along at 34kb/sec.
> It seems so fundamentally ill-suited to the task.
It is. But it is there, so....
Of course, private trackers are superior when it comes to older content and general availability (no DMCA or NTD to worry about).
Legal differentiations in some places between downloading and distributing.
On other hand with torrents you usually automatically upload. And even single uploaded copy can result is some extremely questionable math of potential damages. Which the legal system in essence has blessed.
Just like I can send you a copyrighted image via email and the email server owner isn't liable for the material on their drive.
Also it's hard to track compared to the ease of monitoring public torrent swarms.
With torrents you can just enter the swarm, log all IP addresses and maybe download a bit that it's actually infringing material.
With news the stuff is ascii jumble spread in posts with weird names and you need to download it all to figure out if it's what it says. And just looking for material from the 300TB backlog is a huge pain.
similarly, soulseek is limited to 1 peer at a time, which some might consider "ill-suited to the task"
anyways I love slsk
clients exist:
100% (down) bandwidth saturation, no VPN, no letters from lawyers, and no using my (asymetrical) upload - what is not to like?
You need to find an indexer, the best ones you need to pay for or get an exclusive invite.
Then you need to find a news provider, and pay them too.
On top of that you need software that can plug into the indexer and the server and actually download the stuff.
In addition to the other answers, Usenet binary newsgroups are much older than torrents. So another reason is that it simply never died. Torrents didn't replace Usenet, they just complemented it.
Downloading from Usenet is not like downloading from torrents; it is not p2p. One downloads from the Usenet service provider over SSL encrypted connections. There is no way for anyone else to track it.
Secondly and independently, Usenet providers often don't keep very many logs, so there may not exist anything for the server owner to provide in response. For example, if the Usenet provider gets ordered to list all users that downloaded a particular movie, the provider could respond that they don't keep such records.
Also, think of all the pirate streaming websites that exist.
The worst that can happen here is that a court orders someone to pay a fine for downloading something, and this would make big news if it were to happen for Usenet, etc. It hasn't happened to anyone afaik.
Mullvad and I’m sure other modern VPN clients have “kill switches” built-in that shuts down traffic if the tunnel isn’t on. You can also do a leak check before starting anything up.
Having wrote that, another option, under Linux with network namespaces and Wireguard it's possible to have a pretty fail proof VPN...
I'd be interested to see WHY this is the case. Is it attributable to a larger share of data that cannot be compressed vs more compressible data (e.g., Warez/Movies)?
It just seems highly unlikely this is driven by a growing user base; but, without more details other than this data table, I am at a loss for the reasons why.
> It just seems highly unlikely this is driven by a growing user base;
If you are old enough, there was a time when everyone pirated stuff due to the alternative being rather expensive or unavailable (physical media). Then a golden age of streaming services that were cheap and had high availability basically killed torrenting for the general public. Now people are returning to piracy as the streaming services got worse for reasons I stated above.
The people who still care enough to pirate in the era of ubiquitous streaming are often doing it explicitly to get the best possible quality, because no streaming service now offers anything of comparable fidelity for any price.
gotta hope they decide your connection is capable of 4k (and they decide they want to spare you the bandwidth)
“This program posts news to thousands of machines throughout the entire civilized world. You message will cost the net hundreds if not thousands of dollars to send everywhere. Please be sure you know what you are doing.”
That's pretty insane lol. How many mirrors are there that can actually manage that much storage? If you're using 20TB disks that's ~5500 disks per year with zero redundancy. Double or triple that for a bare minimum... not counting the load of actually serving that data to everybody.
How is this economical for anybody at this point? Or are these Usenet mirrors all massive businesses that can support running hundreds of PBs or storage and I'm just naive?
ding ding ding. The market consolidated toward only a few large backbone operators. Here's a picture of what the network looks like: https://svgshare.com/s/14tF.svg
But I've never heard big stories about the government going after big Usenet providers. Do they? If not, why not? Or does it just not make the news? Or am I just not paying attention?
Indexers are separate entities. Those are the ones who catalog and host the .nzb files. You see those go offline far more often but Usenet is still pretty small compared to torrents so they don't get reported as much.
The most litigious government has a 'safe harbor' provision most of the major Usenet backbones comply with: https://www.copyright.gov/512/
And they do take down a substantial amount of content every day, so while everyone knows piracy is common there, what more are they supposed to do? At this point the main effective thing governments can do to make it annoying to use Usenet for piracy is target the indexers (and they do this, but more always move in to replace the ones that are shut down)
After that, deduplication can probably bring you down at least another 50% - how many of these files are just the same things being posted over and over? Probably quite many of them. Store a file once with a database tracking its SHA1 hash, and whenever you see a file with the same hash come in, throw it away and instead store a reference to the first file.
A backchannel to download papers from sci-hub?
Or are people using Usenet as a way to send encrypted messages in a way that makes traffic analysis more difficult? (If 50,000 people download everything to a group, and post encrypted or steganographic message to that group, then it's easier than seeing that X sent an email blob to Y.)
Or, Usenet as the new numbers station?
Is there really even that much media produced everyday? Or media that does get uploaded?
Any particular piece of content might be uploaded 50 times or more with different resolutions, codecs, sound formats, languages, etc.
When it comes to text-only groups there’s still some spam but a lot of it is cranks howling into the void about their insane world views rather than unsolicited and off-topic commercial offers, though those do still exist too.