usenetthrowa

FreeBSD has a kind, international team of developers. It may be a scary project to contribute to for beginners to open source because it is an operating system, and people have some apprehension about improving operating systems.

Follow-up:

> One posts individual messages (like one email) to a group

Messages can be posted to more than one group; this is called "cross-posting", much like with email mailing lists.

> The end result, with a good news reader program, is somewhat like a large global collection of topic-specific forums.

You can also choose to "subscribe" to a subset of the available groups; your news program is able to pull metadata only for that subset you care about. You can also download individual messages on demand, much like IMAP email access. It's fairly bandwidth efficient for clients.

Commercial binary NNTP providers have basically infinite retention at this point, if your content doesn't generate DMCA takedown requests. E.g. http://www.news.astraweb.com/ $10/mo gets you 2660 days (>7 years) of retention, growing at roughly one day per day. (Four years ago, 3 years of retention was common.) There are also pay-by-download-in-GB a la carte plans that would be good for backup-only use (because upload is free).

Even if it isn't infinite, you can download and repost every 7 years if you care about having backups for longer than that.

I wonder if you could convince cperciva (Tarsnap) to exploit Usenet as an encrypted block store (as a cheaper alternative to S3).

> I thought this would be a nice alternative use for binaries on Usenet instead of piracy stuff. But I never released it because I think that it would lead to pollution of the Usenet network.

> Is anyone interested in this? Or maybe somebody has an idea on how to use this without polluting the network?

No way around it, it's absolutely an abuse of the network. That being said, so is the piracy on the binary subgroups. I think the end use would be small enough to not materially affect the binary NNTP hosts anyway.

One-stop shopping and the legal "protection" of not appearing on a bittorrent peer list. (It's still illegal, but less detectable.)

The media companies have largely caught on to Usenet and have been issuing automated DMCAs on Usenet content immediately since the early 2010s. Since the posts are stored on a centralized server, the providers can and do comply with DMCA takedowns with automated software. As a result, popular TV and Movies with correct titles are usually taken down quickly.

Usenet (NNTP) is a protocol somewhat similar to email. Unlike email, the default communication style is in "groups" (sort of like a mailing list for a specific topic). One posts individual messages (like one email) to a group; replies are threaded, much like mailing list discussions. Historically the groups were organized somewhat hierarchically.

The end result, with a good news reader program, is somewhat like a large global collection of topic-specific forums.

Binary posts are just a bunch of posts to the same group with a message Subject matching some pattern (usually with ascending numbers counting the posts). The content is encoded to work around the limitations of the protocol and/or provider. Historically this was "uuencode", but you can also do base64 (like email attachments) and the new jazz (hah! New in 2003, anyway) is "yEnc." yEnc relies on 8-bit safe NNTP servers and only encodes the few characters not permissible in NNTP messages (periods at the beginning of lines, I forget the rest). So you get much better encode ratios (~101-102% of original size) than with something like base64 (133% of original size).

NNTP providers, especially in binary groups, are subject to occasional message loss. So posters will typically generate Reed-Solomon forward-error-correction blocks using a program called "par2." This allows one to download binaries that are missing some messages, as well as some additional FEC (potentially on an as-needed basis), and rebuild the original content. Generating and recovering from par2 files is extremely computationally expensive.