Hacker News Reader

Comment by mindcrime | Hacker News Reader

I was in an AI conference yesterday, and there was a bunch of discussion in the chat about Slack channels, Discord servers, etc. So I was literally just lamenting the state of shared discussion forums, and commented that we all need to go back to Usenet and drop all these lame "walled garden" proprietary forums.

I'd like to tell people "install Thunderbird, sign up for a free Usenet provider, and join comp.ai, comp.misc, etc and have fun." But I'm a bit leery of encouraging people to do that since I'm not sure what the state of spam being sent to the various Usenet groups is, or how providers are handling spam filtering (if at all).

That said, there's definitely a place for NNTP / Usenet. It might take some work to fix up some issues, but it would be great to see vibrant / fruitful discussions via Usenet again.

View on HN

The spam is the biggest issue. We never solved it there, and it's only being controlled on usenet's imitators through tight control, which is not a feature of usenet.

One setup I could see working is a pubkey setup where anyone who posts, always encrypts with their private key, and anyone you want to hear from, you add their public key to your list of keys you decrypt to read.

Aggregators can occupy public keys as their "address" and any aggregator that gets obnoxious, you just trim it from your list, which only contains things you've either let in or accepted via slates or whitelists.

Then the spammers can do whatever they like, really.

I believe the main reason usenet faded so thoroughly was that it became a piracy distribution platform very early on (not sure which came first, the pirate booty or the porn booty, but I'm guessing porn), and very quickly there came a kind of unspoken "don't talk about usenet" code among those who already knew about it. At the time, Napster and bittorrent were the main targets of the authorities and IP trolls and usenet was just doing a Jim-From-The-Office-smirking-through-the-blinds.

It still does that.

shhhhh

Didn't a lot of servers just stop carrying the binaries newsgroups, or never bothered carrying them in the first place? Even without considering piracy, binaries tended to place a burden on the server provider since the messages tended to be much larger. (Even a small image/program in the 10's of kB would be larger than a heavily quoted message.

Yes, and so these days there are paid-subscription NNTP providers that sync the alt.binaries groups. Presumably all the pirates (both the uploaders, and the pure leechers) are using such providers.

https://easynews.com/

> I believe the main reason usenet faded

I believe it was due to AOL and CompuServe forums, which were most importantly, new, novel, friendly and easy. Technical people were "above" AOL and so those Usenet communities survived a few more years, but ultimately succumbed to web forums which were superior to Usenet in almost every way.

Unfortunately, web forums were mostly hollowed out due to social media, and now that people are sick and tired of that, the simplicity and "innocence" of early 2000's forums, and even 90s Usenet, seems appealing (although tinted by rose-colored nostalgic glasses).

Just my opinion but I can't think of a single way in which I found web forums of the early 2000s to be superior to usenet.

Yah no idea what they talking about me

Discoverability? A novice could type something into Web Crawler and get a forum back as a search result. Then they could click the link and begin reading and participating immediately. Was Usenet ever that easy?

There was a time when "Groups" was section of google search and they indexed the whole thing.

Think this is still around (at least the groups are) but the interface is terrible. Mailing list-tier

Even easier, no web search needed. All you needed to do was search the group list for relevant terms. All the Usenet clients supported searching the group list. Then you just tick the checkbox and the articles were downloaded.

Much much easier than using a search engine, scrolling through the results which were half ads even in those days, and trying out the 12 different forums you finally found which were even active...

> All you needed to do was search the group list for relevant terms. All the Usenet clients supported searching the group list

- Group list

- Usenet client

Most people had no clue what that stuff was.

That’s another reason why Usenet was better. There was an IQ test to see if you were above 90 just to get going with it.

And Usenet died, to no small degree, because it was heavily populated by snobs.

It never died, it reach a low point of participation about a decade ago, and has slowly been gaining steam again ever since. Some portions of Usenet have always stayed active even through the tough times. As social media dies due to various reasons, people are coming back. Have been for years.

> It never died

interesting, given that all your references to it are in the past tense

GP of the reply chain was referring to how at a point in the past Usenet was allegedly more difficult to access. That's why some of my responses were in the past tense - I was disagreeing about how, back then, it was more difficult to access. Hence the use of the past tense. All in perfectly proper English.

bla bla bla, Usenet died because the "high IQ" (according to you) snobs on it were (are?) annoying as fuck posting these twisted explanations instead of accepting they were wrong about something

There's still time to switch your career path. You don't have to do this.

Exactly. When I got access for the first time in the early 90s it took me a few days to wrap my mind around it but after that I never had trouble finding groups of interest. Tools like search, built in group name searches, etc made it easy

...and about 2% of the population thinks Linux is an awesome OS, easy to use, loves the command line, prefers "grep" to clicking on a search box. Fortunately using an OS doesn't really depend on the participation of others, but a discussion forum that is widely viewed as inferior to some other option quickly succumbs to Metcalfe's law and empties out.

Your opinion is, of course, valid, but you do recognize that it's shared by only a small minority, right?

Sure, but they made it as a statement of fact. As opposed to myself who pointed out it was an opinion.

Don’t know about superior but they were definitely an evolution of the concept and were more decentralized p

Usenet faded because ISPs all conspired to drop it, en masse, in the early 2000's. Used to be every ISP from Comcast down to the mom and pops had Usenet. Now it's not just uncommon, it's nearly impossible to find any ISP that has their own Usenet feed. And yes the excuse given to drop it was piracy.

I assume they dropped it because they couldn't monetize it, i.e. spam you with personalized ads and the like. Probably the same reason why RSS News feeds were dropped.

In Germany basically all ISPs never provided the binary groups, so piracy was never an issue. There also was no conspiracy to shut down the servers because of that.

And yet the German groups started a slow but steady decline after their peak in 2001: http://usenet.dex.de/de.ALL.html

I was active in the German Usenet back then and still remember that between around 2001 and 2005 the spammers and trolls took over and destoyed one group after another until they were completely unusable. I also mostly quit around maybe 2005.

So I highly doubt that Usenet would have continued to work if ISPs had just continued support it. Usenet only worked as long as everyboy was nice to each other, it would never work today without much better moderation protocols and tools.

It was expensive. I ran an ISP on the mid 90s, and it took up an expensive server and a disproportionate amount of my time to ensure we had good enough feeds for people to be happy, and so the moment demand was dropping it was very high on the list of things to get rid of.

Had peering been more on demand, rather than a firehouse, maybe people would have kept them longer.

I for a while worked on an aggressively caching NNTP server as an option because of the costs involved.

Yeah, people forget just how expensive both bandwidth and storage were in the late 1990s.

YouTube appeared in 2005 and was losing VAST amounts of money before Google bought them out. So, even in 2005, Usenet probably was still too expensive.

Usenet faded because the UX was terrible. It was common in the early 90s because it predates the world wide web, and most people back then were highly technical and could deal with the warts.

Everything moved to the web, and Usenet clients were hit or miss. Neither Windows nor Mac came packaged with a client for it, so it certainly wasn't easily discoverable for people who joined the internet later. They probably never even knew it existed.

ISPs became client-less after broadband became widespread. ISPs didn't want to write or provide software, they just wanted to provide data over basic cable/DSL. Even AOL instant messenger eventually faded, as it never really adapted itself to a non desktop centric web.

Usenet doesn't have a UX by itself, it's a protocol. The user experience is entirely dictated by the software used to access it. And Outlook supported Usenet all the way from the word go. Macs didn't even come with any email clients back in those times, but popular ones supported Usenet and there were also Usenet-exclusive programs available.

I've been using Outlook since 1997 and this is the first time I heard that Outlook supported Usenet. This is what I mean, even if it was theoretically supported, it wasn't discoverable. If you knew what to connect to and how to wire up Outlook, sure, you could get it going.

But let's take something else from the same era and provide a comparison for the average user: yahoo.com. You typed it into the browser, and you were instantly presented with several hundred interesting links. No config necessary, just click and go. The UX needed to be that simple.

Actually discovering good channels on Usenet took time and investment. As opposed to Reddit, for example, which used upvotes and decay based algorithms (also, see HN) to make fresh subreddits discoverable on the main feed.

I think it was Outlook Express rather than Outlook that supported NNTP. The early version of OE was even called "Internet Mail and News" or something like that.

Outlook express supported Usenet, but Outlook did not.

That’s why I had to retire my company’s internal NNTP setup - because at some point, too many users were just using it through the web and email gateway that it was cumbersome and made no senses to keep it running.

I've never had an experience as smooth and easy as usenet on Free Agent as the gui client. Everything else, forums, socials, etc, contain a subset of the features in that setup.

Gnus was a joy to use. Never has an application fitted me better.

Spam was the issue – there were extensions and initiatives to combat it, but it was a losing battle. It was a major cultural loss – Reddit can at its best approach it, but not replace what existed in the early nineties.

For years, Google Groups supported Usenet access.

Usenet faded because it wasn't monetizable, outside of subscription based binary providers (who were in large part turning around and pumping that money into the legal defense funds, because they were primarily servicing the swashbuckling community).

The platforms that are actively used are codependent. They demand you use them. They send you emails when you haven't logged in in awhile. They foment opposition just to keep you engaged, so you'll view their ads.

Usenet, on the other hand, lets you use it, if you feel like it, assuming you know where to look.

The only sort of people who would use Usenet would be those that make decisions for themselves, rather than doing what the marketers tell them. It never stood a chance against this sort of opposition.

Today, we could all run our own NNTP servers. The traffic on most newsgroups (excluding binaries) would be a trickle compared to the average broadband connection.

I ran my own news server in the 90's, receiving about a dozen groups over dialup with a UUCP feed.

I carried Usenet in 2000 and dropped soon after because the volume was huge amounts of binaries and nobody used it. No conspiracy necessary.

>...always encrypts with their private key, and anyone you want to hear from, you add their public key to your list of keys you decrypt to read.

I think you mean that posters would sign with their private keys and you would have a list of public keys that you would verify the signature against.

That, incidentally, is also a valid approach to email spam...

> One setup I could see working is a pubkey setup where anyone who posts, always encrypts with their private key, and anyone you want to hear from, you add their public key to your list of keys you decrypt to read.

The most important thing that web forums like this one enable is interaction between people who don't know each other.

People could auto-trust those who have been trusted by those they trust; this could have a user setting deciding how many degrees of separation they are willing to trust. Everybody's public key (ie. the ability to read their signed posts) is public, it's a question of which ones you choose to read.

There are many ways to skin this cat, but every one that's gonna work involves cooperation, rather than adversarial control.

I have a few people in my killfile so that I don't see messages from them but don't get much spam in the comp.* groups that I read.

The threading and only seeing new messages is a lot better in my Usenet client than any web forum that I use, including this one.

The threading and only seeing new messages is a lot better in my Usenet client than any web forum that I use, including this one.

Yep. The Usenet experience is nice. The only thing that really got me out of the habit of participating frequently was a combination of two factors:

1. My ISP quit providing NNTP access by default

2. So many other people moved off, that a lot of the groups became nothing but CfP's, spam, and maybe 1 actual interesting post per year.

But in the spirit of "be the change you want in the world" I guess I'll bit the bullet and sign up for a Usenet account somewhere, or stand up my own server and look into what it would take to get peering setup.

> CfP's

What's that stand for?

Call for Papers I'd guess

Yes, Call for Papers.

"Including this one" is a good example, for how the rules of how content is selected for presentation define the medium. Even with exactly the same people, exactly the same dang occasionally nudging people this way or that way, this place would be an entirely different if the effective feed composition wasn't the one it is. So much of the identity of these online communities is an emergent consequence of the mechanisms employed.

i notice that a forum for a productive effort gets much less useful if there is a "general" sub form. It seems to ruin organization permanently.

One could in stead have a "i don't know where to put this" section that forces the mod to move or delete it.

Then on the other hand, I know of some off topic situations that I consider very valuable, because any forum set up specifically for the topic in question would draw a very different audience.

> The threading and only seeing new messages is a lot better in my Usenet client than any web forum that I use, including this one.

Indeed! This is the other less-spoken flaw of walled-garden communication apps. Since they are proprietary, you're stuck with the very limited functionality the company has decided to implement (and it's always very limited).

With open protocols such as SMTP and NNTP, there's no limit to how feature rich the clients (plural, since there can be many clients attuned to different tastes) can be. And you can always pipe things to a shell for an infinitely extensible set of capabilities.

I find all the proprietary communication apps so frustrating, knowing that my email and usenet clients even back in the 80s had tons of more functionality and flexibility.

> sign up for a free Usenet provider

Sorry, but you make it sound like there are many. That allow posting, and free, I think I only know one.

Sorry, but you make it sound like there are many. That allow posting, and free, I think I only know one.

It's been a while since I looked. I could have sworn there were at least a handful, but either A. I might be mis-remembering or B. the landscape might have changed since I checked last.

That said, there seem to be quite few who offer cheap if not actually free access. And by "cheap" I mean, on the order of $10.00 (USD) / month or less.

I don't think 10USD/month for access to a forum is anywhere near cheap or even reasonable, no matter how big it is. I can't imagine anyone paying that for non-warez Usenet access.

That pricing is definitely targeted at users who use usenet to trade binary files.

If you do not care about binary files then use https://www.eternal-september.org, it's free.

Get a provider with a block account... 1GB will last forever on text-groups, and block accounts usually go for 10-1000GB for a few bucks

That seems rather expensive given the service being provided, i.e. read-write access to a text-only discussion board. Where's all the cost going?

Most paid services have binary feeds. Those few, who offer text-only groups don't have big enough client base to make service cheap.

In the last few years some Usenet providers are closed their doors. Aioe.org was pretty popular and known, disappeared this year. One of few ISP-s still offering Usenet access for clients, Finnish Elisa, closed their usenet server 2021. Russian neva.ru was sadly closed (probably forced by goverment) 2021 (I group it with free servers, because it had read-only access open for everyone and some point allowed posting). Albasani.net disappeared 2020.

Some free servers are still around, but don't offer public accounts. Few places give account, if you ask. Eternal September is strongest-biggest still running and free service. Mixmin allows anonymous posting, but sometimes restricts it to fight with abuse. Google still relays groups to Usenet, though they have their own thing going and some servers are blocking posts from google.

https://sdf.org/?faq?USENET?03

Not free, but a few dollars for ARPA membership for life is good.

I'm on sdf-eu. Not sure SDF has external groups anymore, they may do. Just double check because the website has had outdated info before.

AFAIK the best options are https://www.eternal-september.org/ and https://usenet.blueworldhosting.com/ (read only though).

http://www.eternal-september.org/

> Subject: Free News Servers Listing

Web interfaces:

https://www.rocksolidbbs.com/rocksolid/article-flat.php?id=8...

https://www.novabbs.com/rocksolid/article-flat.php?id=715&gr...

Check the alternative hierarchies and connection options while you're at it.

The difference, I suppose, is that Google Groups are run by autopilot with nobody in the cockpit, Eternal September doesn't ask the average user too many questions (it is as accepting as its name suggests), but those small server admins will ask you questions if you start posting crap, and will delete it, and ban the account.

The thing you want exists. It's called the Fediverse. Don't like the politics? You wouldn't have liked Usenet in its heyday either.

The thing you want exists. It's called the Fediverse.

For the most part, when people talk about "The Fediverse" they mean ActivityPub based sites. You could argue the point, since NNTP is "federated" as well, and one could certainly conceive of federating between the NNTP space and the ActivityPub space.

I'm a Fediverse user, fan, and advocate (mindcrime@fosstodon.org), but I specifically mentioned Usenet, because the "thing I want" in this context is, in fact, Usenet. Now if somebody wants to do the work to rebuild the Usenet hierarchy on top of the "wild west" that is the litany of servers on the Fediverse, build client support into Thunderbird, etc., then sure... that could probably turn into something interesting.

You wouldn't have liked Usenet in its heyday

Please don't presume to tell me what I would or wouldn't like. That's incredibly disrespectful. And in point of fact, I did like Usenet in its heyday.

I’ve been very happy on the Fediverse (mastodon) since I took the plunge a few months ago. This is what the “early” internet at its best must have felt like. Real humans, technical discussions, random finds (painting!), sorted by created at desc.

> sorted by created at desc

Interestingly, this seems to be amongst the most controversial aspects; some people _hate_ it.

Personally, I was only quite dimly aware that Twitter had an algorithm now, like a common Facebook; until Musk ruined it I generally only really used Twitter via Tweetbot, so was getting a time-based feed anyway. So Mastodon was just what I was used to anyway. But more people than I expected actually liked the Twitter algorithm, and couldn't cope with Mastodon's lack of magic ML stuff at all.

Same. It's a bit "random" though. Maybe because the Mastodon world has generally resisted having good global search. And even though certain instances have broad topics (like sigmoid.social for AI/ML, or fosstodon.org for F/OSS) a feed tends to quickly fill up with plenty of random stuff.

With Usenet, if you join, say, comp.ai, you know that (spam aside) you're just getting AI stuff. Same for comp.linux, comp.lang.c++, or whatever. There's something to be said for the topic hierarchy there.

I see a place for both ActivityPub and Usenet, personally. Although, again, acknowledging that somebody could probably in principle build the Usenet style experience on top of ActivityPub. But as far as I know, that part doesn't exist today. If it does, somebody please let me know.

For forums over ActivityPub, there's Lemmy.

It's not really like usenet, if only because of the character limits. This really changes the nature of the discussion.

Yes, having images available within mastodon is nice - that was always clunky with usenet.

I’m on infosec.exchange, our limit is 11000 chars. That’s more than enough.

My understanding is that "typical" Mastodon/ActivityPub instances limit you to on the order of 500. Certainly true for Fosstodon.

I never used the Fediverse, but is all/most trying to be a Twitter clone?

Mastodon is like that. Then you have Kbin and Lemmy which are more Reddit-like, and Pixelfed which is supposedly similar to Instagram.

It's been a while but I don't recall a ton of politics on the usenet groups I used to participate in (mostly tech subjects). Of course there were groups for politics, and sometimes a thread would go off on a tangent, but the nice thing about usenet is it was easy to killfile people or threads that you didn't want to read anymore.

When I used it it seemed like it was mostly real names (.edu accounts) at least on the groups I read, so maybe that kept people in check a bit. Though that would have been easy to spoof I'm sure.

My department also had some local groups that did not propagate. So it must be possible to stand up your own NNTP server and have groups for your local users that are at least in that sense "private"

The user was talking about internal group politics/drama. Not national electoral politics.

Usenet was sorted by topic. Same people stick to the required topic in different forums.In the fediverse, you follow people, and therefore can't control very well what people post about. I follow some people because I am interested in technical topics, but I end muting some of those because some are very political, and I am not interested in some of these topics.

I don't think the Usenet model can be replicated on top of Fediverse. Not everyone sees instances as a topic thing (I host my own instance).

Hm. Wonder if any newsreaders could "follow" a person, regardless of the group he posted in. I don't recall that feature existing in rn or trn.

Lemmy is also built on ActivityPub as part of the Fediverse and is arranged into topics (called communities). E.g. you can follow retrocomputing@lemmy.sdf.org (or web interface https://lemmy.sdf.org/c/retrocomputing) and only see retrocomputing posts.

Since it's all ActivityPub, you can follow Lemmy communities from a Mastodon client but the UX in that case is pretty bad (e.g. Mastodon servers won't backfill posts so you won't see any history).

> Don't like the politics? You wouldn't have liked Usenet in its heyday either.

Or just join an instance that is not explicitly political. You don't have to be on beehaw or hachyderm.

Literally perplexed at that statement. That was the only place for solid community, damn good information, and endless free file downloads in it's heyday. The barriers that made people drop it actually make it more attractive these days since the masses will never show up.

The organization of Fediverse is different, and availability of historical discussions is spotty.

Also, some quirky usenet groups aren't available elsewhere, such as alt.binaries.*

Most Usenet servers had strict retention limits; there was no expectation of availability of historical discussions.

I think you're coating your memory with a thick blanket of woke. I was also around in its heyday and the trolls were in full effect, and had already learned how to make women's lives hell.

There's a newsgroup still extant which is named after a woman who spoke up about the CSAM problem on our local university's server (I stumbled across it myself once, there was some nasty fucking shit going through there). The users of these groups did not take kindly to people messing with their access to pictures of children being sexually abused. Not naming her or the newsgroup because I'm sure she's long since gotten it in her rear view mirror.

ActivityPub servers own your account. This is a non-starter for me unless someone holds server admins accountable to laws or some kind of liability

Just run your own server. The situation was no better with NNTP.

sign up for a free Usenet provider

Any suggestions?

https://www.eternal-september.org/

Just a warning: Eternal September's administrator doesn't give a damn about trolls and spammers posting through that server; every single complaint I filed went 100% ignored.

It's worse than that. They won't even take action against harassment of moderators, including their users that repeatedly submit obviously rejectable articles to moderated newsgroups, then reply with verbal abuse and profanity when rejected. The eternal-september admins tell moderation teams that it is their responsibility to deal with these users, not them. Our moderated newsgroup dealt with one of their users (lawfully, of course). No surprise that the individual had an anonymous identity that was a synonym for the devil.

The registration system seems to be borked. Any others?

It worked for me, but I had to disable adblockers.

The free "10 GB" trial accounts available from usenet binaries providers can serve as a text only read/posting account for years. Like with usenet.farm

The free account on usenet.farm does not allow posting.

Ah geez. You're right. Thanks. I set up one for a back-up for when eternal-september is down but never tried actually posting on it.

> since I'm not sure what the state of spam being sent to the various Usenet groups is

Isn't this why some groups were moderated?

Is there a free provider you’d recommend?

No, see above: it's been a while since I looked at the usenet landscape. TBH, I'm tempted to set up my own, but I'm not sure I have the time/energy/money to deal with the peering stuff. But never say never...

Eternal September, mentioned in the article, is good. Another that I know of is https://solani.org/ .

I'm surprised nobody made fun of you with the typical ageist-conformist fashion nonsense that permeates tech discussion these days

Reply on news.ycombinator.com