Usenet rewind archive search engine | Hacker News Reader

Usenet rewind archive search engine

usenet-rewind.com

138 pointscstadler186948 comments

A similar one from earlier in the year:

Usenet Archives

https://news.ycombinator.com/item?id=47655905

I'm trying to figure out which one has more messages

You can visit only 25 pages on the website with a free account, that's a bit too limiting. I'd suggest limiting the number of searches maybe but not the pagination over the results.

I got curious again to find my old posts and figured I could download and grep the mbox files from Internet Archive instead.

Ah this has reminded me of dejanews.com the usenet archive. Before the days of Stack Overflow that was absolute gold for looking up an error message or code issue.

If I remember rightly Google bought the data when they shutdown and incorporated it into Google Groups.

And then axed Google Groups and refused to release the data. Typical Google.

you can bet any spicey stuff went into your dossier, too

Try https://usenet-rewind.com/

Oh my god this is a lifesaver. Just last week I was using Google Groups to search for old Usenet posts about the Commodore 64 and the Mimic Spartan, and I was getting "too many requests" errors 4 out of 5 times clicking a result. Happened on VPN+incognito, and in a separate browser. Just searched for what I was looking for in Rewind and immediately found the posts and even some I never saw in Google Groups results. So glad all of this is archived in multiple, non Google places

Not to be a party pooper but this is a paid service. I am sure there are reasons to pay but I bet their archive is the same as https://usenet.blueworldhosting.com which is free. A free web frontend can be found at https://newsgrouper.org

It has a free account has 25 full searches with advanced filtering etc.

Do those services have many more posts than what can be downloaded from archive.org and searched locally?

Alas, https://newsgrouper.org is not available from the UK (another casualty of the stupid OSA.)

Expensive, too. $40/month.

Newsgrouper is banned in UK for "safety concerns"

Newsgrouper is my site. It's not literally "banned", but after spending quite a bit of time studying the Online Safety Act and Ofcom's guidance, I concluded that it was just not practical to comply with its far-reaching requirements, and since I am in the UK I can't risk ignoring UK law.

Ah, the word used on the site is "Blocked" not "banned". Apologies.

A shame that you decided to not support your own country. Imagine how much you could have done whilst explaining this issue!

I run newsgrouper.org as a hobby project. My architecture cannot support searching within article text, only by author, subject and/or date within one group at a time (or the set of groups you have subscribed to). Usenet-Rewind does support full-text search.

Just found some cool old stuff thanks to this:

1996-01-17: first post! a reply to a thread in alt.music.bjork

1996-03-11: first mention of my website/homepage

No links to avoid my own embarrassment.

> No links to avoid my own embarrassment.

There are only 7 posts in alt.music.bjork on the first date, and one of them has your name on it. There's 1 post with your name on the second date, and it gives a website. The earliest wayback entry for that website redirects to another wayback location that exists.

Well done.

Upvoting for visibility ;)

Selfishly, I hate these things. No one in the 1980s and early 1990s expected the internet to become what it did. I discovered it as a dumb teenager and posted some frankly idiotic stuff. I also put my real address and phone number in the signature, because that's how you rolled back then, the entire internet was like 10 nerds.

Now, 35 years later, a lot of other stuff has mercifully decayed, but people keep bootstrapping these Usenet archives as pet projects every year, and there's never an option to opt out. Today, agents make it easier than ever to build a dossier on anyone you don't like, so this data is bound to cause some harm. And I guess when I die, the best-preserved memory of me that my grandchildren and grand-grandchildren will be able to pull up will be snarky posts from a 15-year-old on some unix-related discussion group.

Yeah, I know there's a bunch of posts there that are of genuine historical interest. But I'm just not a fan and I don't mind burning karma to speak my mind.

Man, I sure knew 35 years ago that whatever I shared in public was.... public. :)

And some of it still is public from way back then -- including dumb and ugly comments, stupid flamewars, teenager stuff, and phone numbers. It doesn't bother me.

I don't feel remorseful for having once been a kid and doing some of the dumb things that kids sometimes feel inclined to do, and those landline telephone numbers stopped being relevant to me decades ago.

Public or not 1991 was a very, very, very different time. The idea that some dumb stuff I posted would still be around in 30+ years would have been crazy.

(Actually, I wonder if the Prodigy forums got archived...)

Those were semi private no ? I would guess not. Although maybe there's an old server sitting in someone's garage with the info

I think this is the best we've got - but seems moribund. Relied on finding people's old C:\PRODIGY directories and finding what was cached

https://www.prodigyreloaded.com/

We're still working on it! It is worth noting that the STAGE.DAT an CACHE.DAT from C:\PRODIGY will never contain things like email messages or forum posts. They will contain things like the pages and program logic that comprise particular parts of the service, and at most parts of the various service database indices and records (e.g., bits of the Encyclopedia, Movie Reviews, that sort of thing).

There's a discord server for the project here: https://discord.gg/2GpsEC5JdP

35 years ago we didn’t have the cultural precedent/zeitgeist of getting cancelled for some stupid shit you wrote on a message board in 1991 so we were less guarded with our opinions.

Do we actually have a culture of cancelling people for the things they wrote when they were 12?

No, and we don't really have a culture of cancelling things for people they say now, either, when it comes down to it because if that were true, Trump would not be president given his long history of [waves hands], a large number of Republican politicians would also be out of work, etc.

So what is it that drives such phrases into common parlance as a description of the apparently-fictitious culture of rote cancellation, might one suppose?

I might suggest that "projection" is a primary motivator, although I work to avoid projecting on my own time.

(Edit: To disambiguate without vagueness: When some fucking party or other is talking about "cancel culture," they usually seem, to me, to be projecting about their own fucking party's fucking ugly policies and tactics.

And in a sane and pure world, I'd like to wish that they'd be recognized as being fucking scumbags for behaving in this way.

But it is my observation that we do not always live in a sane and pure world.)

I can imagine you’d be embarrassed, I did stupid stuff too, but unless you’re running for Congress you should just say “I was a stupid teenager” and let it wash. Anyone fair minded will dismiss it.

The current social zeitgeist is that one is permanently responsible for everything they ever said, even has a child. All nuance is gone. People are looking for a reason to make someone the enemy, and if you're the enemy, violence is acceptable now. I wish it weren't like this, but it is, so I can understand why people want to stay out of the crosshairs.

My view is the complete opposite. In the 90s I thought the digital revolution and the internet will be so groundbreaking that everything connected to it will be remembered. Moreover storage seemed so cheap, why would we ever want to delete anything. The internet never forgets.

Turns out storage wasn't free and the the following generations did not have the same appreciation for everything 80s and 90s, so much of it is lost forever.

Same. It’s hard to understand today, but in a world where search engines weren’t really a thing and almost no one knew what the internet was, the concept of “public” was quite different.

Yes, I should have known better in my teenage years about it. And also yes there was no reason for me to expect that everything I wrote would be easily available to some AI panopticon 30-40 years later

Wild that I put my street address in some of my posts. Those were different times indeed.

> when I die, the best-preserved memory of me that my grandchildren and grand-grandchildren will be able to pull up will be snarky posts from a 15-year-old

Just make a bunch of really good posts and put your name on them

Where is the source of all these Usenet services? I thought people just stopped mirroring when Google groups got so prolific? I wonder how big all of this is minus the *.binaries

Grabbed the UTZOO tapes off archive.org a while back, ~2GB compressed for 1981-1991, and recoll indexed the whole thing in an afternoon.

Are those still up? no brainer to have for anyone serious.

magnet:?xt=urn:btih:67177297E84766DFBF1C9EAAC6CF44B6F40BF3D1

I haven't checked it out myself but I've heard that some posts were removed from this magnet.

It's nice that usenet still exists today, although if not mainly for warez.

It's sad that it's now powered by only four usenet backbones.

If you go local, notmuch beat recoll for me on the mbox split, ~2M messages indexed in about 40 min on a cheap SSD.

Logistically, these collections usually exclude alt.binaries traffic entirely. I often wonder if there are enough obsessive collectors out there to piece together an archive from ancient tapes and hard drives.

The most prolific self-references (to myself that is) I found were UUCP maps updates because I ran a node which was in the centrally coordinated catalog (iirc it drove honey-danber to convert a!b!c paths to user@c so you didn't have to hand-route)

The second-most prolific self references were blovating across all and any groups I felt like. Nothing has changed.

> The second-most prolific self references were blovating across all and any groups I felt like. Nothing has changed.

I can take the moral high ground here because my bloviating was at least in relevant groups. But still, bloody hell, younger me was (mostly) an idiot.

I spent a bit of time haranguing Henry Spencer about why plywood was the perfect material to make satellites out of, in sci.space. I am not a materials scientist or a rocket scientist, and from memory he was the USENET admin at U Toronto department of Zoology, somehow uplifted to being the foremost space/rocketry name on USENET news. That pretty much sums up the zeitgeist.

Very cool. I've been working on something similar. My cutoff date is 1994 as Usenet changes substantially in nature then.

May I ask how you collected the data? Usenet is not fully archived anywhere. No one has a complete set.

I am aware of:

* the NewNews disc series on CD from '91 - '93 which has a substantial text feed for the big groups during that era (most of these are on IA.org, but some are not!)

* The Dejanews archives via Google up to c. 2013. Also up on IA.org These are lacking large amounts of early posts, and are highly spotty before about 1994 (when Dejanews started).

* The Utzoo tapes over at https://archive.org/details/utzoo-wiseman-usenet-archive cover a part of '81 - '91 but only from one host. "Far parts" of Usenet over in Japan or Europe are barely covered.

* Some groups preserve per-newsgroup histories scattered all over and tending to drop offline over the years

* fj.* is well-archived but it's Japanese

The chart speaks for itself I think as to coverage: https://i.imgur.com/WCQm4od.png

I got so desperate as to go scanning zip and tar files on old ftp sites for anything with a Usenet header. That scored a few hundred thousand articles mostly from the 90s.

The big spike in the early 90s is from those CDs archives in the months where I had access. It gives an idea for how limited those Giganews/Google and other surviving archives are.

I figure less than 50% early text Usenet survives, unless someone has vast hordes of data that they are not disclosing.

As for the Utzoo archives - they're legally encumbered and some of the posters sue anyone who hosts them. (They're not available from the IA - taken down.)

You have a takedown page but the legal and copyright issues were one of the main back-burner items keeping me from making my archive live. Some of the more eccentric Usenet personalities are still alive.

Same with privacy issues. I notice you have full text search. I was reluctant to do that for privacy reasons.

I discovered very early on how to index by email but that felt quite invasive. I took that out, and built a graph of all the X-refs: which turned out to be more interesting (walking the discussion trees) and somewhat more privacy preserving in that you can explore but not search.

I was also thinking of an AI-based content warning filter, because I have a "random post" button and it is sometimes wild what that lands on.

Reply on news.ycombinator.com