Verizon/Yahoo Blocking Attempts to Archive Yahoo Groups – Deletion: Dec. 14

Wow, somebody invented a database that's even worse than an Excel file on a network share.

(Also, how are they going to assign new numbers when archive.org takes over? Is archive.org going to give them write access?)

-EJ

> Your email address is not linked to a Yahoo ID. To join this group, you need to link your email address to a Yahoo account.

When I click "link your email address", it just takes me to a page called "Personal info" which doesn't have any obvious way to link my email address.

So I'm not sure how to proceed.

EDIT: Solved it. I had initially only "verified" the account with a phone number, but you have to add an email address as well. It's now working.

For anyone who, like me, signed up for this and filled in the Google form, but then couldn't find the leaderboard URL after closing the tab, it is https://df58.host.cs.st-andrews.ac.uk/yahoogroups/leaderboar...

It seems to be working through a list in reverse alphabetical order. Watching the progress being made is quite satisfying. When I started it was on groups like "sciencefiction" and now it's moved on to "petzluverz".

I assumed I could help by going to a web page and solving a bunch of captchas for you, but when I read those instructions I found there's more involved (forging a Yahoo account, installing an extension) and it turned me off.

If captcha's are the bottleneck, maybe some generous soul here could figure out a way to automate the rest and just give me a page I can go solve captchas? Further reducing the friction might help get you some more uptick from the community - more monkeys like me banging at typewriters.

Sorry I wasn't more help, and best of luck with your efforts.

You might even get the admins to make an announcement.

`Buster is a browser extension which helps you to solve difficult captchas by completing reCAPTCHA audio challenges using speech recognition. Challenges are solved by clicking on the extension button at the bottom of the reCAPTCHA widget.`

I confess I'm doing this mostly to see what people posted on the internet at some point in time :)

Edit: All groups have around 1600 members... what causes this...

I work for VzM, but not historically directly on Yahoo products (product teams have been merged/consolidated etc. over the past few years, but there's still strong tendencies toward products people came from).

So I wouldn't be very clued into what's happening with Yahoo Groups internally. And I've heard nothing about this internally. At all.

As it stands, it's 2:30pm in SV, VzM is top of the HN frontpage, and not a single soul has mentioned it yet on internal Slack.

Will see if I can find out more.

So technically, some legal troll could post some copyrighted information, wait for it to be published on Archive, and then sue Archive for copyright infringement and Verizon for assisting it. As a non-profit, Archive will likely get away with just taking it down, but a for-profit Verizon is a wholly different story.

A main concern of the Archive Group (again, below) is art that was uploaded there.

I'm sure those are not the only two classes of examples. See for example the bird watching group in Delhi that has been collecting data for decades. (In the link of the OP.)

Step 2: It will take a long time to look through all this content and determine which parts deserve keeping.

Step 3: We will inevitably leave out something that someone else thinks is worth keeping anyway.

Step 4: Let's just archive everything.

It's also not horrendously expensive - we are getting better and better at storage as well data analysis techniques, so stuff that seems useless today may be useful 50 years from now and cost less to store than it does now. The key thing again being that we can't benefit from hindsight.

Even graffiti can give insight into a time period, even if that insight is that that time period had an unusually high number of graffiti artists.

People doing important work (esp important work that is underfunded) don't have time to write/record their own histories. But that history can be instructive, to learn what worked and what didn't, and help future travellers do it better :)

And perhaps especially important: ppl engaging in these under-resourced efforts are often working in domains that capitalism is... less curious about, we'll just say. Otherwise, it would likely be able to be more highly documented, as incentive is there to preserve it.

Our ability to improve our present from better understanding our past is a supposed benefit of a digital world that accrues data -- we have records of things that in prior ages just flew by in conversation (for better or for worse). But efforts like this rob us all of that wisdom <3

And again, there is an asymmetry in who gets robbed. It is often the folks working in the commons, those doing invisible maintenance labour (nonprofits, grassroots, community), and generally just people doing work within the cracks of capitalism.

People are conflating internet discussion content with written content because it's stored as text. Whereas the more legitimate comparison is to verbal communication.

Ask an antiquarian about the value of graffiti in the ruins of Pompeii and other archaeological sites sometime. The great historians of the day wrote about their contemporary culture, while the vandals and miscreants and lowlifes and commoners contributed to that culture. Having access to both sources gives us a much more complete picture.

You don't know what's worth saving at the time you save it.

It's easier to just save it all and let gawd sort it out.

You never know what some future person might find interesting. For example, my father took lots and lots of pictures, but they're all set in the living room and kitchen. No pictures of the rest of the house. I'm sure the thought of photographing other rooms simply never occurred to him as being interesting.

For another example, many people are interested in where/when/why certain words first appeared, like the origin of "OK". Massive archives of text that are searchable would help with this.

I write a lot of historical content and often the most useful stuff I find—for example, old flyers or ads from the 1950s or 1960s—would have been considered trash by someone at the time.

So an archivist’s job isn’t to make a judgment. It’s to protect the data as they see fit.

The graffiti on the toilet wall may well speak to the start of a trend, term, movement, or other event, for example.

Think longer timelines, broader scope than you personally may feel is relevant.

En mass, those questions have answers we individually are unlikely to fathom.

... that had access to writing services and were wealthy enough to have their thoughts stored.

There could have been many odd voices out there that would've told us an entire different story. But these are unknown because they didn't have access.

Now we are in the era of (almost) universal access to storing our thoughts and we still don't listen to the everyone or mark them as uninteresting and not worthy.

But most importantly, Groups is a corpus representing many segments of society during a period (starting 2001, with a peak of over 100 million users in 2008). It's a snapshot that embodies concerns, beliefs, morals, language... at several realms. This is more than LoA even. It can be used profusely by researchers and historians to study society for years to come. Or by AI to learn how and who we are/were...

I am in awe of your flair for understatement.

We have that kind of graffiti from Pompeji. It's enormously more fascinating and insightful into regular people's lives than all the stuff about kings and battles people wrote about in the more official works.

When looking through all newspapers and magazines, the advertisements are often the most interesting bit. Especially since you can probably already read about the big events they wrote about on Wikipedia or history books.

what a lovely thought. Thanks for the effort, even tho it didnt pan out. if you've got the dvd torrent it out :)

now im wondering if there's a stratus emulator anywhere and/or the os code. Them things were nasty... individually battery backed hard drives was just the beginning. The slot cards looked like someone had dumped yellow patchwire spaghetti all over them.

The default corporate posture will be : Delete all the data! It's a liability and figuring out what we can keep is an enormous headache.

- https://www.firstthings.com/web-exclusives/2010/06/the-perni...

- https://www.ancientworldmagazine.com/articles/making-myth-li...

- https://history.stackexchange.com/questions/677/what-knowled...

But anyway, no one should delete human littérature, be it inadvertently or by lack of effort.

IIRC, Archive.org is still running its fundraiser today.

We need LOTS of publicly-sponsored and paid-for digital archival centers that, like libraries, are maintained for the common welfare. Or we could, you know, add that duty (and funding) to existing libraries! With -paid- archivists!

I mean, the GDPR makes things kind of difficult in this regard, and I suspect even archives are liable if somebody takes an issue with content they are hosting.

Large corporations are not anthropomorphic entities, regardless of their disarming branding. Rather they are amoral bureaucracies, likely administered by people who have learned to ignore their empathy to get there. Verizon won't change course to accommodate the Internet Archive or general Internet community any more than a combine would pause for a field mouse.

My bet would be that Verizon's network monitoring system/team sees the archive team's attempts as some sort of anomaly to be stopped. It's possible, though I wouldn't bet on it given Verizon's history re: public relations, that making noise might alter the equation and get them to allow the archive team to continue.

To raise the perceived threat level, many folks could support in building tooling or docs to help ppl migrate as easily and streamlined as possible, to minimize the tax on consumer time that they rely on. (E.g., help on comparable plans, cheat sheet for call centre keywords, etc.)

Maybe something team "Do Not Pay" could help run with...! [1]

[1]: https://boingboing.net/2019/10/28/parking-tickets-plus-plus....

It's not like interacting with political representatives or corporate PR/executive types where you're conveying the size of the interested party, in this case newsworthiness doesn't necessarily depend on how many people are sending the email.

The Archive Team has been taking requests for backups of groups for people who don't have the technical facility to run the python scripts. They then intend to make them available on the internet archive. The next project is making some kind of front end, in case group owners want to host that somewhere. Some of us, for example, will be doing that behind some kind of a forum login, so it won't be search engine indexed.

As for your point 2, that was cut/pasted from the link in the OP, where it's describing that many groups are still using the platform. More relevant to this project, is that many groups are losing their archives, and those archives contain anything from scientific data, to hobbyist & howto information, to art and literature, etc.

2. d. viii: "interfere with or disrupt the Services or servers, systems or networks connected to the Services in any way."

I'd also like to point out that the apparent spokesperson Brenda Fowler said in her open letter to Verizon, that "If the problem is that all our attempts to rescue our archives in the time we have left is causing an overload or strain on your servers, then stop making us HAVE to work around the clock, and GIVE US MORE TIME. ..." Probably not the wisest thing to say right now.

Also, archiving the groups with automated tools is against the Use of Services rule, that states the following:

2. e: "Use of Services. You must follow any guidelines or policies associated with the Services. You must not misuse or interfere with the Services or try to access them using a method other than the interface and the instructions that we provide. ..."

As I mentioned in another comment, I really support the cause and am a big fan of archiving myself but it's unfortunately quite clear that Verizon is right at calling out the violations of "terms of service".

New Reddit(without the old.reddit.com interface) for example. Many niche subreddits contain lots of information that would be lost if reddit dies(or just deletes these subreddits).

Youtube is unarchivable in principle due high amount of storage required(even thinking of 640x480) and yet it still contains tons of unique content found nowhere else from rare AMVs(that survived prior deletions) to instructions to repair telescopes - or basically anything in video form that doesn't have backups(i.e.not uploaded to other videos sites).

4chan and similar sites are archived by several sites in haphazard manner(only boards they like) and yet it a huge chunk of internet culture that is going to be lost if these sites die(and its more probable than Reddit due less funding). Usenet is slowly fading into obscurity and dependence on Google Groups. Many forums that today exist, will not exist forever: yet very few are archived anywhere else. Other forum-like sites like Stackoverflow and Quora might disappear in the future with nothing replacing them. Github is subject to Microsoft whims and positions on open-source. Wikipedia and various wiki farm sites don't have much revenue streams. Practically every major website we take for granted is vulnerable - people thought Yahoo Groups was going to last forever.

Things shouldn't be like this. The price per unit of storage and bandwidth falls fast (and, except for the sites dealing with user-generated videos, faster than the amount and size of content grows). Laws shouldn't apply retroactively.

The problem really is that our means of accessing information are services. When you have a physical letter, or an e-mail saved locally, or a text message from 15 years ago, you can just read them. Nobody will know or care. Nobody will come after you trying to apply GDPR or DMCA retroactively. And since storage is near-free, you won't ever lose it until you forget about it (or at least about doing regular backups). Whereas with modern webmail, forums, link aggregators, IMs - you don't have even your own messages, and viewing a conversation that happened 15 years ago is really being provided a service today. Services are ephemeral, they're also subject to ever-changing regulations and whims of the service providers.

Bottom line, while services are necessary for transferring conversations, we really shouldn't be relying on them for access to conversations that already happened.

But giving a "export all the data in xml/json/whatever" button, and maybe even opensourcing the now-abandoned component serving this data, would be nice move. The first part could even become a regulative requirement some day.

AT and the Internet Archive have succeeded in preserving other content, though not all projects are successful. You can see a partial listing at https://www.archiveteam.org/

Even as notorious a "wasteland" as Google+ (a naming I've had some role in establishing: https://ello.co/dredmorbius/post/naya9wqdemiovuvwvoyquq) had many millions of actual active users, and tens of thousands of active communities (https://social.antefriguserat.de/index.php/Migrating_Google%...).

Unlike numerous other shutdowns, Google announced the G+ shutdown well in advance, though they "accelerated" the schedule twice, from "sometime in August 2019" to April 1, 2019, the eventual shutdown date. The tools Google offered for archiving and migrating content, whilst among the best in the industry (an exceptionally low bar), were incredibly insufficient: buggy, incomplete, duplicative, and not readily portable). It was largely third-party tools and assistance -- the Friends+Me Google+ archiver and ArchiveTeam most especially -- that meaningful preservation was possible.

The conceit of large-scale, free-to-use services has been convenience, capability, and trust, the last a point Google explicitly made in its original G+ announcement:

You and over a billion others trust Google, and we don’t take this lightly. In fact we’ve focused on the user for over a decade: liberating data, working for an open Internet, and respecting people’s freedom to be who they want to be. We realize, however, that Google+ is a different kind of project, requiring a different kind of focus—on you. That’s why we’re giving you more ways to stay private or go public; more meaningful choices around your friends and your data....

https://googleblog.blogspot.com/2011/06/introducing-google-p...

That trust has been repeatedly violated.

And in actively opposing archival efforts, Google, Yahoo, Flikr, and others, are violating that trust only so much the more.

In the G+ shutdown, it was the active dismissal, obstruction, and interference of Google and its user-based support team (the so-called "Top Contributors") which were most disappointing. Long-time Google supporter Loren Weinstein made this point specifically and repeatedly:

https://lauren.vortex.com/2019/01/29/googles-g-user-trust-be...

I'll note that this tends to strongly reduce the value proposition of all Web 2.0 / SaaS offerings, given that even the very largest and wealthiest companies are willing to act in this manner.

The consistency of this behaviour and attitude across multiple service providers makes me think that the behaviour and practices are not coincidental or unintentional.

The mean post size on G+ was rougly the same as on Twitter: about 120 characters. (Quite possibly because most G+ posts were themselves repurposed Twitter content.)

Static content does not require ongoing moderation, though it's possible that problematic content will be periodically identified.

The bigger challenge is actually in the publishing engines. Even where these are static, it's possible that vulnerabilities will be identified. That was Google's (not especially convincing) excuse.

A challenge of the Internet Archive / Archive Team method of archival and access is that in preserving the original formatting and packaging of content, the bandwidth and storage requirements are increased tremendously. By about two orders of magnitude in the case of G+.

Were the Archive to focus on the actual originally-authored content rather than all the associated chrome, both factors would be tremendously reduced.

Very possibly the timing isn’t a coincidence, being CCPA is about to take effect.

And we're only archiving things that "any guy on the internet" can see. If someone can access the messages simply by joining a group (with no moderator approval), I'd argue it's fair game.

We're not going to be unreasonable, though. If something private slips through and we receive a takedown request from the author, we typically remove it.

If they have access to it, it is not private.

[0] https://modsandmembersblog.wordpress.com/2019/12/08/verizon-...

I admit to legal ignorance, but this does seem over the top.

Also, the scripts really have been developing during this last 2 months, which is all the time they gave us when they said they'd be deleting all content. Until then, I didn't see anything that would get me all content including files/links/calendars/databases.

Additionally, the 'Get My Data' only gives you access to all the files / photos that you uploaded to the group. These archives should not be considered complete archives of the group - is it really reasonable to suggest that in order to completely back up a group, every member must complete their own data request?

Also, see betamaxthetape, above. If anything is archived, they will respond to takedown requests.

Because some might want to read them or use them in some form.

edit: Oops, I'm also a member of LTspice. D'oh!

The best way to stop being ashamed of stupid things that you said forever ago is not to cast those things into the Memory Hole, but to stop saying those things, and most importantly, stop being the person who would. Then you know it's in the past, and it doesn't matter who else remembers.

I'll let you know how that goes, someday ;)

When the G+ shutdown was announced (see earlier comment, I was one of the "Plexodus" organisers), the whole question of what to do about Communities migration hadn't even occurred to me. It wasn't until a community moderator happened to mention that they had a group with 400,000 subscribers and no idea at all to even begin that I started looking into this.

The first result was to create at least a framework for organising and planning: https://social.antefriguserat.de/index.php/Migrating_Google%...

It's a really basic thing. I've found no evidence that any such guide has ever existed online. I make no claims that what I came up with was the best, or even effective. It does seem to have been the first. Win, me.

We created a "short FAQ", formatted for posting to G+, which we posted again and again and again and again and again, to as many communities and spaces as we could. Because when you're trying to reach a mass audience (10ks to millions of users, 10ks to 8.1 million communities), mass repetition is necessary.

https://social.antefriguserat.de/index.php/G%2BMM_Short_FAQ

It's hard enough to motivate people to turn up at one place once every couple of years to mark a few spots on paper, when they all know that this happens regularly -- election turnouts tend around 30-40%. Getting mass compliance on a complicated, unexpected, and technical process (with social and legal entanglements) is far harder.

Many groups glide by on minimum maintenance for years. Whatever initial technical competence was required to set up the group be long-lost. At best, moderation skills are known (and even that is a long stretch). Migration is an entirely different skillset, one rarely exercised, and, as noted above, extraordinarily poorly supported.

TL;DR: your expectations are entirely unreasonable.

This is the corporate equivalent of throwing your old computer into an empty ditch on the side of the road, and getting mad when someone responsible comes by to recycle it for you.