Google+ cannot be used with customer or brand accounts anymore
plus.google.com
plus.google.com
It was done with the distributed scraper 'Warrior' using a massive amount of small cloud instances to spread the load to 1000s of IP adresses. The dataset is ~1.45 PB.
It is your account of course, but i still dislike the practice since it goes counter to the entire purpose of a discussion site.
Only a small subset of users would be interested in those sites, and they likely don't come up in searches nearly as much as the original reddit post, so there's much more value to the deletion scripts than you are giving credit for.
In particular, they remove the forum requirement of quoting the comment you're replying to, by allowing you to respond to someone's comment directly below it in the middle of an existing thread. It would be exactly as inconvenient to readers of old Reddit and HN threads if that context was deleted automatically 6 months after the fact, HN just doesn't let you do it.
Actually, it does. I discuss things of significance here, where the information may be useful to someone in the future, but I only commented on reddit for entertainment purposes (jokes, witty remarks, etc.) Also, creepy people don't comb through past HN comments looking to call you out in current conversations like they frequently do on reddit, and almost always out of context and spun to fit their agenda. While entertaining, it started getting annoying after a while, so in my opinion, that's why they can't have nice things, so to speak.
What seems unimportant and insignificant to us now might become very interesting in the future. People in the future might consider the arrival of the internet as the beginning of a new age. Furthermore, some things may seem obvious to a person born in our times, but might not be obvious to people born in the future.
So many works, including even movies from the 20th century are lost now. E.g. the original version of Metropolis used during its premiere screening. Diogenes wasn't just a guy living in the town square, he also created some written works. They are lost now.
It would be polite if ArchiveTeam were to now contact everyone whose profile they have scraped, and ask for permission to retain that data. And then delete all the profiles for which they didn't get affirmative consent.
For two, being public and publicly accessible doesn't mean it isn't gauche to scrape it. It's kind of like how nobody sticks a sign that says "take one only" over the bowl of mints at a restaurant; it's assumed you just know that it's not cool to stick a whole handful of them into your pocket.
IA aren't scraping the data themselves, but they're the customer.
As for your second point: there's merit to that argument, and I've discussed same previously -- I'm very much of mixed minds on this. A few considerations weigh strongly, however.
0. The data were already public, as noted.
1. The system shutdown was not a known factor when most of the data were created. The expectation at that time was that the data would continue to exist.
2. The shutdown itself has occurred in a context in which individuals, and far more importantly groups quite literally could not archive the relevant data themselves. Google's own Data Takeout, whilst fairly remarkable (in a positive sense) within the industry makes many things difficult or impossible. Ordinary users cannot archive Community content, and even Owner and Moderator roles within communities could only archive posts from public communities -- neither comments nor private communities were archiveable. (Third-party tools could provide these capabilties). Moreover, technically, cost, bandwidth, or storage-constrained users or communities largely had no viable options for saving their own legacies.
3. The contents sitting on Google+, indexed and searchable by content on both the site and via the public Web, were far more visible than they will be at the Internet Archive, which does not support full-text search of its archives (at least not yet), and which is not as effectively indexed publicly as Google+ was.
4. The Internet Archive does provide for content removal under the DMCA, as well as other mechanisms. For a G+ user, given how content URLs are constructed (they all include the user's G+ UUID as a common element), requesting removal of an entire tree is trivial.
On balance, this favours the Archive.
Also, Archaeologists love the garbage dumps and cesspits of old towns, literally the places where people put their least valuable things, because they weren't raided by earlier visitors and they can derive so much info about it. And it's nicely stratified so it gives some rough chronology.
That's why you need to catalog stuff.
And if you have stored something, you can always get rid of it if you deem it to be unimportant, but if you haven't stored it, most times you can't get it back. Erring on the side of storing unimportant things is an important strategy to cope with that.
Often this argument is used to try to justify things as important as mass-surveillance and as small as logging in a software project. “Just collect and retain everything, who knows what we’ll actually need!” With GDPR and increasing focus on massive & intrusive data collection, I think this mindset is going to have to change. Before deciding to preserve or collect information (particularly information of a personal nature, like social media accounts), you should be prepared to justify the activity. “It might be useful one day, maybe” shouldn’t be good enough.
That being said, I intend to ask the Internet Archive to remove my personal G+ profile.
This is probably my favorite Google Plus post:
https://webcache.googleusercontent.com/search?q=cache:9FEWJo...
Personally i have dug a lot on old websites stored at the web archive trying to find information and files (especially patches, older software and amateur games that few knew about) seemingly lost. For me the Archive (not just the web archive but all the archive.org projects) is as important as Wikipedia (and donate to both).
Current platforms give an idea of permanence, so people stopped worrying about preserving what they care about, and when one closes both are lost.
People saving images solely to Flickr or videos on YouTube is a different usage pattern than the ephemeral likes and comments they make on the social platforms.
We don't necessarily know what's "important" in advance.
Archiving this content can be a nice public service for people who put content on the internet, don't do a good enough job preserving it themselves, and then want to access it later. I've heard "check the wayback machine" as a solution on HN for people whose blog got taken down unexpectedly from some provider and they didn't have a back up solution.
Perhaps the same as the first reason, but some of this content can turn out to be useful as evidence. It's not uncommon for public figures to take down unflattering messages they posted on social media platforms and then deny having ever made them.
It was actually when the Trump administration basically gutted and removed the whole of the EPA's website that caused me to take notice. I realized that while my written notes from a decade ago were still around and usable, every digital note I'd made more than a year or two ago was gone.
Our knowledge culture has changed from memorizing facts to knowing how to get to facts. Information also no longer flows down from a "chosen few" who have the means to publish, but instead from everyone at just about all times.
Archiving is also no longer a horrifically complicated or expensive hobby. There's no reprinting books on archive paper or storing them in helium. Instead you can simply buy a stack of hard drives and start bulk storing data.
Since getting involved, I've come to realize that data goes offline frequently, with no warning - extensively due to bad copyright claims. I can't really fix the copyright process (at least, not very quickly), but I can help save the data and help make it available to others in meaningful formats.
It gave me a little glimpse of what college was going to be like and helped me deal with being so isolated.
Google+ had millions of active users of a wide range of technical skills, but by far the majority at the lower end of the scale. Even the ones with technical chops were often limited by budget, network bandwidth, costs, or reliability, or other factors, in what they could do.
Google+'s promise was to host text, image, and video content with Google's deserved reputation for high reliablity. The company itself is not, as many other failed social and online media services were, going out of business. It's simply decided to exit this particular activity.
The first realisation I had of the problem Google+ Communities faced was when someone commented on the subreddit I'd created about the G+ shutdown that they were wondering how they were going to move 400,000 users and content to a new home.
I didn't even know how many communities G+ had. The online information I'd found said about 5 million, but on checking it was actually 7.9 million as of late November, 2018, and over 8.1 million by January 2019. Through Loysoft (Friends+Me), I got a summary dump of all 8.1 million communities and some overview characteristics (members, posts and posting dates), and could finally start getting a handle on how many significant communities there were. That resulted in an extract of about 100,000 communities of 100+ members and posting activity within the previous 30 days, which was used for outreach and migrations. I made that freely available to anyone looking to assist in migrations.
But a lot of communities were missed, Google itself shut down the G+ Communities serving Community owners and moderators, with no notice, making outreach all but impossible.
Is there a lot of crap data out there? Yes, there is.
But there are also some gems, and the task of sorting between the two in advance of archiving it is more effort than simply archiving everything and making it available. And the Internet Archive has set itself the mission of total archival, where possible. So there's that.
The questions are ones we're discussing though at the PlexodusReddit: https://old.reddit.com/r/plexodus
Sounds like you answered your own question.
When it's digital... if you don't keep feeding it, it's gone forever.
There has been some coverage of historians and archivists concerned about this, here's just the first random thing i found googling:
https://abcnews.go.com/Technology/digital-era-end-history/st...
But yes, professional archivists increasingly recognize you can't save _everything_ digital, we couldn't afford it. Figuring out what to save is a challenge. But if we save nothing, we're not gonna have much history to look at.
arkiver and Fusl in particular have been absolutely amazing in what they've accomplished.
They also managed to pull in 94.5% of all Google+ Communities, which should provide the ability to view posts by Community (they're otherwise scattered among user posts). We're still assessing how much of that was login-page redirects in the last hour or two of the crawl, but it's amazing work.
I'd managed to send of about 80k larger, recently-active (100+ members, <30 day activity) over the past few weeks, with a final grab about 18 hours ago, using the Internet Archive's "save" URL.
If you ever need to use that it's:
https://web.archive.org/save/<URL>
https://web.archive.org/save/https://news.ycombinator.com/item?id=19556665
Also: the Internet Archive (and Archive Team) run off volunteers and donations. You can help, and please do.
https://www.archiveteam.org/index.php?title=Donate
(Not affiliated, but very grateful to them.)
I'm really curious how such a large operation is legally pulled off, especially when services like Google+ intentionally try to make scraping difficult (and presumably for large offenders, will attempt a C&D).
https://www.archiveteam.org/index.php?title=Pomf.se#Archivin...
Is archive team going to respond if someone finds something in the archive they want to take down? Is there a listing somewhere of exactly what is archived (does it include pictures from google plus?)? I couldn't find it, and I don't want to go rooting around in the actual archive itself (if that is even possible).
I still feel the early variant of + with the original implementation of Circles (back in 2012) was one of the most useful takes on social sharing I've come across. And it worked especially well for photographers. Once they moved everybody to Collections I stopped using it with any regularity, and stuck mostly to groups/Communities. Hitching YouTube to it was basically the last straw. When I made a farewell post last month it had been about four years since my previous completely-Public post.
The platform had flaws, and I'm a bit sad to see it go, but I won't particularly miss it.
Cheers,
For all the ribbing G+ gets, the problem is a big one. And it's not one that's specific to Google+. As the regular parade of shut-down announcements of services and firms on HN attests, online mediated services can be cancelled, often quite abruptly. And there's often very little notice.
The world of social media sites is likely to go through more shakeups, for various reasons (and there are a number of sites presently looking pretty shaky), while the options for alternative provisioning of similar services (and the question of whether what we now call "social media" really is a net positive or something people want, need, or even should use) either personally or at a more local scale (though through what institutions isn't entirely clear) is a possibility. Projects such as IndieWeb, the POSSE initiative ("post on (your own) server, syndicate elsewhere"), federated protocols, IPFS, DAT, Beaker browser, and more (I've been discovering a lot in the past six months) may break us out of the current proprietary silo model.
Or not. The technical landscape is confusing, technical skills are limited, and the risks of DiY hosting can be large. It's a difficult trade-off. Though it's one I'd like to explore.
There are huge changes that have and will be happening on the regulatory front, from privacy to copyright to liability to propaganda and disinformation, and far more. Some of these laws and regulation seem written with self-service in mind, many do not. That's a whole 'nother field.
(I've got a To-Do item to get ahold of the EFF on these questions, as well as other groups.)
And then there's the whole fact that the tech world is in the midst of a (very well deserved IMO) backlash for its cavalier attitudes abuses and outright harm inflicted on both individuals and society as a whole. The promise of the 1990s has not been delivered.
Back to the group: we looked at the problem of migrating, realised there were many different users and groups, with different interests, and a wide range of technical abilities, from top-tier Linux kernel hackers (Alan Cox) to none at all. Some are best served by commercial solutions, for now, but many can look at federated or self-service options. We put together FAQs and Wikis and discussion forums and gathered a lot of data (we seem to have the best information outside Google on the actual size and scope of G+ users, data, and communities), and more. All inside six months.
It's been a group effort, and a lot of people contributed. I need to dig through my G+ archives to find the thank yous I'd posted earlier today, but it's substantial, and that was only a partial list.
What I hope is that others can use and be helped by what we've done.
The wiki is https://social.antefriguserat.de and there's a subreddit at https://old.reddit.com/r/plexodus Both will continue to be active over coming months, we're only part-way through the process, and still need to establish ourselves in our new spaces.