Why the Library of Congress is wrong to archive Twitter | Hacker News Reader

Why the Library of Congress is wrong to archive Twitter

scripting.com

10 pointsmgunes31 comments

LoC archives Twitter because only LoC can archive Twitter. Other projects, notably the Internet Archive, already archive open web sites including blogs. But IA doesn't have a hope of archiving Twitter. LoC arranged this because if they didn't, no one would have a Twitter archive.

I think LoC has partnered with the Internet Archive in the past, but now they're planning a much more ambitious version at http://www.netpreserve.org/about/index.php

LoC is actually relatively forward-thinking when it comes to digital archiving. For example, for a while now they've been encouraging people to digitize archives of their own personal data: http://www.digitalpreservation.gov/personalarchiving/

You could also argue that as a government agency the LoC should prioritize important archiving tasks that aren’t already being done by the private/non-profit sector. Despite being a corporate controlled medium, Twitter is still an important communication medium and should be archived. For whatever reason, others aren’t doing it, so it’s an obvious thing for LoC to do.

Who says the Library of Congress isn't archiving the open web?

http://www.loc.gov/webarchiving/faq.html

Specifically, it states:

The types of sites archived include, but are not limited to: United States government (federal, state, district, local), foreign government, candidates for political office, political commentary, political parties, media, religious organizations, support groups, tributes and memorials, advocacy groups, educational and research institutions, creative expressions (cartoons, poetry, etc.), and blogs.

Did you miss the note?

http://scripting.com/stories/2012/07/17/whyTheLocArchivingTw...

"(Note: I mean bloggers. The equivalent content that's on Twitter, but not in a corporate blogging silo.)"

Did you miss the last word of his quote?

No I didn't. But I also noted it was the last word. :-)

LoC is one of the organizing members of this: http://www.netpreserve.org/about/index.php

The mission of the International Internet Preservation Consortium (IIPC) is to acquire, preserve and make accessible knowledge and information from the Internet for future generations everywhere, promoting global exchange and international relations.

They've got a working group working on smart crawlers right now. If you've got some good ideas for making it easier to archive blogs, perhaps you should consider contributing.

So your complaint is not that they don't archive blogs, but that they don't list it first among that which they archive?

I wrote a blog post that explains my point of view, it's linked to at the beginning of this thread.

And as esonderegger pointed out, your second point is incorrect. I also think your second point is the crux of your points; that is, everything you said assumes that the Library of Congress favors Twitter over blogs. How does that change your point of view?

Okay you're making an editorial/legal objection, you'd like me to find a way to merge the note in with the first part. Consider it done.

We need a new acronym to go with IANAL, something like TIJABP -- This Is Just A Blog Post.

http://scripting.com/stories/2012/07/18/tijabp.html

Considering how important you think blogging is, I'm surprised you don't feel the same about the quality of the content.

All of his arguments boil down to "open web good, twitter bad, ergo LoC shouldn't archive twitter".

With the slight correction that it's "ergo LoC should probably try to archive the web before it tries to archive twitter". But yeah, that's the gist of it. But so what? That doesn't mean it's an invalid argument. In fact, I'm inclined to agree with it.

If they had a good process for archiving blogs on the open web, then I wouldn't have a problem with extending that to Twitter.

Disingenuous.

He said "twitter private, Library of Congress public. Public preference to one of several private corps not in public interest, especially when there is a larger, more archive-worthy corpus that doesn't require private APIs or vendor discrimination".

Which part of that do you disagree with?

I would assume they would want to do both. The advantage to using Twitter is it copy's a wider range of conversation AND it's easy. Set it up and add some server-space and be done with it, where to even start on blogs would require 10x the effort and even more space.

GP is disparaging, but doesn't state any disagreement, his argument boils down to huff puff mumble mumble, get off my lawn.

I think the advantage Twitter gets from being archived is greatly overestimated here. How many people actually consider that when they decide to sign up for Twitter? I'm also skeptical that Twitter is an alternative to a traditional blog. The 140 character limit doesn't really let you replace a blog if that's what you want to do. I think most people who want to blog would have a blog in addition to Twitter. Likewise, people on Twitter who don't have a blog, probably just wouldn't publish anything at all in the absence of Twitter, so it's a net gain in that respect.

Finally, Twitter does have a corporate API, isn't most of it accessible through the open web any way? Any user's profile can be accessed through the website unless they've specifically restricted it.

And generally, I don't see the point in NOT archiving something if you can. Sure it's fine to argue that other things should be archived first, but if someone is going to archive something, why stop them? We only gain by having a more complete and accurate historical record.

Tweets can detail events of local, national and global importance which may only have value in hindsight. To risk losing a stream of tweets regarding a milestone event because you don't like Twitter's API policy seems quite small sighted.

99.9999% of what comes through twitter is wasted bits. I still see value in archiving it all to guarantee capturing a real time log of an historic election, the death of a notable person, the announcement of a major discovery, or the day to day thoughts during a protest or revolution.

We have 19th century diaries detailing a rector's daily meals for his entire adult life while we have recovered nearly nothing regarding the poor living in slums in London, except through the distant scornful commentary of the gentry.

The LoC shouldn't be endorsing Twitter as a company, but the users of Twitter have endorsed it by logging the course of human events.

I posted a follow-up to this piece.

http://scripting.com/stories/2012/07/20/moreOnLocAndTwitter....

Dave

I always thought this was just a publicity stunt orchestrated by Twitter.

And the NSA is archiving Twitter and everything else.

I don't really get it. LoC also archives commercial content that was, at some point, closed off to non-payers.

As for why Twitter and not your average blogger...well...because there's a specific endpoint to get all the tweets, each of which is a discrete, well-defined object. Finding all the blogs on the Internet is likely not as easy.

Core assumption is there is some business advantage in having LoC archive. Not sure I agree.

Much more fun is to use an analogy. LoC will arbitrarily by their own high level decision only and exclusively archive ABC TV shows, NBC, CBS, etc, too bad. Or perhaps if they only archived Ford corporate materials and ignore GM and all others.

Well there certainly is a business advantage. Or why would Twitter have done it, and done a big rollout of the deal (I believe it was at SXSW). It was a huge publicity bonanza, the message was that Twitter is Historic. Don't believe us, believe the Library of Congress. They didn't do it out of altruism, they did it because it was excellent publicity. (Probably the LOC did it for that reason too.)

And we have a say in this in the US, since we pay for the LOC.

Well there certainly is a business advantage. Or why would Twitter have done it, and done a big rollout of the deal

You cannot assume companies always act logically. In this case they could have agreed to it just because it was "cool".

They didn't do it out of altruism, they did it because it was excellent publicity.

This is probably accurate, but it doesn't give them a long term advantage. It probably gave them a small boost in marketing (I say small because I have never heard about this). Marketing doesn't hold up without investment. (for those of you who say iconic brands will hold up, keep in mind that every iconic brand advertises constantly.)

Further, the government arbitrarily "subsidizes" businesses constantly. In many cases much more so than Twitter. I agree that any subsidy (whether monetary or otherwise) should be examined, but in this case it seems to be a complete nonissue to me.

>>the government arbitrarily "subsidizes" businesses constantly

And people object to it constantly.

Sure, it's great PR. I'd love to be able to put out a press release with my name and LoC on it.

But you really think it is affecting users' actions now months later? I'd be surprised if anyone considers LoC archiving a feature that keeps them locked into Twitter instead of starting a blog. (Notwithstanding LoC's efforts to archive all sorts of digital content... you make it sound like Twitter is an outlier.)

I don't know, to me it says that Twitter is small. How big is it in Libraries of Congress? Less than one!

Reply on news.ycombinator.com