Hacker News Reader

Comment by Shank | Hacker News Reader

I think more broadly, a problem that needs to be solved for journalists is archival rights or something for their work. In the past, you could just buy newspapers or keep copies of your works after they were published. You can still do that now, but it's not as important until it goes down. It would be better to negotiate into contracts something like archival rights to writing, or at least a permanent credit/record.

Journalism is unique in that it's almost always public in some form. It should be a reasonable expectation that it stays in its original medium, or is accessible in archives if that medium vanishes. Major newspapers often offer reprints or back issues. NYT offers the "Times Machine" [0] with basically everything they've ever run digitally. This should be the standard, not the exception.

[0]: https://timesmachine.nytimes.com/browser

View on HN

"Major newspaper often offer reprints or back issues."

Perhaps younger readers have never experienced looking at microfiche of old newspapers in a library. Analog machines, not computers, but, if memory serves me correctly, the archives were far more complete than the so-called "tech" company-mediated, advertising-based nonsense what we have today on the www. And the analog fidelity was great.

Google ultimately failed to "organise the world's information" to the extent that it previously was organised by public libraries. Instead they turned finding public information into a game designed to support a massive programmatic advertising racket. The www has more volume perhaps than pre-internet libraries but 1. an enormous amount of it is garbage and 2. it is not organised for systematic browsing (leading to discovery) nor serious, methodical research. It is optimised for "clicks" and "views", selling ad services, not learning. There is no way to "browse the stacks" to see how this information is organised. It's all "secret". Imagine a public library that had to hide its operations as a "trade secret". Imagine that it tried to intermedite which stacks a patron will browse, for commercial purposes.

The point is that public libraries had extensive collections of newpapers and periodicals. The public could do serious research. But the www is dominated by advertisers and most of the information published is influenced by commercial motives. And it's all intermediated by a gatekeeper that sells ad services: Google.

If I want to use the internet (cf. www) for library research I still somtimes search library catalogs via z39.50 if the library's website is cludgy.

Volume of data published historically is pretty small compared to today, especially with the web.

they mentioned that in their comment

Looks like it has expanded significantly from what I recall replying to

It’s wild to think in the information era we may actually have an information black hole of this time in 50 years.

Man, I miss the original Google mission.

It's simply a call to support the Internet Archive more robustly, and ensure data stored is durable globally.

Who would you trust more? Big Tech and Silicon Valley? Or a non profit with a quirky benevolent dictator whose mission is universal access to all knowledge?

The Internet Archive needs all the support we can give so that when hack writers who have Reddit personality disorder blame them for their books not selling during the pandemic sue that the Internet Archive legal team can vaporize said hack from orbit.

Then you have hack crybully journalists using their uncle to delete stuff about them they don't like from the Internet Archive. No thanks.

......no

The entire point is to make the Internet Archive have a legal team that can tell crybully journalists and their uncles to eat a fat dick

My honest answer is Big Tech. I’ve known far too many benevolent dictators of non-profits.

Better them than the advertisers, which "Big Tech" ultimately is; it's hard to have more compromised morals than that.

Those microfiche archives were amazing, and you can still today find things in them that do not exist anywhere else in the world. I'm not even talking articles that had errors when digitized or were edited/removed years later, but things like advertisements and all the other stuff that was in a newspaper that "wasn't the news".

The UK has something like this: it's called "legal deposit". Publishers of books and periodicals are required to send copies to the enumerated deposit libraries (one of which is in Ireland despite no longer being part of the UK).

Laws were passed to update this to the digital age, allowing the deposit libraries to archive UK websites. Unfortunately they don't right now archive video. It's also a bit limited given that web publishing is not national, so much UK content is published on US run websites.

See https://www.bl.uk/legal-deposit-web-archiving

Canada does something similar - you can register to become a publisher, which then allows you to issue ISBN numbers for your works (eBooks count), with one stipulation - you have to send a copy of anything published to Archives Canada:

https://www.bac-lac.gc.ca/eng/services/isbn-canada/Pages/cre...

Also MTV News was archived, until today that is. How long until the new CEO of NYT will decide the costs of archiving are not bringing enough shareholder value? From that moment on there won't be a source for reprints either. Am I too negative?

I was under the impression that (public or state) libraries keep their own archives of many publications to prepare for this case.

If the NYT archives only exist at the whim of the CEO, library archives only exist at the whim of local governments.

The logical decision then is to entrust the preservation to once of the few organizations capable of surviving thousands of years, and forming a religion based on the preservation of works.

OK Leibowitz...

By that logic the only archive you can trust is the one you build yourself. Not really feasible.

It's hard to claim to be a "paper of record" if you don't have those records.

Gawker and numerous shuttered local newspapers (the former Santa Barbara News Press comes to mind: <https://doc.searls.com/2024/04/21/archives-as-commons/>) have also seen this occur. The challenge for former reporters and staffers who lose any ready access to their portfolio is profound.

If you're your own rightsholder, obviously you can do whatever you want. The NYT archive is one such example.

In the more common case where you are not, though, the law already provides exceptions to copyright for archiving and redistributing materials that are no longer available. This generally only applies to content that isn't audio/video/graphical, but a wider exception is provided specifically for "news" (journalism).

If you want to negotiate a contract making such archival exceptions explicit within the contract itself, that's probably not a bad idea. Even if not though, the law provides for archival protections already.

Further reading: https://www.law.cornell.edu/uscode/text/17/108

NYT offers the "Times Machine" [0] with basically everything they've ever run digitally.

Not just digitally. Times Machine has everything going all the way back to the first issue of the New York Times on September 18, 1851: https://timesmachine.nytimes.com/timesmachine/1851/09/18/iss...

That's like reading this HN thread in the year 2197.

> You can still do that now, but it's not as important

I think you hit the nail here.

Most contemporary journalism is just not worth archiving. There is a small percentage that definitely is worth archiving, but the bulk of it is produced for the moment.

Who are we to decide? Many television tapes were wiped or lost last century because people didn't think them important, including a big chunk of early Doctor Who and a lot of the Apollo moon landing coverage.

> a lot of the Apollo moon landing coverage.

Even NASA (accidentally?) recycled the master tapes containing the moon landing AFAIK, and some of them are ultimately recovered.

Shortsightedness, I may say.

> Most contemporary journalism is just not worth archiving.

I don't agree. There are many interesting tidbits which go unnoticed as bog-standard / disposable news until the dots connect in the future, and one wants to construct the history backwards from that tipping point.

When these "contemporary articles" go missing, many important details of the history is lost.

The challenge is that it's hard to know.

One of the people who recovered a lot of old Usenet archives once remarked that the reality was that no one cares about the details of some long ago SunOS bug whereas a lot of the cultural discussions about government policies etc. provide a window into the time.

I’d challenge you to find one article where this is the case.

Almost all “journalism” today is opinion notifications without substance, source, or first hand accounts. I suppose that’s its own kind of historical artifact, but not the kind I’ve seen be meaningful in my study of history.

Who knows, maybe in the future one of those opinion pieces may be the first digital record about a specific topic found by some future civilization. In the same way that the Complaint Tablet to Ea-Nāṣir doesn't provide much historical value, but is interesting in being the oldest written customer complaint.

History isn't just about the big things. The history of the mundane is also important in understanding a culture.

Are you really interested in reading a 50 year old newspaper article about a collision that happened midday on Saturday afternoon in your community newspaper?

Of course not. BUT on the other hand that particular collision may become highly relevant in those 50 years. Imagine your own scenario where the facts reported on the day might conflict with something claimed to be true much later, and the person wasn’t notable at the time but they’re highly notable now.

The issue is we don’t know what may be important after many years, decades or centuries have passed. Given how easily text compresses, it would be a shame to not have at least text of real news sources archived perpetually. I know there is also an endless stream of listicles which are just generated to get clicks and ad views, but newspapers are worth archiving.

> Imagine your own scenario where the facts reported on the day might conflict with something claimed to be true much later, and the person wasn’t notable at the time but they’re highly notable now.

I do see your point. I'm reminded of the photograph who snapped basically a digital throw away picture of Monica Lewinsky weeks/months/years before anyone really knew who she was. later on, he was happy to have that picture, since it was one (or the only one) of her at some event hugging Clinton.

Text is easy enough to store, but making it useful to search and access seems like another problem to solve.

Storage is also massively cheaper than in the past. It's worth archiving everything that was published on the Internet because you never know what it might reveal in a future context.

Maybe the comment thread on a trivial clickbait article contains a post by a future dictator, and it will be a crucial piece of her biography fifty years from now.

> but the bulk of it is produced for the moment.

But to a historian it is all a series of moments. To understand how/why one happened, it is useful to know the ones that came before, and things created for the moment might better describe it than those with a wider purview.

Of course you don't need it all, but deciding what to keep is a complex (and bias ridden, accidental or otherwise) problem in its own right.

>Most contemporary journalism is just not worth archiving

Considering how much historians love ancient rubbish bins, one mans junk is another mans treasure. Even the junk produced during and before the Trump/Brexit 2016 election/referendum would be massively valuable to a future historian in explaining how those things happened.

The text of every article is easily worth archiving. If we take average article size to be 500-800 words, then 2KB each is plenty. You want to hang on to 10 billion articles? That will cost you 1 (ONE) hard drive. Remember to make backups.

If we allocate 200KB per article for pretty strongly compressed images, then it's still 100 million on a single hard drive. Why not archive the whole lot and let God or future historians sort it out?

You only need to start getting picky when video is involved, but on the other hand when the alternative is total obliteration you can crush video down to 1MB per minute and have a tolerable VHS-like experience. And even 8MB Shrek gets the point across.

> when the alternative is total obliteration you can crush video down to 1MB per minute and have a tolerable VHS-like experience

That's still pushing a terabyte a day[0] on YouTube alone. What about TikTok as well?

[0] https://www.statista.com/statistics/259477/hours-of-video-up...

So, a single laptop hard drive a day? 50 tape drives a year? For the biggest repository of video there is? That's literally peanuts.

There's some logistics in handling backup tapes, but that's a well understood process.

I think you undershot that by a factor of 60. But when I said "get picky" I meant things like not archiving all of youtube. And for articles specifically, archiving every video article is a lot smaller.

https://routenote.com/blog/wp-content/uploads/2020/08/pex-yo...

And going by this, restricting to a reasonable view count will cut the space you need by a factor of 10 to 25.

If we round that to a petabyte per year, it's not in the range of a typical personal archive, but it's a reasonable thing to picture several big libraries doing.

(Though a single person could handle that much if they really wanted to, spending $10 a day on data tapes.)

In a way it was archived, into the minds and training data of AI models.

This may in fact be one of the most effective forms of archival available to us. (Handwaving away the preservation of compatibility with future hardware or the maintenance of an “emulation chain” to get us back from whatever GPUs are there in 2124 to something our today models would run on)

We could train a model every year and preserve it, then future historians can quiz this model that thinks it’s 2024 and ask it whatever they need to know. It’s fascinating because it will probably “know” the kinds of everyday normal people things that are very hard to glean from only reading old news stories. Things like how the average person feels about their world, or how they feel about current events and why.

Most contemporary journalism isn't even worth reading, let alone archiving.

It's published at virtually no cost compared to what newspapers and magazines used to cost. And with LLMs it's going to get even cheaper as you no longer need people to write the stories. We are spiraling into an era where we will be drowned in content that is all worth next to nothing.

> Most contemporary journalism is just not worth archiving.

It is impossible to know presently, how valuable something will be to someone in the future. I don't know if any historians have this as their motto, but I like to think they do.

there is perma.cc which gets around the archive.today and waybackmachine gray area by only allowing link archival through affiliate libraries. It's as close as you get to the government archiving journalism/culture, besides the loc.

https://en.wikipedia.org/wiki/Perma.cc

https://perma.cc/

interesting. the sing-up friction and overall restrictiveness clearly demonstrate why the grey area is necessary

>You can still do that now, but it's not as important until it goes down.

Yes. It's very easy to get into the mode of "I can always download it"--until you can't. Or at least it's very hard to find. I've been pretty good over time. I probably have copies of 75%+ (probably more for stuff I actually care about). But I've definitely written for sites that don't exist any longer, at least a couple of which were behind paywalls.

Reply on news.ycombinator.com