Hacker News Reader

I can tell you that a project is already doing this today for archiving with no signs of mitigations (request denials, poisoning the responses, etc). It’ll likely eventually end up in the usual places when you want to see something from way back in the day.

They arent scrapping the whole site. The entire site is 99% spam. Its scrap on request, if its not already cached.

Sure, but that doesn't make a difference. Even the top 1% of the most requested timelines and tweets are huge amounts of requests. That's many many orders of magnitude more data then new articles and requires a large number of accounts.

That should be “scrape”/“scraping”, unless you mean they should be throwing the site away :)

> But it's only a matter of time before that changes.

If anyone at the helm is paying attention, which seems iffy these days.

>- X has the technology to reliably detect such accounts unless they are incredibly well "hidden"

Has, or had? With massive layoffs, they might not have the staff or know-how left to do this, or the time to implement it.

>- X has the incentive to reliably detect such accounts as it has shown in the past by shutting off data access. It seems to care much more about this than most news websites

OK, but still wanting to do something, and actually doing it, are two different things. I'm not convinced this hollowed-out shell of a company still has such ability.

>especially with all the other stuff going on at X right now. But it's only a matter of time before that changes.

This seems to assume X is going to continue to survive and in fact thrive; I don't think that's a good assumption.