Hacker News Reader

Comment by pogue | Hacker News Reader

The EFF just recently wrote an article with instructions on how to persevere & archive your own tweets on the Wayback Machine, but it involves exporting your own backup and uploading it to them. Since the API is completely cut off from Twitter, there is no official way to backup other people's accounts.

But archive.today uses scraping and all sorts of tricky methods to bypass paywalls. I honestly don't understand why Nitter can't just stay logged out and rotate IPs. Although I'm sure that gets pricey when other people are accessing it constantly.

https://www.eff.org/deeplinks/2024/01/save-your-twitter-acco...

View on HN

If the scraping model is impaired due to aggressive countermeasures, end game are browser extensions that scrape as users view the site and ship scraped data back to a processor, similar to recap the law (uses an extension to scrape the PACER legal database and ship digital artifacts to the Internet Archive). Care will need to be taken around potentially sensitive data that could be shipped if users are logged in.

https://free.law/recap

Oh, that's a very cool project! How successful has it been? If it wasn't for Sci-hub that would be a great idea for the scientific publishing world as well.

Very successful. Millions of court filings extracted and indexed, and a crucial component in driving down PACER costs.

https://news.ycombinator.com/item?id=24086570

> I'm the director of Free Law Project. For the case mentioned in the article we actually did a full expert testimony figuring out roughly how much per page it'd cost to run PACER using AWS GovCloud and a handful of other assumptions. It was...half a ten thousandth of a penny per page

https://www.courtlistener.com/docket/4214664/52/15/national-...

https://news.ycombinator.com/item?id=24085158

> Government’s PACER Fees Are Too High, Federal Circuit Says

https://news.bloomberglaw.com/white-collar-and-criminal-law/...

This model also works well for deep web content archiving.

There was a gaming message board where someone wrote a browser extension that would back up all topics someone visited in the background while they were reading them. It became important for archiving as much content from those forums as possible as the forum was in the process of shutting down.

Reply on news.ycombinator.com