Hacker News Reader

Comment by beeboobaa | Hacker News Reader

Anyone archiving every website they visit? I often want to search for something I read recently, which is awkward to do if you don't remember where you read it.

I'd like to store everything I visit with my browser once the page is loaded, but haven't found any nice solutions for this.

View on HN

A bit of a late response, but yes I've been storing full text of every website I visit and it's excellent for finding stuff again.

The idea is to index pages as you visit them using a browser extension, thus avoiding all the pitfalls of being treated like a bot.

Here's the project: https://github.com/iansinnott/full-text-tabs-forever

YaCY (self-hosted search engine + index) can do this when using it as a proxy.

https://yacy.net/

Interesting! How well does this proxy deal with bot detection (Cloudflare and such)? Most proxies I've used, usually without archiving capabilities, have me fill out CAPTCHAs for every other site.

On MacOS HistoryHound gives you a searchable index of webpage content, though it doesn't store the full contents of pages in a way that can be viewed offline.

https://www.stclairsoft.com/HistoryHound/

Created Linkwarden exactly for this:

https://linkwarden.app

Nice! This is very close to what I'm hoping to find. My main issue is that it seems the pages are fetched from the linkwarden server, instead of preserved by my browser extension. For example if I use the browser extension to save a HN page, the saved version is an unauthenticated view of the page instead of showing my username.

If the pages were preserved directly from my browser now additional requests to the server would be required, and it would be realistic to archive everything automatically, instead of only when I press the button.

I'm using the Forethink web extension for that. It keeps a local index of every page I visit for searching

https://forethink.ai

was that not literally a built-in feature of Opera 12?

Reply on news.ycombinator.com