Hacker News Reader

Kudos for your courage to make your great ambitions public from the start.

1. Does the site do any crawling on its own, or is the public index only fed from submissions?

2. It appears Umlaut/Unicode handling needs some work: When I search for "Käse" (German for 'cheese'), I get the response "0 results for 'Käse' in 'www' (0 ms)".

At this point I'm not sure if there's actually 0 results or if it was actually searching for the escaped string.

View on HN

Thanks!

1. You may submit a page. When I have a little more capacity that just 1 CPU/1 GB RAM I will also crawl.

2. I'll look into it. Thank you.

Altavista ran on an AlphaServer 8400¹: maximum 14 × 612MHz CPUs, 28G RAM.

¹ https://en.wikipedia.org/wiki/AlphaServer#Turbolaser_Family

Would the common crawl dataset be useful to you starting out?

http://commoncrawl.org

Yes absolutely. I have been holding off crawling because I have no server capacity yet. That will probably sort itself out pretty soon from the looks of it. When I have the disk space I'll start using their data.

Reply on news.ycombinator.com