Show HN: A Simple Search Engine

Well. Forgetting lowercase but doing the rest of the search parsing was a bit of a mistake. That should now be fixed, though it might take a little while to propagate through the caches.

---

There's really, really, not much to the tech behind it. Nothing surprising, and nothing really special.

The website itself is 446 lines of a Python bottle app, behind an aggressive nginx cache.

For example, the homepage is:

    @bottle.route("/")
    @bottle.view('home')
    def home():
        # Get some random articles
        random_articles = []
        for url in list(set(get_recent_urls(N=9))):
            random_articles.append(record(url))

        random.shuffle(random_articles)
        random_articles = random_articles[:3]

        # Get the top tags
        tags = top_keywords(N=10)

        count = url_count()
        update = datetime.now()

        return {"articles": random_articles, "tags": tags, "count": count, "update": update, "tag_kind": 'Trending'}

There's no... "magic" there.

The link-fetcher is slightly larger at around 700 lines of Python, but most of it is taking edge-cases into account. Like finding where the timedata is hidden on a BBC page (comes as an epoch), or on CNN's lite site where it doesn't have a tag (and has to be manually parsed), etc.

The vast majority, about half of the fetcher, is a series of fallbacks to keep trying until it finds something, before it finally gives up.

The index will be interesting as it gets bigger.

I've put a preference on timedata. So any three sites with a recent article can hit the frontpage. Because I want hackaday appearing as much as I want the BBC, etc.

Sites that I can't grab the publishing time for will still appear, but they get ranked down and you'll need to find them by searching.

You won't see a difference in results between browsers. There's 0 Javascript on the site.

If you're seeing something funky with the CSS in Firefox... Then you probably have a dark mode addon. And the addons and Chrome disagree with how dark mode should work. Try disabling it.

I'm seeing 40+ responses for each of those words when searching, across Firefox, Chrome, Firefox Android, Chrome Android, and even Lynx.

That is the entire current corpus.

It sounds like you were hitting the site whilst it was under a bit of heavy load. From my logs, about the time you posted here on HN, someone was tossing `siege` at the site, and it is not a heavyweight server.

Running certain searches do seem to be able to trigger a pathological response, however, so I'll need to look into that a bit more. Likely to do with some of the nltk stuff it uses when it tries to handle searching summaries.

I get this on Firefox: https://imgz.org/i3mvCHVa.png

I only use Firefox. So it's probably not just that in itself.

It sounds like you're using something that adds enables a dark mode preference to Firefox - and currently some of the Firefox addons and Chrome disagree about some of the finer details of how that should process.

Try it without asking for dark mode.