Ask HN: Resources for Doing Search Well?
I work at an early stage startup and we need to build a search experience. Interested to learn about the space.
Why not buy?
If the company can buy, why isn't it?
If the company can't buy, it is under-capitalized and that's a sizable problem because search is a commodity.
Anyway, the best search experience is not needing to search for the same reason that I don't enjoy searching for my car keys when I am trying to commute to work.
Good luck.
Start with open-source Meilisearch, and don't look back.
The "buy" options people talk about are tired-old snake oil, and you'll end up with buyer's remorse.
First things first: You might have seen all the news around the failure of Neeva, a search engine startup. There is a core structural reason why Neeva has failed, and I predict that all pure-play search engine companies not named Google will eventually suffer the fate of Neeva. Keep this in mind when you decide to chain the future of your technology stack to the fortunes of a search engine company. I can give this advice with confidence because I work in such a company, and our management is clueless.
Back to answering your question: any answer you get can't be complete, because the term you used, "search experience", is ill-defined.
Here are the 3 types of search use cases, in increasing order of complexity:
1. Traditional, barebones, keyword-matching search, with some level of tolerance for typos. Meilisearch is a rock-solid option if that is your use case.
2. Traditional search with deep intent recognition of search queries. This is where the snake oil search engine companies try to sneak their way in. I'll go into detail further down below.
3. Search-aided knowledge work, a problem for which LLM technologies (the ChatGPTs of the world) are the solution. Before November 2022, for lack of a better option, the space was being (very clumsily) served by traditional search technologies.
Details on number 2. My company spent a decade and burned several tens of millions of dollars on developing, refining and marketing a search engine codebase that was outmatched in an instant in November 2022, when ChatGPT became a thing. How will my company make all of that money back? Management doesn't know, but they hope against hope that they can charge startups like yours for access to the snake oil.
All of that engineering effort was expended on writing code by hand, without help from a copilot-type code generation tool. Based on what I have seen in the codebase, I estimate that close to 90% of that code is boilerplate that an AI could easily generate, so all of that effort can nowadays be easily replicated by one mid-level engineer working for a few weeks (search is a very well-understood problem, very competitive and very commoditized).
My point is, if you're going to buy instead of build, buy from a company that has benefitted from the AI revolution, as opposed to being disrupted by it.
To close out, I will leave you with this ranking page, those down-arrows should tell you everything you need to know about the search engine space: