Ask HN: If you were to build a new search engine. What would be your plan?

First, I would gather some key buzzwords such as machine-learning, privacy-first, de-google, and whatever else the kids/investors like these days. Then, I'd do my best to create a brand associated with some such trendy buzzwords. Ideally I'd have some friends, or friends who have friends; friends who feel like investing and/or shilling hard to other investors, and who can also shill hard to potential users. If all goes well, after a couple years or so of heavy shilling and growth hacking and whatnot, me and my friends get our accountant and lawyer friends to do some sort of equity/legal sleight of hand, we the founders make a nice exit just before or just as our grassroots, cool, hip new product degenerates into some spurious incarnation of what it claimed it wouldn't become, and we the founders live happily ever after. Somewhere in that plan we build a search engine or whatever, but that's not really important.

I would start with certain category of the web, by topic and start with a topic niche search engine, and start to cover more categories one by one if the search model is successful.

I've been working on and off on a search engine for the past few months. The way I see it you have 2 choices if you want to build a general-purpose search engine:

1. Scrape results from other search engines, similar to Searx

2. Build a Bing wrapper, similar to DuckDuckGo, Ecosia, et al.

With Ask.Moe I've tried both. The problem with scraping other search engines is that they (especially Google) are very quick to CAPTCHA your requests, so you'd need to setup and maintain tons of proxies and you'd likely struggle with search engines returning results based on the proxy location since they like to ignore URL parameters/user settings. The problem with building a Bing wrapper is that it's incredibly expensive ($3/1000 transactions for just Web, $7/1000 if you want Web/News/Image/Video/etc. as well). Let's say reaching the frontpage of Reddit/HN netted you 100.000 visitors submitting an average of 10 queries - that would end up costing you $3000 or $7000 depending on which plan you were using. And you most likely wouldn't have any ads/affiliate links to offset the costs since Microsoft Advertising (I believe that's what DuckDuckGo/Ecosia use) is invite-only and won't even entertain talking to you about your new search engine.

The reason you see a new search engine pop-up every month and disappear again the next is because it's easy to build a Bing wrapper but incredibly difficult to make it profitable.

My plan is to build various categories (so far I've built https://ask.moe/math and https://ask.moe/currency - podcast or news will likely be the next) that are inexpensive to operate and can hopefully generate some revenue. I'm also strongly considering supporting accounts where users can buy credits to use the general-purpose search engine, but I think it will be very difficult to convince people to pay to use a search engine.

Who needs to search for something the most? What topic doesn't have a destination website? I would figure out a niche that needs search and is an industry that is highly fragmented and likely to stay that way. Unfortunately, i am not sure if there are any :), at this point it might make more sense to build a destination site to claim that niche. I.e. you go to amazon for books/shopping, you go to facebook groups or reddit for info from people, travel stays at booking/airbnb... etc etc. what niche really needs external search? /phone ramble

Im actually planning to build an ad-only search. People will only see ads and the highest bidder goes on top. It will be 80% as good as google