Hacker News Reader

> Even in niche areas, I think it's still an enormous task to challenge Google. Despite the fact that 90% of my searches for "WTF does this stacktrace mean?" end up on StackOverflow, I invariably start my search on Google because every now and then it gives me useful non-StackOverflow tidbits, and I know I can always drill in more with specific tags on StackOverflow later.

It's telling, that a Google search scoped to a particular web site (xyz site:website.com) almost always produces better results than that web site's actual built-in search engine. If a web site that knows in detail its own content, its own users, and its own domain, still can't make their built-in search better, how does anyone else have a chance?

I don't think the fact that the web site owner hasn't implemented better search than Google for their site is evidence that they can't do it, simply that they haven't.

This entirely. Big companies grow complacent. Which to me, in time, equates to Google's timely downfall.

No empire lasts forever; when was the last time you went to Sears?

I guess the motivation is low as I frequently have to do this it seems.

Google knows about external inbound links, so has more data on which to rank your pages than you do.

Assuming someone has ever used that link you should have that data as well and with potentially even more relevance than google because you can see which pages are most often visited through external links.

I think the most important data that is difficult to obtain is a history of which links people click on for each search query term. I think that's really the special sauce that Google has that others do not. There is a catch-22 where you need a lot of users to get the data but you can't get the users without the data. Initially Google pulled ahead with inbound link counting, but anyone can clone that. It's the click data that's missing.

hah, that one is easy. People click the one at the top the most, then nr 2, after that nr 3, etc you see the pattern?

Google was epic when it started but quickly ended up indexing it self in a near closed loop. Everyone else was not important, it was all about the picture in front of them that was really just a mirror. Everyone stopped making home pages, forums and weblogs because it just doesn't make sense to do it.

Shit, people looking to buy something talking among their own before doing the purchase? KILL IT WITH FIRE!

my take is that the big bottleneck in search is not the index or the search but in understanding what it is the user is searching for in the first place.

that becomes a big can of worms very quickly, and starts to put into perspective some of googles other most successful verticals: NLP, Translation, Maps ("Near me"). Of course a lot of the verticals are also a generic means to get value out of all the server resources and experts required to operate the engine and then additional related branches to those verticals (email -> drive -> docs)

> I would be interested in anyone posting what they think is an economically viable path to challenging Google.

You only search whitelisted sites, which removes a lot of the SEO and blogspam from the results, and makes the index is wildly smaller. Search is free, but users can also pay a cheap subscription for the ability to flag sites as spammy, submit sites they think are missing, and submit queries the site has trouble with. Someone manually goes through the top N most-repeated submissions each day (early on this is all of them) and determines whether to tweak the whitelist.

Because your initial audience has to be people who are very enthusiastic about better search results, you should identify what they're interested in and focus your efforts on initial whitelists to things they care about. That might be technical questions (MDN StackOverflow, every random framework's docs site), or maybe whatever it is journalists would want to have access to (I'm not qualified to answer).

I don't think VC-encouraged hypergrowth is possible with the above strategy, but you could probably get to ramen profitable in a reasonable time frame, with slow growth ever after.

> I don't think VC-encouraged hypergrowth is possible with the above strategy, but you could probably get to ramen profitable in a reasonable time frame, with slow growth ever after.

The fate of runnaroo seems to provide evidence to the contrary.

Surely the only viable path to challenging Google is to own the web browser. Most users will use the default search engine.

I don’t know about “only” path but not sure why this is downvoted as it’s a valid point of view.

I would have thought this back in 2005, but eventually IE did fall. If users eventually dropped IE, why can't they also learn about Kagi or DDG or bing?

>It would just take at astronomical amount of capital to challenge Google at generic Internet search, [...] For example, Amazon is quite content just being the place you want to go to first for the things you want to buy, as that is obviously the most lucrative area.

I understand your intent with that domain-specific example but surprisingly, that reasonable intuition sometimes doesn't work...

E.g. for Amazon product search, I often deliberately use Google's engine to find the Amazon page because Amazon's own search can't do it. See example screenshots in my previous comment about that: https://news.ycombinator.com/item?id=27539199

"though I think there are a few areas that Bing is better"

Curious what you think these are. I prefer Bing's UI to Google's but aside from that Bing is (IMO) a dumpster fire that quite literally makes me feel negligent (towards myself) when I use it - am I missing something?

Porn, in particular, but surprisingly I find that video search in general is better in Bing. Surprising because most of those video searches end up on YouTube, yet I seem to get better/higher quality results when I search on Bing than I do when on Google or YouTube.

Would there be a way to crowdsource the web crawling effort? Crawling@Home? I think that even with this you'd need a legal department to deal with takedown requests and illegal material.