Hacker News Reader

Comment by xnx | Hacker News Reader

> Because direct licensing isn’t available to us on compatible terms, we - like many others - use third-party API providers for SERP-style results

Crazy for a company to admit: "Google won't let us whitelabel their core product so we steal it and resell it."

View on HN

Seems like an open question as to whether that violates any laws.

Another way to look at it is that if you publish a service on the web, you have limited rights to restrict what people do with it.

Isn't that the logic Google search relies on in the first place? I didn't give permission for Google to crawl and index and deep link to my site (let alone summarize and train LLMs on it). They just did it anyway, because it's on a public website.

Google's stance is "I can copy you and you can't stop me" as well as "You can't copy me, I'll sue you"

Maybe it has changed but Google doesn't look like it uses litigation as its primary weapon. It defends itself but rarely attacks.

The are however more than happy to use technical measures, like blocking accounts. And because of their position, blocking your Google account may be more damaging than a successful lawsuit.

Google at least claims that noindex will keep your site from getting crawled [1]. Do people think this is false?

[1] https://developers.google.com/search/docs/crawling-indexing/...

Strictly speaking no, that doesn’t prevent crawling - at the least Googlebot has to fetch the page to see the meta tag or the robots.txt to see what’s allowed, and it will periodically recheck for changes.

It doesn’t even prevent indexing. If a page is linked from elsewhere, Google will show it in search results even if noindex’d.

And why does Google get to set these rules on my site anyway? I didn’t agree to them.

It is basic antitrust practice. If a company starts to control a vertical so much so that they start to exclude others, they get broken up into components and ordered to offer the basic infrastructure service to others. This is how it worked for 100 years (read up on telecoms/fiber; train companies/railroads; heck, even roads used to belong to people in the UK). This is why we have net neutrality - I recommend Tubes by Andrew Blum to go the heart of the matter. Imagine Internet if Google was able to throttle other services if you are not using their own? Here the author is arguing the search index is like infra that needs to be shared for public good. The state will not confiscate it - Google will break it into an independent company, will start paying for it, and let others to pay as well. It's not whitelabeling, stealing and reselling. Gosh - just read a bit people.

What's the alternative? Building a competing search index as a relative nobody on the web is very difficult, from the outset, and is made more difficult from sites taking extra measures to stop bots in general now.

Google's crawler is given special privileges in this right and can bypass basically all bot checks. Anyone else has to just wade through the mud and accept they can't index much of the web.

Pretty standard business practice though. There's no ethics in making money.

Is it much different than what Google AI Summaries do?

But in this current climate, they can admit it and then dare Google to tell them to stop... After Google has just had an antitrust ruling against it for dominating the search market.

Google doesn't really have a leg to stand on and they know it.

Strange to pick on Kagi when there's much bigger companies on that list.

Those companies allegedly have used SerpAPI (probably to check visibility), but not to resell a Google Search knock-off.

> knock-off

Is it though? It feels so better than Google results[1], while being still built partly with Google results.

In the last 3 years as a Kagi customer i have rarely if ever felt the need to use bangs !g and on few occasions i did use them, it was with instant regret.

In the previous decade or so using DDG, using bangs !g Google would be 30-50% of searches, i would have to consciously try the results first instead of starting with !g and then think to myself DDG was at least getting the query data to improve their results.

[1] While the de-cluttered UI is a relief, on just the results list comparison, Google search is so bad that less time saved in not redrafting the queries constantly, filtering out the spam, the AI summaries, sponsored content, all the "cards" , recommended search listicles on is worth more than the $10/month.

DDG's results are primarily Bing results while Kagi's results are primarily Google results. It makes sense that you feel the need to escape from Bing to Google more often than from Google to Google.

Perhaps, but each time I do go to Google the results are painfully bad. I don't think Kagi is just proxying Google- while they do use it as a core source, they rerank much better

The blog post talks about that specifically - Bing unlike Google does have Index licensing program but their terms forbid reordering that is key reason Kagi is not also using Bing in their index mix.

My point is Kagi is similar to a car tuning company like Hennessey, Brabus they take a base product and make it much better for a premium, they are not selling knock-offs.

Even the article posted (and search itself) has Google IP address.

Crying to Big Daddy Government because those other mean companies won't give away their secret sauce is pretty lame and doesn't make me want to reinscribe.

Reply on news.ycombinator.com