You have to balance being good with maximizing profit. If you aren't good, your userbase will go elsewhere.
When you have +95% of the market, you can merely be good enough that users don't leave.
In this case, until Google literally starts serving ads-only and links to other google products (Hi Chrome!), they are not going to lose any money.
Bing's AI shows some promise, but I can't find anything better than Google. DuckDuckGo is shit. Classic Bing is shit. Brave Search showed promise, but it's also full of spam and with a smaller index. Marginalia showed some promise for smaller websites, but it's small, too.
All of them are unusable for local searches, except for Google, which is where I need search most. That and searching for obscure programming-related error messages.
Kagi doesn't even use it's own index, you're basically paying for a UI making API calls to Bing and Google. How well does changing the ranking work anyway, if you don't have your own index?
I want to believe that if Google were more aligned with users, search would be better. But where is that better search engine to showcase it?
Do a poll on HN. I bet that the vast majority of users here still use Google's search, and you can't blame HN users of not knowing of alternatives.
If Google really keeps its market share due to their monopoly, where is that better option that's being ignored? Tell me and I'll jump on it.
Although Moore's law brought down the price of hardware and information processing dramatically in the last 20 years, it is still fairly expensive to crawl the Web, index it and rank it. Hardware cost + engineering cost can escalate pretty quickly, unless you decide to have smaller index than major search engines but then users will complain that search results are not good enough.
Due to the defaults it's the only search engine most people ever heard of. Think of Plato's cave allegory.
If all your life you've only used Google and never anything else, making it the ground truth for you, how would you know it's bad in order to motivate you to look elsewhere?
LLMs
I decided to first try ask an LLM, so I asked Bard some targeted questions and asked for sources, and had all of the answers I needed, conveniently bullet pointed within 3-4 minutes, and all I had to do was go and verify the sources, job done.
My favorite is recipes.
The blog spam around recipes is notoriously bad and is a direct result of Google Search.
Using these LLMs, all I need to do is tell it what I'm looking for in general and it will give me an entire recipe with no extraneous information.
It can handle adding or removing ingredients, substitutions. It can adjust servings. It can flip the recipe to work in a slow cooker or pressure cooker.
I've even had some limited success where I list restrictions based on picky eaters in the house for creating longer meal plans. No search engine can compete with that.
There were apps designed around these use cases. I foresee these sort of nuanced and personalized interactions being a key to drawing people away from search engines.
I'm going to give it a go for recipes, because as you say, blog spam is a nightmare. My wife has a lot of cookery books which she likes for her style of cooking, but I'm a little more "ad-hoc".
I haven't used ChatGPT extensively so I can't make a comparison, but with the right questions, I've been able to get great answers to technical things I can't be bothered to look up by hand.