Comment by SpikeyCoder | Hacker News Reader

Comment by SpikeyCoder | Hacker News Reader

Parent Full thread

If the goal of the web was just to transmit objective facts like 'who created Linux', this wouldn't be a problem at all. Wikipedia handles that pretty dece.

The issue arises when we move away from objective trivia and into subjective, localized buying intent, which is where the web monetizes itself.

If I ask an LLM 'who created Linux?', there is one right answer. But if I ask an LLM 'who are the best commercial roofers in Houston?', there isn't one right answer. There are dozens of highly qualified local businesses that do possess unique value, unique pricing, and unique availability.

When an LLM answers that roofing question, it typically cites 3 to 5 businesses. The other 40 legitimate roofing companies in the area are left out. My study isn't arguing that every single one of those 40 companies deserves to be in the answer; it's pointing out that those 40 companies currently have no idea they are being left out.

In the Google era, if you weren't on page 1, you could look at Search Console, see your ranking, check your backlinks, and understand why. In the AI Search era, businesses are being scraped to build these answers, but they have zero telemetry on whether they are actually making the cut. This index is just an attempt to provide that missing telemetry.

>But if I ask an LLM 'who are the best commercial roofers in Houston?', there isn't one right answer. When an LLM answers that roofing question, it typically cites 3 to 5 businesses. The other 40 legitimate roofing companies in the area are left out.

I don't see how mentioning the other 40 should follow from that fact. You get exactly what you're asking for - the best, according to agent's judgement. Judgement and processing of simple search results is exactly what you're using the intelligent agent for. Ask for a list of all commercial roofers in Houston and then you can expect a list.

It can be argued that models are clustering their replies around a few options when asked to choose from a list of equally valid ones (due to mode collapse, distribution biases, primacy/recency biases etc), but these options are not equally valid for the agent, it looks at their sites or possibly in some other places like review sites to rank them. Its ranking criteria might be not good at all, but that's another question.

That's not _entirely_ true. Search Console now has a Generative AI page where you can see impressions per page now. Bing has something similar.

Also, I would assume that really "GEO" is just like "SEO". If you rank on the '1st page' of results for whatever common searches, you are _very_ likely to rank the same way for LLM questions, because all the LLM is doing (nearly all of the time for 'best roofers in Houston') is doing a web search and summarising the first x results. So if you are on page 1 for that term, it's very likely IME that you will get mentioned on LLM answers for that.

Makes sense! Good point.

You're effectively reminding us that brands don't get understand well how to influence ai seo

Small businesses don't. You're right. But somehow those larger companies seem to get cited. Wouldn't it be nice if we all knew how we were being influenced, both potential consumer and small business owner? Let's have the lid off!

Good, let’s hope there’s no SEO arms race for this too. Maybe the best roofers have a chance of getting cited over the ones with the best SEO agency. Seems unlikely though.

How are we defining the best roofers?

The model does it in this case, not "we". Classic SEO does influence it because it relies on traditional search, but then it estimates the best according to whatever cognitive capabilities it has, to its understanding of user's request, to what its harness tells it to do, and to its character engineered during post-training (and models' biases are very carefully engineered).

I would expect a judgement of a modern model to be superficial for this request, but still better than nothing. A deep research agent might be a lot better and much less susceptible to "optimization".

That’s why I, from now on, only allow citable information to be scraped. The return of cloaking, lol.

Not really, ahrefs has already started working on this and there are several tools out there that track GEO.

Also what is the incentive for me to rank on LLMs? People do not use it to click sources or even take action. Do you have any proof that being ranked as the best roofer in houston drives sales more than not being ranked as it? If not, why should i care as the roofer?

It depends. If you are doing blog content with the idea of upselling users to your product, probably not as much (because the LLM can just give the user the answer).

However, if you are looking for the best product/service/whatever, then yes it really does matter. I've bought _so_ many products because of LLM recommendations. For example, I wanted a new webcam, I asked the LLM to find me the best ones with a large sensor and Linux compatibility. It gave me a shortlist then I chose one and then I bought it.

This experience is far better than trailing through dozens of pages of (even pre LLM) SEO slop.I just tried the same on Google search and all the links recommended a camera with ~10% the sensor size that I bought.

Reply on news.ycombinator.com