Hacker News Reader

Comment by codexon | Hacker News Reader

Google has been killing all but the most widely known domains for a very long time. I've mentioned this repeatedly on ycombinator multiple times, but only people who have made their own website 15 years ago and tried to grow it know what I mean.

https://news.ycombinator.com/item?id=38923627#38933675

My recommendation is to start moving to some other closed platform that is not part of Google search like Facebook, Twitter, Youtube (yes I know Google owns it but its still not part of the search ecosystem).

Tying your entire business to how high you rank on Google search is always going to eventually end up in disaster like this.

View on HN

Read past the provocative title, and Google actually seems to be doing the right things here. They cracked down on product reviews that aren’t actually testing the product in 2021, and the article says big media companies (presumably with lower quality review content) suffered as a result.

But then those media companies found a loophole with "The Best X" lists that weren't subject to the 2021 Products Review Update changes, which lets them continue spamming affiliate links while avoiding the new requirements.

So now independent sites with actual reviews are in a holding pattern for these search terms, waiting for Google to bring the hammer down again on sites that are evading its quality metrics. This article is pretty clearly an open letter trying to bring attention to this issue.

If the team at Google working on ranking for product reviews is reading this, I hope you have another update in the works to close this loophole. H1 planning just wrapped up!

Edit: The title on HN has changed to be less click-baity. The original title was "How Google is killing independent sites like ours".

Title aside, the article is quite excellent and does a great job of explaining the product review niche of SEO. Kudos to the authors.

I still believe the original title is warranted.

The fact that Google has to manually step in to intervene or else the big domains get all the top rankings tells you that they are very heavily biased towards big media domains.

does it?

I would happily believe that Google is corrupt in this manner, but the reason big domains have the advantage here is because they can afford to pay teams of people with the express purpose of gaming the system. this is true in all industries, everywhere, and it can only be fixed with society-wide change, which, short of a world war (or, more likely, two), isn't going to happen

It's not about being corrupt, it is about doing the lazy thing in order to fight spam. No one ever got fired for suggesting New York Times or Better Housekeeping.

To me it feels like a big brand website hardly needs to try in order to rank #1 even without doing a bunch of SEO.

> It's not about being corrupt, it is about doing the lazy thing in order to fight spam. No one ever got fired for suggesting New York Times or Better Housekeeping.

> To me it feels like a big brand website hardly needs to try in order to rank #1 even without doing a bunch of SEO.

To rub salt into the wound, any forum will happily let anyone and everyone post in support of any big brand, but instantly spam-block some lone developer trying to showcase their product (HN is different in this regard).

Microsoft/Jetbrains/Apple releases a new paid product, available as a paid subscription only - dozens of people will get upvoted telling the forum about it.

Some lone indie dev releases a free-tier search tool with optional paid tier, and they're banned for spamming.

Now I understand why you'd want to block people who self-advertise, but there's gotta be a middle ground.

Why is advertising on behalf of a company that has a larger ad budget than all the current readers salaries combined okay, but advertising for your own product is a bannable offence?

There really should be a middle ground where (like on HN) the audience understands that someone who posts something that took them 6 months to create is not the same as someone selling love potions or stock tips.

It's become bad to the point an indie dev can't even post to get feedback even when the thing he's poured his blood into completely matches the forum's topic. Extra minus points if he offers it for free (so suspicious!)

You are now expected to develop in the dark, without ever mentioning anywhere the thing you are working on and fully knowing that your "blog" gets completely ignored by Google

I personally refuse to host my blog on a "trusted" domain. I likewise refuse to use Twitter, Youtube, Tiktok, and whatever else works nowadays because search engines can't be arsed to tell the difference between legitimate content and spam AND because people's attention span have become smaller than a tadpole's

This is a sore point for me personally. That's why I went through the effort of creating my own personal search engine that filters out all this SEO spam and made it public for others to use https://www.aisearch.vip

Big brand websites usually have decades of trust and backlinks to lean back upon. As a person working in SEO for 20 years, I'd argue calling search engines lazy is just lazy. It's a billion dollar industry. Whatever ranking factor they decide on, someone will abuse.

Consider how bad the results would be if I could generate 100 scam sites in a day and outrank traditional media. Only to repeat it tomorrow. Now step it up and have tens of thousands of people doing the same thing. Every day. Trusting a 30 year old domain isn't lazy.

Google's job is to fight abusers to give you good results. Is that not the point of a search engine? What else are they going to spend their time working on?

Putting the burden of SEO on everyone else is going to cause people to abandon making websites and seek other platforms like reddit, facebook, instagram, youtube etc... as they have been.

It also wasn't as heavily weighted to manually trusted domains before. I used to have a bunch of backlinks too, but all blogs/forums were manually downranked.

Now these "trusted" domains get free reign to do whatever they want like this.

Years ago Google banned BMW (I think) for their then very spammy tactics. Then they were forced to restore BMW because people searching for BMW expected to find BMW in the results.

The Google spam team has always been underfunded, and understaffed, and when the choice was 'do the right thing about spam' vs 'do the thing that is profitable', they always choose profit.

the solution there is to not ban sites, but simply deprioritise them for a while. you get caught being naughty, your entire domain gets hit with a nasty downranking that gradually decays. put a little flag next to the link explaining to the user why the result is so low. except that the second thing will never happen because it seems that Google policy is to keep search users as powerless and uninformed as they possibly can.

What is the loop hole?

That Google cracked down on "product reviews that aren’t actually testing the product",

but not on top 10 lists (that mentions some in-fact-fake testing).

So therefore the big companies publish loads of trashy top 10 lists

Google search has become worthless for me. I use bing instead because of how horrible google results are.

On most searches, especially with my phone, the results are almost all sponsored and rarely what I'm actually looking for.

Google search has gone from being one of the best to being ask jeeves at it's worst.

> Google search has become worthless for me. I use bing

I’ve been thrilled with Kagi. It’s the first time in over a decade that searching became fun again.

The Quick Answer feature (Kagi’s LLM) filters through SEO better than Copilot, and the results are noticeably higher quality than ad-based engines. At $5/month for 300 searches, it’s cheap to try out (both for experience and if you actually notice the search limit).

Same, I finally gave up and tried it after Google just stopped being remotely useful, and DDG is just a reskinned bing. A week on Kagi and I signed up for an annual plan, and never looked back.

The main problem with Kagi is that it's a paid service with no free tier.

I get their reasons for this, and it totally makes sense -- but that's also a big problem for their growth. I know very few people who would pay for a search engine.

> that's also a big problem for their growth

I agree, but it’s a good early filter for conversion. The difference in quality, for me and everyone I’ve gifted a month to, is stark enough to make paying for search for the first time worth it. Given the absolute cost (for the cheapest tier, paid annually, less than $50) it’s a psychological hurdle more than a financial one for most Americans.

Also, drawing those eyeballs from the ad-driven engines has a disproportionate effect on their marginal ad prices (in the long run). So if you need a sense of vengeance to get you over the hill, there you go.

Regarding ad margins:

If kagi saturates the market of people that can afford to spend $50/year for a decent search engine, then Google ads will only reach people that cannot. This would greatly reduce the value of their ad inventory (far more than the percent market share they’d lose).

> a big problem for their growth

But do they need to grow to the size of Bing, Google or just DuckDuckGo? If they just want to grow a sustainable business, then it's a feature of their business model.

No I'm sorry. I don't mean "growth" as in infinite and unsustainable growth like VC-founded startups. I mean "growth" as in adopting a bigger market share.

But, again, do they need to? It seems to me like "market share" is a metric relevant to companies pursuing VC-founded, unicorn lottery-ticket scale. If they generate enough revenue to pay competive wages, cover their operating costs, and make a reasonable (real-world, not VC-world) return on investment, they're a gosh-darn success. It's only within tech, where valuations and evaluations sailed off into ZIRP-ified bizarro-world, that people think of that as a failure of ambition or execution. I think it's time to re-assess our mental models.

> do they need to?

I don’t know what Kagi’s minimum sustainable size might be, but it’s probably bigger than what it is now. Particularly if they want to stay competitive with LLMs.

I suspect you're correct, and am rooting for them to hit sustainable, as soon as possible. I don't know what their maximum sustainable size is, either - that's equally important, though always a moving target. I only wanted to point out that neither inflection point, for a paid-service business model, has to do with "market share" - that's a VC thing, to which I'm increasingly allergic.

I use Kagi for finding things and LLM for asking questions. Two different use cases. I want them stay separate but I am probably in the minority.

> But, again, do they need to?

Why not? There are tens of millions of people who need/want a high quality search engine and can pay for it. Kagi deserves to be successful for having made a better search engine than Google. And their success can inspire other entrepreneurs to start delivering quality information products, so that maybe we can get out of this ad/scam fuelled quagmire once and for all.

Good products and ideas should be successful, that's progress.

That is a complete non-sequitur from the question of if Kagi will die if it doesn't grow.

This is orthogonal to questions of morality and justice.

> This is orthogonal to questions of morality and justice.

What?

If Kagi doesn't grow, I fully expect the owner to eventually shut it down and move on to more fruitful ventures. Nobody owes anybody to keep a business running. So yes, it would die.

Sorry If I wasn't clear. But I think the question is pretty clearly stated in the post you responded to.

The question is: Does Kagi needs to grow to be sustainable, and if so, how much.

>If Kagi doesn't grow, I fully expect the owner to eventually shut it down and move on to more fruitful ventures.

If you had a business that made you a $1 million per year profit, would you shut it down just because it wasnt growing?

Companies need to make a profit or they go out of business. However, most businesses don't need to continually increase users/ revenue to stay afloat. The coffee shop down my street is 100 years old, and didn't need to double in size every year.

I agree that nobody is owed anything. I also think that Kagi is "owed" or "deserves" tells us nothing about how many users they need to keep staff paid and the lights on.

> If you had a business that made you a $1 million per year profit, would you shut it down just because it wasnt growing?

Yeah, I would. We have to remember that these are guys who beat Google at their own game. They beat a company of a monstrous size and revenue at their own game. With that kind of capacity, I don't expect them to be satisfied with a million a year in profit to share. I expect them to go as far as they can.

If you're nobody special doing nothing special, then you can be happy with just needing to pay staff and keep the lights on. Like the coffee shop down your street, or my day job. But Kagi is clearly in a different category as a business.

Last I heard Kagi needs to grow a little bit more from current user base to break even.

Well if their goal is to make money, they haven't beaten Google yet.

Also, if you have a business that makes a million dollars a year and that's not enough, the typical solution is to sell it to someone else for 20 million or so instead of Burning It to the Ground

> that's also a big problem for their growth.

Frankly, I see this as a good thing. Maybe someone else will come along and solve the universal-search-engine-that-stays-good problem, but Kagi's best hope at being useful for me into the future is for them to stay where they are: tiny and used only by a small cohort of extremely savvy and skeptical geeks that aren't worth the effort to SEO-jack.

They just need to be sustainable—growing large would actually be counterproductive.

If they can keep is sustainable and profitable without eat-the-world "growth", that's not a bad thing.

There are few consumer products that have held up against the competing demands of billions users in thousands of different markets and cultures. I'd say there's maybe even been none.

The kind of "growth" you're talking about is a bad but understandable habit among founders and cold financiers, but it's not a requisite part of running a business and generally runs counter to having a good product that serves a specific need well.

> I know very few people who would pay for a search engine.

It's actually maybe ChatGPT et al. that have done most to warm me up to the idea. I've tried Plus for a few months, basically using it like better search. I don't think I'll stick with it mainly because it's a pretty steep cost (enough that I want to go back to not having it for a bit at least, see how much of a problem it really is) - but it does make me wonder if perhaps Kagi can get me a lot of the way for half the price (the non-LLM tier).

>> I know very few people who would pay for a search engine.

A 'fact,' which, if true, makes little sense when you look at it from the PoV as a tool.

With real physical tools, if you only use it occasionally, get a cheap one. But when you use it all the time, it pays to invest in a quality model.

Considering the frequency even the n00best of tech normies of the world use a search engine it makes sense for everyone to obtain such "a quality model." Sadly, that doesn't mean everyone will do so.

[me: Kagi unlimited user since they did the pricing change a couple of months back]

To be fair though, I don't rent my physical tools. I don't pay £20pcm for access to a quality drill or whatever, I just bought it once.

I'm not saying that should be Kagi's model, obviously it does have marginal costs for my usage that Makita doesn't, I'm just saying the analogy doesn't really hold up I suppose.

I think I'd probably prefer usage based pricing, but I know a lot of people here at least would complain that it gave Kagi an inverse incentive to build a good (at finding information quickly and easily) search engine.

Problem? No.

It’s a feature.

That’s honestly their loss. As long as Kagi can sustain itself with its paying members then it can silently retain and grow its users forever.

I know very few people who would pay for a search engine

If Google Search continues its downward trajectory, people will start to pay for Kagi or some other similar search engine. 10-20$ per month for unlimited search is nothing, at least in the western world.

We just haven’t reached that point yet

I didn't think I would. Then I tried it. Then I paid for the cheapest option because I really liked it. Then I paid for unlimited plan because I can't go back to crappy search after I tried the non-crappy one. And, thinking about it, why not pay for a good service? It costs less than cofee+pastry per month, and it improves the quality of my life. I think it makes sense. Some people may disagree, but as long as the service itself works, why would I care?

I wouldn’t have paid for ad-free YouTube until the alternative became unbearable. So too with lousy search results.

I use kagi for better results few times a week and it’s always free.

Oddly as a Kagi user, it does have a fault.

It actually sucks at finding the low cost product.

Want the cheapest esp32 c3... google is a better place to start. I can quickly find the "price to beat" and go deeper elsewhere.

Google and Kagi give the same top hit for “cheapest esp32 c3” for me ($2.50, ali express).

If I add a ? to the end of the query, Kagi additionally suggests an $11 reference board and a redit forum on the topic.

Did I know that without clicking? Who has the 2nd best price because shipping is part of cost...

Yes, the "cheapest" keyword incidentally gets me the answer in this case, but does it in every one?

Product spam has its upside when you're looking for product spam.

> At $5/month for 300 searches

Wow, that's a lower search count than I would have thought. I am pretty sure I'd blow through that in a day or two...

I blew through it in three days and then happily signed up for the $10/unlimited plan!

You'd think so, and maybe that is indeed true for you. But I expected I would use several tens of searches per day, likely with some peaks over 100, but in the past 19 days I've only averaged ~15 per day, for a total of 291. The 300/mo plan wouldn't work for me either, but I don't blow past it as far as I would have expected.

I use an LLM for 80% of my queries now. Fighting Google isn't worthwhile, unless I need a trusted source.

What do you use?

I use the Azure GPT-4 offering. It's not always 100% correct, but for technical questions in areas I'm not very familiar with, it's close enough. I can get much more done in a given amount of time than I would have been able to reading docs and SO.

I know lots of people will point to examples where it's wrong, but I'd suggest trying it out yourself. If you're not intentionally trying to trip it up, it really works quite well.

How do you know when your LLM is bullshitting you

You don't, but at least you have one answer you need to verify vs 100 listings of random garbage to wade through.

How do you know when a Google result is bullshitting you, or if their pre-LLM AI summaries of results were bullshitting you?

The top 5 results (at least in the travel category) are written by copywriters across the ocean. It feels like a machine-generated average of the internet is not so bad.

>Google search has become worthless for me.

Ditto. My most recent example, I asked Google what the thickness of the Pixel 8 is including the camera visor, which was something not listed in the spec sheet since the official dimensions sneakily only list the thinnest point on the phone, not the thickest.

And Google proudly and confidently gave me the answer at the top ... but it was the thickness without the visor, something I already knew since that's in the specs everywhere. I looked through the other results lower on the page and nada, no correct answer.

So I asked Bing and it gave me the exact answer I was looking for at the top measured by some Android review site. And man is that phone a tick boy in that spot. You can probably put your weed in there.

Sure, that's sample size=1 so probably not an accurate test, but still, to me it feels like Google sucks for anything but the easiest context searches where it works because it knows a lot of info about me like where I live and where I work so it can correctly deduct the context, but for other shit not related to me, it's like you're drowning in SEO junk.

This is honestly a terrible example even if it is completely valid. You can't even get Google to find the most basic possible content about, say, oranges, without it being some SEO ad-infested fandom.com page about fruit, let alone product specifications.

>This is honestly a terrible example

I know, I'm not saying it was scientific, I was just sharing an anecdotal mainstream search query which I though was very relevant today for me and maybe others as well and also not super difficult for Google.

> So I asked Bing and it gave me the exact answer I was looking for at the top. And man is that phone a tick boy in that spot. You can probably put your weed in there.

And the most annoying thing is, your phone will not. sit. flat. on a table, because the damn camera will always be unbalanced in height, which makes it an excellent attraction for feline companions. Tap on it and it wiggles. Tap harder, it wiggles more, and eventually the phone will fall to the floor, your feline will look at you with big round eyes and ask for f...ing treats.

I went today to a carrier showroom where they have the Pixel 8 on display to mess around with, and I though the reviewers were exaggerating, but that damn visor is nearly as thick as the phone itself. It almost doubles in height of the phone at that spot. The Pixel 7a next to it had a much much thinner visor despite sharing a similar design.

What the hell did Google put in that thing, lenses from the Hubble space telescope? It's not like they have a 100x zoom lens in that thing or a camera sensor so large it makes a Hasselblad wet itself. And that's before we get to the visible PCB screw heads poking through the OLED panel. On a phone that costs 600+ Euros.

I feel like Google is at least 5 years behind the competition when it comes to HW and industrial design. Or they just culturally as a company don't give a shit about HW, thinking their SW is gonna be the main selling point and the HW is treated like some last minute "who cares, just ship it, it's gonna sell anyway" afterthought.

It's still the only real option, all of the Chinese brands lock their bootloaders, and so does Samsung.

Samsung phones can still be rooted, at least outside of the US (the only place I see complaints about is when people buy Samsung phones in the US from a carrier). It is a bit annoying though that you have to link your phone to a Google and a Samsung account and keep it active and connected to the Internet for at least 7 consecutive days because of Samsung's anti-theft system.

It's quite difficult to obtain EU SKUs for Samsung phones in the US, even without buying through a carrier. (If anyone knows an easy way to get them, let me know!)

eBay?

The recent Pixels specifically don't rock, since the visor goes all the way across.

FWIW I just tried "thickness of the Pixel 8 including the camera visor", and got a snippet from Google:

> The actual dimensions of the Google Pixel 8 are apparently 150.5 x 70.8 x 8.9mm, with the thickness rising to 12mm at the camera bar.

And nothing on Bing.

In Bing I get nothing in the results, although I do get the written Copilot answer on the side:

The Google Pixel 8, announced in October 2023, has the following dimensions:

Height: 150.5 mm (5.93 inches)

Width: 70.8 mm (2.79 inches)

Depth: 8.9 mm (0.35 inches)

However, if we include the thickness of the camera bump, the Pixel 8 measures approximately 12 mm3. The camera visor itself is notably thin compared to previous Pixel phones, but the overall thickness accounts for the camera bump and other components.

In summary, the Pixel 8 is a sleek device with a slim profile, and its camera visor adds a touch of functionality without significantly increasing its overall thickness.

I second this. The quality of google search has reached a point that's only good to search things that can't be bought.

... and uncontroversial.

Searches for "coronavirus" seem to be hard-coded, or interfered with. I get pages and pages of Covid-19 results, but that's not what I searched for. I even get a wikipedia link to its covid-19 page, but no wikipedia link to its coronavirus page within the first several pages of results.

Something as basic as: "2,5 cm to mm" won't show up the unit conversion widget if it's not formatted as "2.5 cm to mm", at least for me. WolframAlpha also fails at this query. However, ChatGPT understands it and gives the right answer.

I use Bing, too. People are suprised when I recommend it, but for most general searches, it's quite a bit better.

In general, I find the same is true for all mainstream search engines, including DDG and Bing. You can't even search for things to buy anymore, ironically; it'll just dump you into Amazon or some "top 10" shitpost on CNN fake news or Forbes, much like the trash pit of websites shown in the header image of this article. Like others, I also find myself searching on Reddit or HN directly. What the point of a search engine is at that point, I don't know.

It's infuriating that any qualifier you use is ignored. "Lightest laptop" will just return random lists of 10 laptops. No mention of weight.

> Google search has become worthless for me. I use bing instead because of how horrible google results are.

Although Bing is generally OK at dealing with general queries, it's far, far worse at surfacing niche content, no? My non-commercial, hobby homepage fares reasonably OK on Google (although some queries are dominated by SEO spam). But on Bing, it ranks below a good number of spam websites, including ones that simply copied my content and serve it with ads...

I subscribe to Kagi, and it’s great.

Same. Done with Google search. In addition to the results having become useless, it's Google's frenetic sprint away from the "don't be evil" ethos, which turns out to have only been in Incognito mode all along.

Even when the results are not sponsored, they're all generic answers from generic domains. I'm traveling at the moment and it's impossible to find anything written by someone in a 1000 km of the city I'm visiting.

How to use public transit? What to see and do? How to get an airport transfer? Here are machine-generated answers from myairporttransfers.com and copywritten posts from cheapairteavels.com.

I am a small business owner who started their site and SEO and within three months I was beating multi-million dollar competition on the most important keyword google search terms for our market and industry. I did this with no budget, no adspend, just basic SEO and good keyword research. It's totally possible for mom and pop websites to get traction with google, even easy. You just have to be halfway decent at SEO.

Same. Numerous 1st page top results, and even snippets. Honestly no idea how... I do know SEO basics, but didn't know I knew them well enough for this. Within a year I had the first result for a very, very popular search term. Granted it was a lot of hard work (18 hour days, sometimes).

You spent 18 hours a day on SEO? Doing what??

No, no. I spent 18 hour days posting on social media, writing articles, designing cover images, researching, etc. I honestly didn't do much work in regards to SEO, at least I don't think so. (There were a few times I knew something was going to be released soon, so I wrote about it before anyone else did.) But I do believe all of this contributes to SEO.

But yeah, this brought my site to having every article I write today to be listed on page 1 (top 5 results, at the very least) almost within a day, numerous snippets, etc.

I wonder how easy this will continue to be in the chatgpt era. What takes you 18 hours can probably be generated in 18 seconds with enough fidelity to get traction.

Oh and doesn't that just define the problem so well

>> You spent 18 hours a day on SEO? Doing what??

> No, no. I spent 18 hour days posting on social media, writing articles,

Yes, yes. Based on my 10-years-ago contact with SEO, you just described it.

The only thing I did was everything google recommended I do, literally. Run PageSpeed, do everything suggested. Log in to search console, do everything recommended. Super hard. And the keywords I dominate are for such things like, jewelry repair, jewelry repair near me, jewelry repair Seattle. Not niche. The reason I am beating such big spending rivals is because they usually have sites that break a lot of SEO rules funny enough. They try so hard and pay so much to "SEO Experts" but their site is templated spaghetti and no one can fix it, seemingly.

Same here. I'm right up there competing with billion dollar companies with decades of presence and I don't know how many backlinks. While my small business has no backlinks from others and only relies on content for ranking, being entirely dependent on Google to be honest.

What's the site and what keywords are you targeting?

Parent poster is, I hope, wise enough to not answer.

There's no reason not to answer that question. It's a commercial site, it's already public. (I just don't believe them.)

If they are unaffected by big media sites jumping into their niche then it makes sense to stay under the radar.

I would guess you're working in a niche that has not been targeted by the big media sites alluded to here? Being halfway decent at SEO will come to nothing when you have 20 or so competing sites that can rank overnight for anything.

> It's totally possible for mom and pop websites to get traction with google, even easy. You just have to be halfway decent at SEO.

If this is really true, then you should be running your own SEO consulting business. You would undoubtedly make much more money than your existing (presumably not SEO consulting), small business

Comments like this are similar to people (ahem, Internet randos) talking about their investment portfolio returns, where they wildly exceed the very best professionally managed hedge funds. I always say: "If you are so good, why don't you run your own hedge fund? It will easy to get funding." <<crickets>>

Knowledge of my SEO prowess comes only recently. I have hung my shingle as a proven SEO expert so we will see. If anyone wants to hire me please let me know. I come with proven metrics.

The same issues you have with google search engine optimization are present in every other closed platform too. Welcome to the attention economy, you better learn how to go viral.

Reply on news.ycombinator.com