Hacker News Reader

> then I would point out that it’s very unwise to build a business whose success is based entirely on the whims of another business.

When that other business is a monopoly, what choice do you have? The rise of google has effectively killed all other sources of traffic. Web sites used to get their traffic from things like webrings, directories and a variety of smaller search engines. Now? Google, or one of its properties, and to a smaller degree Meta are basically it. The curator and community discovery model is a victim of google, not a solution to it.

Building a business that relies solely on Web traffic is iffy for reasons unrelated to Google. Supply far, far exceeds demand, to the point that you have to pay to get people to visit your website. Which really explains a lot about the current Web ecosystem.

Every single business currently depends on one platform or another. Google Search, Google Maps, social media or Booking.com have become gatekeepers for entire industries.

You could say the same about any business that requires some of a person's time and attention. Or some of a person's money.

Reddit, Instagram, Facebook, Tik Tok, even YouTube (Google property but not Google the search engine) are all paths to discovery. To a lesser degree sites like HN. There's also nothing actually stopping anyone from going to Bing or something. My Chrome browser takes me to Bing if I type in it or change the search engine...

Also de-ranking SEO spammers is a net positive IMO.

They are paths to discovery, but are they actual drivers of traffic?

My impression is that if you aren’t listed on google, your site is pretty close to dead. In any case, if you somehow managed to drive traffic via other social media, you are one underpaid moderator’s hasty decision from ruin.

The point isn’t about deranking spammers it is about concentrating a huge majority of traffic (and therefore power) in the hands of a single entity which is unaccountable to anyone.

I think traffic/discovery patterns are a really good distinction. Something like Hacker News, Reddit etc will drive a lot of traffic in a short burst. People are sharing "cool stuff" and visiting it in one go.

Ideally, we need a fix as well for more regular discory of problems people have. If I want "Norwegian cooking blogs" google will probably return me mostly spam, and waiting for someone on reddit to maybe post something in that category obviously isn't an option.

Oddly enough using google to search “site:Reddit.com Norwegian cooking blogs” is actually a great strategy.

Another thing to consider is that those sites would rather you did you click those links. They don't benefit when you leave the platform and they down rank things they think will cause users to leave the walled garden regardless of the content.

There is nothing stopping people from going to Bing sure but I think you are forgetting that most people now a days interact with the internet through a mobile device.

Typing w w w . b i n g . c o m on a mobile device is are a huge barrier to entry vs just using the google search box Google pays the phone manufactures to bake into the the OS.

There is absolutely something stopping me from going to Bing and I wager it's the same reason with most other people:

Despite how awful Google has become in recent years...Bing is still consistently worse.

> There is nothing stopping people from going to Bing sure but I think you are forgetting that most people now a days interact with the internet through a mobile device.

If anything I'd say mobile demonstrates that Google doesn't have a monopoly... TikTok literally occupies a majority of gen Z's app usage... 4 of the most used mobile apps (Instagram, WhatsApp, Messenger and Facebook) all belong to Meta. My wife doesn't even know which browser is on her phone (Samsung Internet)...

Wait: which manufacturer's OS doesn't let the user switch the search engine to Bing in Settings?

They totally let you. How many people do you think go in and change the default? Otherwise paying to make Google the default would not have any value.

> Reddit, Instagram, Facebook, Tik Tok, even YouTube (Google property but not Google the search engine) are all paths to discovery.

Add blogs, News sites, and forums - to that as well.

In a sentence, the theory is that as search added optimization layer based on ML, after optimization layer based on ML, it's nigh-debuggable.

Put another way, Google launched 1000 experiments that got +0.2% CTR and seemed innocuous, and now they have a system that only wants to give out Reddit and Quora links.

I've seen this story on Google's private Blind section multiple times. Usually, coupled to discussion about a more cautionary approach taken by the pre-2020 head of search, who was worried about this outcome.

IMHO they're feeling out the same outcome without the benefit of the above knowledge. I don't see this theory well-understood outside Googlers, modulo a pair of popular blog posts, whose name I can't recall, that hit the nail on the head.

(disclaimer: xoogler as of oct 2023, didn't work on search)

I agree. But it's a tough problem, isn't it?

I don't just mean filtering out the spam; that's a hard problem on its own. But the good-click metrics... if serving up reddit and quora links at #1 makes more people satisfied with the first result, defining satisfied as 'reads the #1 page and doesn't go back for more', then...

What's wrong with that metric?

That isn't a rhetorical question. It's tempting to claim the users are wrong to prefer it, but not very nice. Why do they do so?

> if serving up reddit and quora links at #1 makes more people satisfied with the first result.

It does not, what google is optimizing for is ad click-through not organic results click-through. The whole model is you will search for X see a couple of ads and a sea of bad results and opt to click on the ad.

It's the same reason that if you search for "graphics card" on amazon and sort by price you will get 20 pages of "Graphics card Holders" and "Graphics Card Elbows". They want you to not actually use the search just click on their ad.

This is a good explanation of the method behind the madness. The manipulated results are for steering people to what they want to serve them. Clicking on their ads or corporate preferences.

The problem appears to be it's hard for them to realize that people are getting fed up with the antics and are seeing through the manipulations. It's like an abuser or bully, that thinks they will keep getting away with the poor treatment of others, who they think are "below" them. At some point, enough people can take a stand, and not put up with the lack of transparency and black-box manipulations. They are going to keep demanding change or find alternatives.

Users aren't wrong (tangentially, I strongly believe they don't like it)

Bravely flattening a root cause analysis to one agreeable-sounding villain, I'd do Goodhart's Law. Click-through rate not being a great proxy for user satisfaction.

i.e. I always clicked reddit first, I used to append reddit to my searches. But I get a very uneasy feeling now that everything has reddit 3 times in top ten. Impossible. It's a big world. It'd be like if Encyclopedia Britannica started just including the snippet-sized version of Wikipedia articles. Not what I'm looking for from that product.

You have it backwards. People started adding reddit because the default google was serving up has gotten so bad. If the default was good, I wouldn't need to add reddit or quora.

> I think that the right model for content discovery is either crowd sourced by a like-minded community, like hacker news or curated;

I'm curious how you'd scale that. Google gets 8.5 billion searches per day. 8.5 billion "Ask HN: Where's the best resource for X" isn't going to scale to that.

> Where's the best resource for X"

Reddit is a pretty credible alternative for that.

That is not my experience. The mods there close any question that's been asked before even if the previous one is out of date and is closed to new answers. They also just seem to close most questions period.

Are you sure you're not thinking about Stack Overflow? I hate defending reddit ever since the API debacle, but there are scads of subreddits, many with different sets of mods, and all of the ones that help me with product reviews don't seem to have aggressive moderation?

Edit: for example, I was searching for woodworking design software this morning and got a lot of helpful-looking results from reddit.

It was a while ago. I would presume many posts and replies to posts are by marketers and/or LLM bots, especially for posts asking for product/service recommendations.

I may be wrong, but I don’t think op is responsible for scaling their suggestion up to serve everybody on the internet.

Then it's a model for discovery of a tiny sliver of content :shrug:

Google is now moving towards the position AltaVista used to have: everybody is using this search engine because of its breadth of coverage, the users are often unhappy with having to search for a long time and adjust the query many times, and complain about the inefficiency of it.

I hope Kagi will grow and attain a large enough market share to afford indexing more of the web. Maybe we'll have a really useful search engine again, like it's 2001. For a few more years at least :-/

Speaking of which, I would love a curated Android app store. It says a lot that you still can't filter by ad-free apps in it.

F-droid and Aurora

F-droid is good, but where's the curation part?

What type of curation are you looking for? F-droid curates for open source and "anti-features" which are documented on each app page. And Aurora allows you to search Google Play while filtering out apps that utilize ads. What exactly do you want?

It's curated. It clearly marks "anti-features" of the apps.

You can help create one, since unlike iOS, Android doesn't have an app market monopoly. There are already many alternative app stores.

App Finder in the Play Store will filter by ad-free and a bunch of other parameters.

> I think that the right model for content discovery is either crowd sourced by a like-minded community, like hacker news or curated

This is how all the subreddits that are remotely related to politics or news got so biased.

Of course I don't know what the alternative is. If I did I would be making that ideal site instead of scrolling HN.

Also the article did a terrible job by confusing LLM and AI query results with Google’s not-LLM search engine.

And yes, like you alluded to, search engines aren’t the only way to acquire your readership or regular customers. Some media outlets basically don’t even use search engines to drive traffic, like so many social media based businesses out there. There are hundreds of thousands of businesses that barely have a linktree page.

"I think that the right model for content discovery is either crowd sourced by a like-minded community" and now you've re-created the original Yahoo search, which was killed off in the original Search Engine Wars by the likes of AltaVista and later Google because it didn't take long for the amount of new content on the web to overwhelm the manual curation process.

A manually curated system can only truly work with a constrained domain of content/subject matter and tolerance for high lag regarding new information. Wikipedia is an example of what that kind of system would look like.

> it is slowly getting planted by LLMs and also for years has been gamified by SEO.

I think that is basically what they are complaining about, from a content creator's perspective. They are upset that their quality content (according to them at least) is losing to ML generated garbage, and that google engineers don't even seem to be able to understand why that happens.

Maybe the Google Search engineers should ask the team next door, that has been receiving the lion's share of Google's funding over the last 3 years years, with the express goal of doing exactly that?

My take on what the author was getting at, is how Google's manipulation of its algorithm has led to poor results and backfired on them. Not just for the users of its search engine, but small businesses counting on being discovered that way.

Arguably, the problem is black-box algorithms and how they are manipulated. The same problem can be faced by any curated community, where there is little to no transparency. Curated can still mean manipulated. People can jump from place to place, but that may not be advantageous, because where they want to be or the most likely places for discovery are not available.

> it’s very unwise to build a business whose success is based entirely on the whims of another business.

Show me a business free of dependencies and I'll eat my hat.

Bad argument. Having all of ones business dependencies to be that from a single business, subject to its whims, is unwise regardless of the fact that no business can exist without dependencies at all.

Yes, full redundancy in one's entire supply chain, corporate structure, business services, infra, and finances is both efficient and feasible. 9_9

Everything exists in a system. I suppose the least (and therefore most resistant) to dependency related issues is small family farmers that run things like U Pick farms and grow a diversity of crops

I think you should read the blogpost again. Google wants these smaller websites to come into search results but it can't coax the The Algorithm to do it. That's why they are actively trying to debug these issues.

> I think search in general is becoming a very poor way to discover content as it is slowly getting planted by LLMs and also for years has been gamified by SEO.

Did you mean supplanted here?

It's a problem that rankings changed so dramatically and no-one really understands why.

Perhaps the ranking was terrible before and now is better. Or is it the other way around?

The capability of A/B testing should have made this experimentation a one-armed bandit problem, which is far more ammenable.