Hacker News Reader

Hacker News

Comment by extr | Hacker News Reader

Full thread

extr

I can believe this. A lot of my google search usage now is something like:

> "what is the type of wrench called for getting up into tight spaces"

> AI search gives me an overview of wrench types (I was looking for "basin wrench")

> new search "basin wrench amazon"

> new search "basin wrench lowes"

> maps.google.com "lowes"

Notably, the information I was looking for was general knowledge. The only people "losing out" here are people running SEO-spammish websites that themselves (at this point) are basically hosting LLM-generated answers for me to find. These websites don't really need to exist now. I'm happy to funnel 100% of my traffic to websites that are representing real companies offering real services/info (ship me a wrench, sell me a wrench, show me a video on how to use the wrench, etc).

View on HN

> The only people "losing out" here are people running SEO-spammish websites that themselves (at this point) are basically hosting webpages containing LLM-generated answers for me to find.

Agreed. The web will be better off for everyone if these sites die out. Google is what brought these into existence in the first place, so I find it funny Google is now going to be one of the ones helping to kill them. Almost like they accidentally realized SEO got out of control so they have to fix their mistake.

At one point these SEO pages were in fact providing a real service, and you could view them as a sort of "manual", prototypical, distributed form of AI. Millions of people trying to understand what information was valuable and host webpages to satisfy the demand for that info, and get rewarded for doing so. It obviously went too far, but at one point, it did make sense to allow these websites to proliferate. I know without AI, I probably just would have clicked on the first link that said "types of wrenches" and read a little bit. I probably would have gotten my answer, it just wouldn't have been quite as laser-targeted to my exact question.

True, the early days these sites were genuinely helpful. The monetization model was a little different though which is what I think kept them useful. You'd use the content just to drive traffic, which would result in ad clicks on your banner ads, etc.

Then "content marketing" took over, and the content itself was now also used to sell a product or service, sort of an early form of influencer marketing and that is when I think it all started to go down hill. We stopped seeing the more in depth content which actually taught something, and more surface level keywords that were just used to drive you to their product/service.

OTOH, the early web was also full of niche forums, most viewable without an account and indexable, of about any topic you could imagine where you could interact with knowledgeable folks in that niche. Google would have been more helpful to users by surfacing more of those forums vs. the blogs.

Those forums are IMO the real loss here. Communities have moved into discord, or another closed platform that doesn't appear on the web, and many that require accounts or even invitations to just view read only.

"Live by the SEORD, die by the SEORD"

Or was it, "Live by the AdWord..."

Hard disagree. I put a great deal of work into my website, putting hard-earned information on the internet for the first time. Now Google reaps all the value I create without as much as a "thank you".

Unfortunately the new victims in the system of LLMs and the way they distribute knowledge is the specialized content website.

Now an LLM just knows all the content you painstakingly gathered on your site. (It could also be, and is likely that it was also collected from other hard to find sites across the internet).

The original web killed off the value of a certain kind of knowledge (encyclopedias, etc.) and LLMs will do the same.

There are plenty of places to place the blame, but this is a function of any LLM, and a funcamental way LLMs work, not just a problem created and profited from by Google- for example the open-weight models, where no-one is actually profiting directly.

> There are plenty of places to place the blame, but this is a function of any LLM, and a funcamental way LLMs work, not just a problem created and profited from by Google- for example the open-weight models, where no-one is actually profiting directly.

First time learning that scraping and training on data that they have often been explicitly disallowed to obtain for free or for that purpose by the rights holders is "fundamental to how LLMs work". If not, then there is no reason those who gathered the information wouldn't stand to profit by selling LLMs this data.

We killed forums and now were killing specialty websites. Soon we will get right back to pre internet levels of knowledge. Ai will be able to find all the stuff that was easily available in encyclopedias and text books and none of the stuff that made 90s-2000 internet great.

I wonder how long it will take AI agents to automatically make accounts and submit credit card info to scrape data that is paywalled

I don't think there is any reason they couldn't do that

The Internet had plenty of specialty knowledge before "Gathering that knowledge, SEOing it, and stuffing ads all over it" became a viable business model, and it will have plenty of specialty knowledge after this era is over.

Specialty knowledge can also be a guy reviewing air purifiers with obsessive care. It can be someone writing detailed travel guides or in my case an essential resource for immigrants. There were a lot of good websites, and if you could not find them, blame Google for letting its search engine go to the dogs.

Gathering that knowledge is work, and if anyone should capture that value, it's the people doing the work. Seeing bug tech slurp it all up, insert inself in the middle and capture all the value is heartbreaking.

I hate that AI is destroying the economics of the independent web, and people cheer for it because they landed on Forbes one too many times. It's insulting to all the people who did their job right, and still get their work slurped up by an uncaring corporation with no way to stop them.

We will get the web that we deserve.

That was already dying if not dead thanks to YouTube and short form video being the preferred way for people to make content and I guess consume

>The original web killed off the value of a certain kind of knowledge (encyclopedias, etc.) and LLMs will do the same.

I wonder how many people will decide to just stop sharing technical knowledge because of that, and how much we will lose because of it.

Maybe (I can't emphasize enough that I say this with a high degree of skepticism) they'll come up with a way of doing cited/credited/paid work where the LLM will include who "taught" it something.

Eg: instead of writing a blog post, you'll submit a knowledge article to an AI provider and that'll go into the AI's training set and it'll know "you" told it. And maybe (even more skeptical) pay you for it.

Again: highest degree of skepticism, but at the same time, that's the only way I could see people continuing to write content that teaches anything.

This will only happen as an absolute necessity, for example if the EU threatens massive fines or the US antitrust people grow some teeth.

Even then, you'll get malicious compliance. The best case scenario would be a bit like Spotify: everyone getting fractions of a penny.

Agreed. I'm particularly pessimistic about how this will play out.

One problem with monopolies is their massive multiplicative effects on otherwise manageable problems.

You put your information on a public website for free, and are surprised that people use it as they see fit?

Don't be surprised if there is no more free information in the next 10 years.

There will always be people putting information out there with 0 expectation of any kind of return. What will happen is there will be far fewer websites enforcing return, as they'll be automatically deranked for that enforcement.

I am tech blogging for free, and my expected return is people reading my content, recognition, attribution, social media attention and people discussing my content.

I have zero interest in providing free labor for LLM companies with no human actually reading my words. I don't think I'm alone in that stance.

Similarly, I used to help people on forums for free. My reward was getting respect of my peers, the feeling of helping another human being and sometimes them being grateful, rare side job opportunities thanks to people finding my specialist posts. That was fun, being anonymous question-answering bot for AI to scrape is not.

> recognition, attribution, social media attention

Expectation of these in particular makes your blogging a product.

> with no human actually reading my words

But it - or at least the idea - is being ultimately read by humans, as long as some article is in the top results for relevance to some LLM prompts. It just may be summarized, or particular parts extracted first, or reworded for clarity (like I may ask for an "eli5" if I encounter something that's interesting but I find concepts going over my head), or otherwise preprocessed in order to fulfill the prompt parameters. All actions which the very human users may have to do manually anyway in order to efficiently consume your content (or give up and move on if they can't be bothered), is now automatically done by LLM agents.

The number of people who have the privilege to publish content for free for others (or spend their free time contributing to open source) is minimal. And now they will feel like they're being disrespected and exploited by LLM's and its users who think of the authors not as people but as just as the source of content. You can be sure most of the content will be gone because nobody wants to be just a content cow for LLM's.

Why do you think they'll feel disrespected? They literally have no expectations; they're just putting content out there for others to do as they wish with. Several here have weighed in to say they don't care about LLMs accessing and morphing their content.

Of course they won't be nearly as many as those who publish with expectations, but also history has shown that whenever there's a gap, someone tends to fill it just because. I'm not worried about content gaps at all. What I see is an overall increase in quality as no-expectation sources float to the top of search results.

Well I'm one such person so you can hear it from me instead of guessing. I feel disrespected.

I can also confirm that I will do a lot less of it since it's threatening the parts of my business that supported me and gave me so much free time to release things for free. It almost halved my audience, so I have to do the same amount of work for nearly half the pay, half the community, and half the credit.

You do have expectations though. You expect to grow and maintain an audience, gain credit, which makes what you're releasing a product, not something free.

Free as in beer, not as in public domain.

>whenever there's a gap, someone tends to fill it just because

lol what. That only works when the demand is actually paying. You are talking about FREE content!

>They literally have no expectations

I'm sorry but this is completely false.

If that were the case why do we have so many licenses? Why GPL/MIT/Apache, CC BY, CC BY-NC, CC BY-ND, CC BY-SA, CC BY-NC-SA, CC BY-NC-ND, and so on, when it's either "free or not"? Surely we don't need all this fluff when the only thing free for real is Public Domain?

Just consider this: we're living in an age where even people who publish MEMES on Reddit watermark them because they don't 9gag/Instagram/Facebook pages reposting it without permission/credit. And they are MEMES!! Even I find this cringe. But it proves that the author has some expectation. Even if you don't agree with their expectation, it proves that the expectation EXISTS.

What is next? Are you going to extend this to say that all web comics accessible for free on the web are "free free" so you should be allowed to remove watermarks to repost them on Facebook? You are filling the gap of having a single place where people can read funny comics for free, except you didn't make any of the comics and you have no right to post them. In fact, this is a great example. How is ChatGPT different from a guy that just reposts comics and memes on Facebook? It's literally the same thing.

And then next you are going to say that all videos posted on Youtube/TikTok are "free free" so you should be allowed to rip them off too.

I feel like you're just going to make an enemy out of everyone who publishes anything for free on the Internet if you start thinking like this.

> That only works when the demand is actually paying

You should see the internet 20+ years ago. It was rich in forums, interest sites, etc where people just shared because they had this interesting thing they wanted to put out there. Reddit still has a bit of it, though it's mostly a mess now.

> why do we have so many licenses?

Because the way how the law works in some jurisdictions is that permission must be explicitly granted by the author via license, and some authors (who are aware of the legal requirement) just do up a thing without checking to see if there's something that fits their desires. And some want to tweak a license in some way to account for some other thing and end up creating a new license. See also CC0, 0BSD, WTFPL, Unlicense, and others [0].

BTW I'll clarify that it's not "expectations", but specifically "expectations of return". It's OK, and expected, for instance, that someone putting something out there for free wouldn't want to be held legally responsible if that thing is used in an illegal manner.

[0] https://en.m.wikipedia.org/wiki/Public_domain_equivalent_lic...

Not always creators puts the content for free free, they expect views, gratitude, and/or ad revenue..

It's not free if there's any kind of expected return.

If the return is also free, yes. A smile or a thank you is free to give.

Attribution would be one thing. See ooen-source and free software licenses.

There are specific constraints on "free" in that context. For example *GPL also generally forbids providing functionality without making the source available.

If I gave my surplus eggs for free, I'd be annoyed if they ended up at the Best Western's buffet. These eggs were meant for my neighbours.

There is a difference between working for free for your community and working for free for a trillion dollar company's investors. Doubly so when you strip consent or attribution from the equation.

Then you should make sure you are giving the eggs to your neighbors and not to Best Western.

I can't help if the Best Western plunders the honesty box in front of my house. There is no way to stop them.

I hope that you understand that this is a metaphor for free labour intended for a community being exploited by big corporations with no way of stopping them.

Just because something is on a public website accessible for free doesn't mean it is then public domain. Sharing is not necessarily giving.

(Though unfortunately in the Wild West Web of today it seems it does, practically speaking.)

Unfortunately, existence is a thankless job. Just gotta enjoy the ride.

> The only people "losing out" here are people running SEO-spammish websites that themselves (at this point) are basically hosting LLM-generated answers for me to find.

And anybody who creates original content and wishes -- not just to be paid for that content -- but for people to actually see that content and engage with it. IOW the very people who fed the LLM revolution.

No one is using AI search to read Tammy’s original content unless it is terribly written, and no one is reading the 4 page fluff peice about the process of troubleshoot XY just to find the command that fixes their problem when they just want to solve the problem.

People who want to engage with original content will; people just won’t be forced into appearing they are engaging with content just to find answers to simple questions or find the specific information they are looking for before they leave the site…like always.

The result is likely to be more time on site with lower numbers of users; a more genuine reflection of an actual user base instead of search-fed propping of ad revenue numbers through effectively fake impressions.

To that point: There is not a single website or blog, ever, that I started visiting regularly by having ended up there from a search result. Literally, ever. From something a friend shared? Something I saw on HN? Something I found through a recommendation, an article that was posted on a different site, etc? Absolutely.

That's ... not what original content makers who aren't named Tammy are saying, including CNN and The Verge [1], the Daily Mail [2], and several lawsuits [3] [4]. In fact, the only people who seem to believe that AI search isn't negatively affecting content makers long-term is google and you, and google (for now) has a vested interest in believing that.

Discovery isn't search, but search can be a form of discovery, despite your experiences. They don't match mine.

[1] https://www.npr.org/2025/07/31/nx-s1-5484118/google-ai-overv...

[2] https://digiday.com/media/the-winners-and-losers-of-googles-...

[3] https://www.reuters.com/legal/litigation/googles-ai-overview...

[4] https://www.reuters.com/legal/googles-ai-previews-erode-inte...

Your excessive hyperbole aside, I’m sure they are saying such things. Unless you forgot, they also said that about Twitter…and Facebook…and Google Search…and before that it was magazines, and BBS systems, and the internet, and Craigslist, and…and…and…

You confuse original content creators with news conglomerates that always cry wolf about how they will be “put out of businesses”. Of course, ignoring the fact that there have never been more paid content creators in history, you choose to only be concerned about the one type who complains most frequently.

It’s really nothing new, but hey, benefit of the doubt, maybe it’s your first time.

Using the term "excessive hyperbole" is itself hyperbole and is a little unrealistic for what we're talking about. What you might be referring to is the excessive number of sources I used and honestly you could take a page from that to back up what you're saying.

My original comment regarded people who create original content and want to be paid for it. That includes substack creators, CNN, and lots of enterprises in between. They all have the same problem with large LLMs either taking advantage of the tragedy of the commons or ignoring their robots.txt files and scraping their content even if they choose to not participate.

I haven't forgotten that news organizations said the same thing about Twitter, Facebook, etc. If you haven't noticed, news (especially local news) has been declining steadily for at least the last 25 years and several news organizations (again, especially local ones) _have_ either gone out of business or been bought out and gutted by hedge funds. Some of this for sure is due to miscalculations by those orgs, but the nature of those miscalculations matters. It's worth reading up on the history of media's financial relationship with social and search. It will help inform a lot on how it's going to go with LLMs and AI unless they find a way to make some deals. It behooves both sides.

Alternatively

Wikipedia as search engine, no Javascript, no "AI"

Query string "wrench tight spaces"

Basin_wrench is the #1 result

usage: sh 1.sh wrench tight spaces > 1.htm;firefox ./1.htm

   #!/bin/sh
   ns=$(
   for x in \
   0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 \
   100 101 \
   118 119 \
   710 711 \
   828 829 \
   2300 2301 2302 2303
   do
   printf \&ns$x=1
   done
   )
   export x=https://en.wikipedia.org
   echo "url=$x/w/index.php?search=$@$ns&fulltext=1&offset=0" \
   |(echo user-agent=\"\" ;
     echo header Accept:;
     tr \\40 +) \
   |curl -K/dev/stdin \
   |(echo "<base href=$x />";cat)

That only works for some queries, particularly simple ones. Even elaborating the given query into “type of wrench for tight spaces” puts “basin wrench” in 6th place below incorrect wrenches like “oil filter” and “hydraulic torque” (https://en.m.wikipedia.org/w/index.php?search=type+of+wrench...).

That's not the same search URL used in the comment

The "ns" parameters are significant

If use the URL in the comment with this query string then Basin_wrench is #3 result

Monkey_wrench is #1 and Wrench is #2

"type of wrench for tight spaces"

Basin_wrench is #3 result

Delete preposition: "of"

"type wrench for tight spaces"

Basin_wrench is #2 result

Delete preposition: "for"

"type of wrench tight spaces"

Basin_wrench is #2 result

Delete prepositions: "of", "for"

"type wrench tight spaces"

Basin_wrench is #2 result

Delete unnecessary noun "type" and preposition "of"

"wrench for tight spaces"

Basin_wrench is #1 result

Delete preposition "for"

"wrench tight spaces"

Basin_wrench is #1 result

"tight spaces wrench"

Basin_wrench is #1 result

"tight wrench spaces"

Basin_wrench is #1 result

Much less typing, no unnecessary nouns and prepositions

I like it, others might not

I also tried a number of less popular search engines with a non-AI search from the command line

Query string "type of wrench for tight spaces"

For several of them the #1 result was for star-plumbing.com

https://star-plumbing.com/what-kind-of-wrench-is-used-in-tig...

The LLM probably wouldn’t know the answer to your initial question without the SEO spam sites that they were trained on.

I don’t know if old school search engines are too ingrained in me but my search would look something like: reduced space “wrench” -site:www.amazon.com -site:ww.ebay.com

Also have the same experience with regular Google search now (and other engines, too). I think part of the problem is the degradation of Google's product, but also the endless amount of absolute shit that humanity is vomiting onto the Internet now. The commercialisation of (almost) the entire Internet during the mid 2000s has wrecked it all.

Now AI searches that: search, pull various pages to examine real contents, continue searching etc, then summarise/answer is realistically the only way to filter through all of said bullshit.

AI searches help with the clickbait problem as well, since even "reputable" news outlets are engaging in that fuckery now.

It's either; we use AI to sift through the dead carcass of our old friend, or we enforce rules for a clean Internet - which I can definitely not see happening.

Reply on news.ycombinator.com