Hacker News Reader

Comment by troupo | Hacker News Reader

What amazes me that none of this is surprising, all this behavior (not just what's described in the post) is on par with what the companies are doing, and have been doing for decades... And yet there will be many people, including here on HN, who will just cheer these companies on because they spit out an "opensource model" or a 10-dollars-a-month subscription

View on HN

The situation didn't change when it was search index crawlers being called-out. At the end of the day, this sort of "abuse" is native to the world of the internet; like you said, it's decades old at this point.

HN will cheer on a lot of things that are counter-intuitive to their wellbeing; open-weight models doesn't feel like one of them. You can't protest AI (or search engines) because after long enough people can't do their job without them. The correct course of action is to name-and-shame, not write pithy engineering blogs begging people to stop. People won't stop.

> HN will cheer on a lot of things that are counter-intuitive to their wellbeing; open-weight models doesn't feel like one of them.

Except for the fact that they come from undisclosed sources from a company that does this: https://x.com/Tantacrul/status/1794863603964891567

And I'll be damned if Tanta isn't having his tweets used for training Elon's AI. The parochial circle regresses as it goes around, I'm done acknowledging the make-believe barriers we pretended the internet clung to.

You post it, others consume it. Same as it ever was.

Do you feel the same way about Google spidering for their commercial search engine?

Google sent traffic to your website. You were at least getting something in return.

Emphasis on were, since they've made their search such utter shit in the quest for ad revenue that they're now going to have their own AI sum up your results (badly) instead to attempt to solve the problem they created.

Yeah, and when Google stopped sending traffic, people sued them, as they should.

I don't.

Just 3 AI spiders put more load on our servers than all search engine spiders and all human traffic combined.

Some numbers I have handy from before I blocked the bots:

ClaudeBot drove more requests through our Redmine in a month than it saw in the combined 5 years prior to ClaudeBot.

Bytespider accounted for 59% of the total traffic to our Git server.

Amazonbot accounted for 21% of the total traffic to our Git server.

Google has never even been close to breaking out of the single-digit-percentages of any metric.

Generally Googlebot is well behaved and efficient these days, though I have discovered that it is currently horribly broken around 429 / 503 response codes... And pays no attention to Retry-After either... Also Google-Podcast which is meant to have been turned off!

Someone needs to start adding all these AI's homepages to the browser "malware" lists.

Not originally since they just sent people to the site.

But these days where they just rip content from the site to give people as answers, completely depriving the site of traffic, yeah that seems basically just as bad as the AI bots.

Exploitation of the common man being a key ingredient of a product has never once inspired any actual consumer revolt. Fundamentally, no matter what they say, as long as they get their fleeting hit of dopamine from buying/using a thing, people just don't care.

As Squidward says: nobody gives a care for the fate of labor as long as they get their instant gratifications.

Reply on news.ycombinator.com