Comment by eGQjxkKF6fif | Hacker News Reader

Comment by eGQjxkKF6fif | Hacker News Reader

Parent Full thread

It's not fuck the bots, it's fuck the bot owners for using the websites as they want, and not at minimum, asking. Like 'hey cool if I use this tool to interact with your site for this and that reason?'

No, they just do it. So that can scrape data, which at this point in time for AI which has hit the cap on what it can consume knowledge wise, scrapes it because live updates and new information is most valuable to them.

So they will find tricky, evil ways to hammer resources that we as site operators own; even minimally to use site data to their profit, their success, their benefits while blatantly saying 'screw you' as they ignore robots.txt or pretend to be legitimate users.

There's a digital battle field going on. Clients are coming in as real users using IP lists like from https://infatica.io/

A writeup posted to HN about it

https://jan.wildeboer.net/2025/04/Web-is-Broken-Botnet-Part-...

A system and site operator has every right to build the tools they want to protect their systems, data, and have a user experience that benefits their audiences.

Your points are valid and make sense, but; it's not about that. It's about valuing authentic works, intellectual properties, and some dweeb that wants to steal it doesn't get to just run their bots against resources at others detriments, and their benefits.

> Like 'hey cool if I use this tool to interact with your site for this and that reason?'

They do ask: they make an HTTP request. How the server responds to that request is up to the owner. As in the article, the owner can decide to respond to that request however he likes.

I think that a big part of the issue is that software is not well-written. If you think about it, even the bots constantly requesting tarballs for git commits doesn’t have to destroy the experience of using the system for logged-in users. One can easily imagine software which prioritises handling requests for authorised users ahead of those for anonymous ones. One can easily image software which rejects incoming anonymous requests when it is saturated. But that’s hard to write, and our current networks, operating systems, languages and frameworks make that more difficult than it has to be.

Kind of, but they lie in the HTTP request - their user agent isn't true, they don't disclose they're a bot, they try to replicate other traffic as a disguise, they use many different IPs so they can't easily be blocked, etc.

It's kind of like me asking to borrow your car to go to work and then I take your car and ship it overseas.

oh yeah I was horrified recently starting up a "smart TV" and going through the installable apps to find a lot of repackaged youtube contents, even from creators I like, eg. a chess channel. The app just provides the same content as the youtube channel does but at the bottom of the long free-to-use license agreement there is a weirdly worded clause that says you grant the app the right to act as a proxy for partner traffic... So many smart TV users are unwittingly providing residential IP's for the app developer to rent out.

Yeah, it's a disgrace. 'bUt YoU AgReeD tO iT So I HaVe The RIGht To Do ThIS' it's just cyber warfare.

Plain and simple.

Sorry I had forgotten who it was. Now time to name and shame: the culprit calls itself https://brightdata.com/ Also LG relying on the developer's own disclosure for what they call "Data Sefety" is really poor: "There is no relevant information provided by the developer" is all the app reports in the LG app store... Also no way to rate or report the app, I only found a mention that I shouldreport this to lgappsreport@lge.com

So that there's a trace on the Internet on how seriously LG takes the social engineering going on with oblivious Smart-TV users to enable creation of residential-proxy armies, I did report the BrightData apps to LG at the right email address, and received this useless reply:

Apps Support 02:49 (9 hours ago) to me

Dear valued user,

Hello. This is your LG webOS "Report an App" Service Representative

Apps (LG Store, Below: Apps) is an open-market service that enhances the user experience in LG smart media products. Customers can download and use apps in a variety of categories, including education, entertainment, life, news, etc.

With Apps, customers can download the apps of their choice using the app recommendations, search, or browse features.

Apps that are available in Apps are registered after going through a verification according to LG's process.

And the registered app will be applied on the terms of conditions form the app supplier’s own policy.

If you have any further inquiries regarding the inconvenience, please contact app supplier Customer Support E-Mail

- sdk@brightdata.com

Thank you for using LG webOS, and we will continue to strive for better services for our customers.

Thank you.

LG webOS "Report an App" Service Representative

Reply on news.ycombinator.com