Hacker News Reader

Comment by vector_spaces | Hacker News Reader

You are assuming that because you are giving the operator money that you are entitled to use their website however you please -- that's not how it works. If you enter a brick & mortar establishment and break their rules, even as a paying customer, you might risk e.g. being kicked out and banned, depending on the behavior. This is not unique to e-commerce

To be fair, I do not generally support wholesale banning of scrapers and non-browser clients, since this tends to have lots of ugly side effects.

But regardless, I'm interpreting your last comment as saying that you should be entitled to behave as you please because you are a paying customer. If that's the case, then you are in for a rude awakening one day

View on HN

I think he's saying that his use of the site, via the LLM, is consistent with what the site owner-operator likely desires.

To give a concrete example: I am a physician and have a site for my private clinic. I am not only unopposed to being scraped by bots/LLMs, I prefer it. The only value I gain from users being on my site is referral to my practice, and that same information being surfaced by search engines and LLMs just means I'm getting in front of more potential patients.

I think the implicit assumption is that if you explained to the owner of the website what the scraper was doing, they would be more than happy to allow it.

The bot was also being generally courteous, only scraping the 2-3 pages needed to get the information for the human who spun up the LLM.

I don't think GP was at all suggesting he should be able to act however he wants, just because he's a paying customer.

I run an ecommerce site and I'm actually more worried about bots being rejected than accepted because of stuff like GP. We get a decent amount of business through ChatGPT.

That's an incredibly dishonest characterization of what I wrote. Please re-read the guidelines of the site, they say exactly not to do that.

> saying that you should be entitled to behave as you please because you are a paying customer

No, I'm saying my usage of the site was exactly as if I used my browser agent instead of the llm agent. It visited the category for the product I was interested it, then it visited the pages for the various items to read their specs. Exactly what I would do in my browser as well. In the end I bought one of the products. Point is that I used the site as desired. No extra calls / mass scraping, no stealing of content without them getting anything in return, no breaking of rules.

If we want to be pedantic, we could say that your automation is like a particularly aggressive DoorDash driver; runs into the store, grabs all items as fast as possible, and is perhaps rude to the checkout girl on the way out. And if you scale this to checking N vendors - and only buying from one - you fall right into the bad behavior.

I’m not sure if your bot is like this, but my understanding is that many of them are - and they interfere with the other shoppers and generally change the economics of it.

I think you are on the right side of this, but you are almost indistinguishable from the negative case.

The site guidelines also say to assume good faith, to converse curiously, edit out swipes, not go fulminate, and to avoid grandstanding.

Before calling me dishonest, did you stop to consider that maybe you didn't understand my comment, or that you weren't clear with your meaning?

I genuinely don't think you understood what I wrote.

If I operate a business, you are not necessarily entitled to transact with me how you please. I can implement whatever rules I want so long as I am not outright committing discrimination against a legally protected category and so long as I am upholding other fiduciary obligations.

For all you know, the site operator is opposed to AI agents on moral grounds, or has beef with the developer of the specific LLM interface you are using.

More practically, an e-commerce website often won't make a profit if a purchase is too small. If they find that LLM users are easily bypassing their upselling tactics that usually result in higher basket sizes, then ideally they would adapt their approach to LLMs or implement a minimum order size, but they are also perfectly within their rights to simply ban you.

I am not saying that this is the likely reason -- just that there are reasons, many, and often they aren't very obvious unless you are in their shoes.

Every kind of business does this sort of thing! A software consultant or designer might only take money from healthcare companies, and not talk to anyone offering (apparently, to a naive outsider) equivalent work in a different industry. A restaurant might implement a dress code. A cab driver might insist you not wear strong perfume.

Some rules that externally seem flippant and unreasonable exist for reasons that are entirely sensible, but also entirely nonobvious externally. In any event, I don't agree that your usage is equivalent to a manual session, if it were, the operator wouldn't be able to detect it! Therefore they are within their rights to detect this and disallow it. There are plenty of reasons that they might do this, and unfortunately you probably don't have the information needed to interrogate those in a meaningful way

Reply on news.ycombinator.com