[disclaimer: I run https://pure.md, which helps websites shield from this traffic]
Why?
There's no value to the website for a bot scraping all of their content and then reselling it with no credit or payment to the original author.
What I don't see however is any way this new way of searching will give back. There is some handwaving argument about links, however the entire value prop of an llm is you DON'T need to go to the source content.
The LLM SEO game has only just begun. Things will only go downwards from here
The ball is in their court. You don’t get to demand civility AFTER being a dick. You apologize and HOPE you’re forgiven.
What for? Why would I serve anything to these leeches?
I fell for a class "To Serve Man" situation.
And what if there was some standard sort of way for robots to tell your site what they're trying to do with some sort of verb like GET, PUT, POST, DELETE etc. They could even use a standard way to name the resource they're trying to interact with. Like a universal resource finder of some kind. You could even use identifiers to be specific! Like /items/ gives you a list of items and /items/1.json gives you data about a specific item.
That would be so awesome. The future is amazing.