> the bot explained that it had manipulated the system to book him onto classes months in advance - against the normal rules of the system.
> The agent replied saying it had succeeded by cancelling another gym-goer's booking.
This wasn't just breaking the rules of the gym, it also caused damages to someone. Someone had their booking canceled, maybe lost some money, definitely lost time dealing with this topic.
I disagree. If an endpoint is open, my assumption is that it's free for me to call, the same way I expect to be able to purchase items on the shelf at the store. If you fail to clearly label or separate them, that's on you.
The famous "innocent as long as their assumption was" defense. "If the window is open, my assumption is" works the same way.
The law is clear on this, no need to make assumptions. The guy knew immediately the agent broke the rules. A modicum of common sense tells you that those actions can't possibly be above board. The gym will probably let him off the hook and not press charges but not because he was entitled to do it.
> the same way I expect to be able to purchase items on the shelf at the store
I'll try to make my point using the same logic. Why "purchase"? Each item isn't clearly labeled as "must be purchased before exiting the store". And why just from the shelf? Why not the shelf itself? Why not "items from a desk drawer in a back room of the store", or even "items from the pocket of a cashier"? They're all right there, unlocked, and unlabeled.
> If you fail to clearly label
Do you carry a visible label with things people are not allowed to do to you? Without that label do others often make the same assumption you do? :)
We have laws, and LLMs should NEVER break them unless the user states its fine with some qualifying condition. Yes, every country has a different legal system, but I think there's a decent idea of what constitutes intrusion thats agreed on in most parts of the world.
In grey area scenarios, the user should be able to override this, with a warning of clear consequences of, and should they accept, users should be held criminally liable.
If the LLM does so unprompted, the responsibility should be the providers'.
LLMs should be considered tools, legally speaking.