It's the frictionless aspect of it. It requires basically no user effort to do some serious harassment. I would say there's some spectrum of effort that impacts who is liable along with a cost/benefit analysis of some safe guards. If users were required to give paragraph long jailbreaks to achieve this and xAI had implemented ML filters, then I think there could be a more reasonable case that xAI wasn't being completely negligent here. Instead, it looks like almost no effort was put into restricting Grok from doing something ridiculous. The cost here is restricting AI image generation which isn't necessarily that much of a burden on society.

It is difficult to put similar safeguards into Photoshop and the difficulty of doing the same in Photoshop is much higher.