Teen sues to destroy the nudify app that left her in constant fear

I can' see how this kind of service can be at all legal if it can produce CSAM images - there's been people prosecuted for producing drawings of underage children which arguably don't actually harm anyone, whereas this app can drastically harm people, young and old.

It was always possible to create these kinds of images. Only you needed to be (or find) a highly skilled yet unscrupulous artist to do it, which didn't happen in practice. Now AI has commoditised this. It's not going to go away.

Regulating services can only go so far because as long as general purpose computing exists people will eventually be able to perform all of these abuses locally.

I think the only solution that will work in practice is to go after the abusers based on their intent. Going after technology providers is never going to work because the technology is fundamentally general purpose. Wherever the line is drawn, it will always be possible for abusers to take it and specialise it for abuse locally.

Edit: to be clear, I can't think of a legitimate use for this service and it sounds like their behaviour is abusive and they should be shut down. But that won't stop the abuse because sooner or later abusers won't need a service to carry out this abuse. They'll be able to use a generic tool to do it locally instead.