That said, this does follow a very old and well-documented pattern: build a consumer image generation tool, and a significant portion of users will try to push it toward sexual or nude imagery, especially involving women.
Even companies with massive resources struggle here. Try generating anything even mildly suggestive involving women with ChatGPT and see how many hoops you have to jump through and that’s after multiple layers of prompt and output filtering.
At that point, content moderation becomes an arms race. Keywords, rate limits, paid tiers, moderation APIs, users will route around all of it. Without huge ongoing investment, it’s a battle that’s very hard to win.
So your conclusion that the problem isn’t the product but the market resonates. A B2B pivot makes a lot of sense, because the incentives and user behavior are fundamentally different.
Edit: Even OpenAI seems to be acknowledging the limits here and has indicated plans for some form of adult mode next year. It will be interesting to see whether that also includes more relaxed image generation policies.