>If I have users that use OpenAI through my API keys am I responsible?

Yes. You are OpenAI's customer, and they expect you to follow their ToS. They do provide a moderation API to reject inappropriate prompts, though.