but would they have liability if they submitted their "output" to a senior artist, who immediately shot it down as obviously infringing? Surely not. It's not illegal to draw Mario - just illegal to make money off your drawing.

I think the real question is whether OpenAI should be allowed to charge for generating infringing content. Even though the unit cost of the Mario drawing is negligible, the sum total of their infringing outputs may be making them a lot of money.