I think this is a great way of putting it, because I think there's a real and necessary distinction here. The sophomoric relativism of "Gee aren't all games dark patterns?" feels like a smart thing to say, but is wrong for reasons that feel obvious but get kind of tedious to unpack.

As a different way of making the same point as you, I think critical differences are: (1) whether the compulsion to play serves you or the game creator, (2) whether the game mechanics serve an entertainment purpose or are grafted on to a game that would be just as fun without them, (3) whether it's designed around an "upgrade treadmill" as the core interactive structure of the game, as opposed to elements that have some degree of human craftsmanship to them (e.g. the mobile game Sorcery is fundamentally about the story).