I know exactly where my inverted Y-Axis comes from: Flight Simulation. Like in the real airplane, pushing the yoke forward pitches down, pulling the yoke backwards pitches up. So I do with the mouse, and the joystick, and the controller.

Once adapted, it is impossible to change afterwards.

Drones are even more complicated: left hand/right hand preferences (regarding power and lift) can never be re-learned, so most remote controls come with multiple configurations.