Off-hand, I don't remember what that rotation equation is to be most effective.
Off-hand, I don't remember what that rotation equation is to be most effective.
A mouse is in general a horrible input device for inputting arbitrary 3d rotations, because it doesn’t have enough degrees of freedom.
By far the best tool is a 3-DOF trackball set up so that arbitrary rotations of the ball are directly converted into rotations in the computer.
Next best is some other physical device with gyro/accelerometer so that (slightly less accurate but still) direct manipulation can be input.
Next best after that is something with multitouch or perhaps one of those 6 DOF “space mouse” devices.
Of course, that's just the way 3D rotations work: SO(3) [1] is non-Abelian. However, doing this with an input device that has one too few degrees of freedom makes it very awkward to use.
When you have a well-defined "ground" plane in your scene, then turntable-style rotation is the most intuitive, where horizontal dragging rotates around the ground plane's normal, while vertical dragging changes the azimuth angle. This makes the final orientation independent of the cursor's path, on the other hand, orientations where the ground plane would be skewed are unreachable.
Some people swear on Ken Shoemake's ARCBALL [2] technique, but I personally found it less intuitive than the turntable method (although I might have implemented it incorrectly ¯\_(ツ)_/¯).
[1] https://en.wikipedia.org/wiki/3D_rotation_group
[2] https://tommyhinks.files.wordpress.com/2012/02/shoemake92_ar...
With non-turntable rotation you always end up having to make little circles with your mouse to rotate the view horizontal again.
Neither of these control styles are 6dof; they don't provide a roll control, only elevation, azimuth and position. Roll is rarely useful IMHO and including it in the normal control scheme makes things hard to use. Put roll behind an explicit modifier key, and have a reset-roll action somewhere.