Audio software will typically allow you to configure a soft takeover, where you must move the fader or knob until it matches the software value, and only then does it take over, to eliminate these surprises.
Rotary encoders are just not as good for controlling volume as faders.