In fairness, your operation of the vehicle has a substantial impact on the outcome of operating the vehicle, the the decision loop that your brain implements while driving is largely incapable of effectively processing and executing on the volume of information and variables that are in play in the circumstances that lead to high speed collisions.
Anecdotally you might feel in control, but it is probably worth noting that most people, drunk or sober, tired on fully awake, fully alert or texting, felt like they were in control when they were suddenly the cause of, or involved in a collision.
So now you are in an accident caused by someone else. If that accident would have been prevented had they turned on the automated driving system that is in their vehicle, then do you still prefer this? Is having someone to blame really worth being in an accident that would not have happened if the computer was in control?
So, is it truly reasonable for you to refuse to engage the self-driving system in your car, putting other people on the road at greater risk?
I wonder if there will be increased liability, both financial (insurance) and legal, when someone willingly chooses to take control in a situation that would be well-handled by their self driving vehicle.
So? That's something humans tend to do, and anyone trying to sell self-driving cars will have to account for it.