Most roads are built primarily for motor vehicles, so when you are driving a car, you are driving on a road that you paid for specifically to drive your car on, and somebody else is leisurely (except for in certain big cities, most bicycling miles are done for leisure or sport) enjoying the same road for free, on a bicycle, putting themselves at risk and putting some of the burden of their risk on you (if they veer into your lane and you hit them, good luck trying to get out of that).
A grossly exaggerated analogy would be seeing people come to your apartment complex and using the garbage chute as a slide for their kids to play on. If the kids are playing in the chute and you put garbage down it and injure one of them, you’re on the hook. Imagine how this would make you feel, every time you see parents bringing their kids to play in the chute.