If you can't move at or above the posted speed limit, let others pass. It's as simple as that. You're not entitled to impede others. All users of the road
are entitled to move at the speed limit, when there there is no problem ahead such as an accident, congestion, obstruction or construction. If you're not yourself impeded by such a predicament, and have an open road in front of you, you have no right to arbitrarily impede those behind you. It is simply rude.
Don't go down roads where it's impossible to pass you, unless you can do the speed limit there. Don't go down roads where there are parked cars, but only a small clearance between parked cars and traffic moving faster than you. Or any analogous situation: like small clearance between a concrete wall and traffic or whatever. Find roads where there aren't parked cars, or else where there is a generous space between the parking strip and traffic, so that you can avoid doors, and there is room to pass you. Where I live, many of the larger streets that have two lanes in each direction, and a solid car's width between parked cars and traffic. Someone opening a small door will not door you, and you have time to avoid a large door being swung fully open. You can also find roads where there is simply not much traffic, so you can have them all to yourself most of the time.
Drivers are not supposed to approach you closely when they pass; that's against the traffic code where I live, and likely in most places. A cyclist or motorcyclist must be passed as if they had an invisible car around them, so to speak. However, by straying farther into the lane, you're encouraging a reduction of the space between you and and the car that passes you anyway, which is counterproductive.
Parked cars are a menace, no doubt. In some countries, they don't allow street parking in cities. To register a car, you must prove that you have some place off the street to stash it when you're not driving it. North American cities should follow suit.
> little damage to their car, and so they have little reason to care
You're simply a lunatic with this remark. Most drivers are normal people, not psychopaths, and are absolutely terrified of the idea of injuring or killing a pedestrian or cyclist. That's still true even if some of their behaviors need improvement.