You are putting the cart before the horse. The problem is not in prioritization, the problem is in having the correct ontology to even get to the "prioritization" stage.
Does the car know what a fruit stand is? Does it know what a gas pump is? Does it know how the fruit stand relates to the gas pump in "expected outcome when being hit by a car"?
If you say "we can program that in", read my post again.
On a level below identifying stop signs and lollipop ladies and push carts, a SDC's stack needs to be able to identify:
1) Driveable areas. If something looks like a cliff maybe don't go there.
2) Fleeting obstacles. Dust blowing in the wind. A stray plastic bag, winging its way northwards to the waiting maw of a baby turtle. A person with a borderline credit score. A stray cat chasing a bug. That sort of thing.
3) Anything else that's physically present in the path of the car. Doesn't matter what it is. Do Not Hit The Thing is the second lesson anyone learns when being taught to drive, after Make It Go So Hit The Thing Is Even An Option.
"the problem is in having the correct ontology to even get to the "prioritization" stage."
That part isn't done by the program, it is done by whoever enters the prioritization numbers. That is, someone, possibly a committee, can dial up the "avoid gas pumps" weighting relative to the "avoid baby stroller" weighting if they are concerned that cars might swerve so widely to avoid coming near a stroller than they are risking hitting a different hazard. Or they can dial up the weight of grocery carts relative to dogs, since children might be in a grocery cart. Etc.
Those are humans, who can do whatever ontological analysis they need when deciding on the the settings. The car doesn't need to access any of that, it just needs a general look up table that can help make optimal decisions based on the human-entered value system.
It's completely feasible to imagine writing a list of the top, say, 100 things in the world that a car needs to make morally-significant decisions about, and then deal with every other accident or near-miss after the fact. Interactions with unrecognised objects should be rare enough to be a rounding error when comparing accident rates between autonomous and human-driven miles.
If it’s choosing between hitting a pole and hitting a pony, it should always hit the pole so long as no one is injured.
The real problem is that occasionally these cars today mistake roads for oceans, walls for roads, and people for inanimate poles.