You know, this one I understand entirely. When I walk on a road without sidewalks, I always walk on the wrong side of the road, so I can see what's coming and whether they're paying attention. Getting mowed down from behind by someone not looking where they're going isn't on my bucket list.
If you ride with traffic and get hit, your relative speed at collision will be 15 mph and the collision will push you away from the car.
If you ride against traffic and get hit, you're dealing with a head-on collision at 45 mph relative speed.
That's why I always ride with traffic.
If you are going to stay on the road with other vehicles: You _never_ ride against traffic. Drivers are less likely to see you if you are biking where they normally don't anticipate any traffic. Even in a rural environment, biking with the flow of traffic gives the driver better visual cues that something else is on the road (you) and they have more time to plan their approach. It also gives them more time to find something appropriate to throw at you.
If you ride against traffic, then you still "don't see" the vehicles that are going in the same direction as you, and you are now on their left. Sure, they're on the opposite side of the road, but you don't know if they are paying attention or what. Thus: no more safety gained by riding on the opposite side of the road, where you aren't supposed to be.
The most dangerous area is, obviously, an intersection, where being on the wrong side of the road is demonstrably less safe[1]. After that, it's the blind spot over the crest of a hill. If you are going against the flow of traffic, you will be moving your slowest when you are in that blind spot. Moving with the flow of traffic -> moving quickly through that blind spot.
Knowing when a vehicle is coming up behind you, with plenty of time to react if necessary, is solved with a $12 helmet mirror.
Pedestrians, in the absence of sidewalks, do have the option to face incoming or outgoing traffic, and the usual choice is to face oncoming traffic as you suggested.
Additionally, the increase in speed differential (which is much more significant for cyclists than pedestrians) created by cycling towards oncoming traffic greatly reduces available reaction time of both parties.
When you are travelling in the same direction as the traffic there is usually still a speed differential but it is substantially less. While you no longer have the ability to see potential problems "coming at you", drivers have significantly more time to notice you and react accordingly.