Couldn't you just mount your sensors next to an existing car's sensors, measure the car's actions then tell your network "do that" based on the mounted sensors input?
To understand what the difficulty is, it's important to consider that the size of the sensor input is very large. Don't think of it like twenty range finders around the car, rather a 360 degree medium resolution color + depth image (about 0.5 million data points coming at 30 fps).
It's difficult because you will never encounter the same set of sensor inputs twice, so you can't treat it like a search space problem. Once you've accepted that, you're in AI/ML territory where you might try to reason about what the closest set of known sensor inputs and action would be (classical AI, expert system), but that is impractically difficult with as 0.5 million dimensional search space, or train an ML model to 'reason' about the sensor space to make a decision about the appropriate action.
Approaches using a small number of sensors can do automatic breaking and smarter cruise control, but haven't been seen to be successful about navigating and making strategic decisions. The current belief is that more can be done by using denser sensors and more data and seems to be the case. There are people working on reducing the sensor density requirement, but the main focus right now is building a successful and safe self driving car, regardless of sensor and compute costs.
Because of this I'm leaning towards thinking waymo isn't trying to mimic actual human input.
https://www.theatlantic.com/technology/archive/2017/08/insid...
In the future we'll likely have super-human spatial and temporal resolution, right now more improvements have been gained from highest possible spatial resolution with minimal plausible temporal resolution.
I hope there is a better, more technical explanation that ML researchers are using, because as someone who is somewhat of an expert on human vision and building products around it, this foundation is godawful if it is to be taken at face value. Which again, I am sure this is a simplification. Or at least, that's what I am telling myself.
File the promises and the problems under fiction because it appears to be more important to keep the world order, its financial system and these ridiculous media darling fluff piece corporations alive while they bleed money.
And no, im not closed to the idea of successful work being done on automated driving but 30 fps, WTF? too much going on in the larger context of the world, this shit isn't happening in 2020 or 2024 or whatever else many might say.
It's probably a very valid tradeoff.