You have a problem that is unconstrained, with infinite variables, where even a simple mistake can have catastrophic outcomes. Society itself may object to self driving cars for a ton of reasons, from safety to simply driving like a grandma and slowing everything and everyone down. The cost to develop this technology, plus the added cost of hardware to each car, will be enormous and is not obviously a cost savings over a $15 per hour human. If the self driving car is doing anything other than getting from A to B, you still need a human (or a human-like robot) to handle the unloading / delivery / whatever at the end.
Now, I’ve worked at companies with extremely talented and intelligent engineers, and something as constrained and seemingly simple as making a login form can take a long time to perfect - and no lives are at risk! Just imagine the challenges and requirements for building self driving cars. New hardware, software, real-time processing and analysis of tons of data, all to drive split-second decisions that can kill people if done incorrectly.
Huge challenge - huge risks - huge money - uncertain payoff. This is not something that will appear suddenly. If there aren’t convoys of self driving trucks operating in desert highways overnight, where it’s dry and straight and flat and no one else is there, then we aren’t going to see city taxis for a very long time.
It may not even be possible to solve without something radical like banning human drivers or inserting electronic nodes directly into our roads and infrastructure to aid autonomous vehicles. The existence of human drivers might make the problem simply impossible to solve in a way acceptable to society.
Software professionals are somewhat notorious for underestimating the difficulty of their projects. There's a massive amount of literature about that, proposing various techniques of mitigating this on a personal, team, or organization level.
That being said, I think in this particular case the problem was more of an overestimation of what the ability to work with highly dimensional data (as in ML algorithms) can give you (and underestimating the practical problems of deploying the ML-based systems). Basically, people tried running 30+ years old algorithms on GPUs while feeding them the ungodly amounts of data and realized that, with sufficient horse-power, they can make them (finally) work (as in: do something genuinely useful). This built up a lot of hype, of which self-driving cars are just one offshoot, I think.
Anyway, (if it's not evident from the above ;)) I find your arguments convincing and share your doubts. Self-driving cars are probably not impossible to achieve, but to get there we either need decades of research and progress or a couple of very high-profile breakthroughs in tech and theory. I won't hold my breath for neither :)
People are conditioned to accept that we will kill each other with cars every once and a while. I'm skeptical we will get to the point any time soon where the general public hears about a family killed by a self driving car and just shrugs it off. There will be intense pressure to get them off the road.
I'm not sure this is correct. I come from the optimal control world. The problem is not unconstrained and definitely does not have infinite variables (if you think in state-space, consider the state-space equation x' = f(x, u, theta). The x-space (state) is large but finite, and the u-space is fairly small -- steering, gear, brake, etc.). The x-space is also stochastic, and there are many observability issues.
That said, we've been designing control systems against the real world for a number of years now, and the key is not to model all the unknowns (because there will always be something you can never anticipate), but to model the known and safe path that the system can fall back to.
The problem is a complicated one, and it will take more than several attacks to make it work, but it is not by any means an impossible one if you break it down into the fundamentals.
Also, control systems have been used to control much more complex entities than just factories.
I agree it is an extraordinarily complex problem. But I also believe progress can be made to a point where it can be feasibly solved.
Not the OP, but like you say, the article is about a company who claims to have developed a technology that will solve self-driving cars' problems with rain.
People in industry make claims all the time. People in the sciences do, too. Just because someone makes a claim, doesn't mean it's true. It's only a claim.