Then there was the naysaying when Apple was first getting into phones. What does Apple know about phones? How can they possibly compete with the likes of Nokia?
Often industries are disrupted by newcomers who aren't constrained by the thinking of the existing players. Yes, Musk can't violate the laws of physics or economics. But the cost structures of existing players aren't always dictated by the fundamentals, but also by their own organizational baggage and structure. Hence, SpaceX can compete effectively with DoD defense contractors hooked on hugely inefficient government contracts. Is it that Boeing or Lockheed can't things as cheaply, or that SpaceX quality has to suffer? Maybe the way the government dictates things get funded and manufactured leads to inefficiencies, like spreading things around congressional districts.
Likewise, for other car companies, the Big Three in particular, they are somewhat constrained by their existing investments, existing unions, and just plain old conservativism and aversion to risk taking.
So maybe Musk will fail utterly. No pain, no gain I say. He also has a chance of succeeding spectacularly.