121 karma · joined December 20, 2018
Elon: "A major part of real-world AI has to be solved to make unsupervised, generalized full self-driving work, as the entire road system is designed for biological neural nets with optical imagers"
Seems like he's inching closer to reality by the day.
Elon Musk has been pretty successful at getting top talent to push the limits of engineering, but with neuroscience, it seems the research isn't there yet. What I'm really interested to see is how long Karpathy will stay at Tesla as other self driving projects drop like flies (ATG, Zoox, and most recently L5)
What I'm saying is there shouldn't be outrage until we figure out that the OP isn't lying, which there seems like there is reasonable doubt for. Because for very evil circumstances I wouldn't expect there to be transparency either for legal reasons.
https://fortune.com/2018/09/29/google-apple-safari-search-en...
However, it doesn't look like this is happening anytime soon.
GPT-3 and other projects seem to drive hype cycles in the tech community and convince people like Elon Musk that the AGI revolution is near. But I think recent progress is just another example of machine learning models being able to generalize on super large datasets, even if it's the biggest model so far. It's not clear to me that larger models will solve this in the limit; take the way GPT3 fails on addition past a certain number, and the fundamental inability for transformers to learn certain algorithms. It is certainly still possible for this type of large dataset, large model style of ML to make human life better in many ways - like Tesla is trying to do with self driving cars, or Covariant with automating Amazon-like jobs. But I think when it comes to tackling the hard problems of true intelligence, we're missing a dimension somewhere.
Read slide 17. The key assumptions are that: the confounder present at bunching point is representative of the confounder present everywhere, and the confounder affects outcome linearly. The second assumption is what strikes me as too strong, and you can read the paper Appendix C for how they address it. The reasoning is quite weak and they even phrase it as such: "Our main empirical findings therefore do not seem to be an artifact of this linearity assumption". Overall I would take this paper with a grain of salt, from a cognitive science point of view it just makes no sense that doing more reading would have no cognitive benefits (in fact, we know the opposite to be true.)
Also, the paper hasn't even been reviewed yet. We should revisit it after it has been peer reviewed.
Wikipedia says "the scalars can be taken from any field, including the rational, algebraic, real, and complex numbers, as well as finite fields."