(Note: I am very skeptical towards CTM and quackery. But also towards AI and ML in general. Any pointers would be great. Why is this the new industry to look out for, for example?)
AI is a tool that we are now seeing solve many problems which computers couldn't even touch 10 years ago with better than human accuracy today. This field is very new and there is significant room for improvement and new approaches. IMO, a lot of the excitement comes from the really impressive progress in a short time and a very wide open field of problems still to be explored.
I would like to ask you and anyone reading this: what exactly gives AI/ML the "edge"? I mean, faster computers are a given these days compared to any other time in humanity. They are giving us simulations so accurate that people in the 80's would be extremely jealous. However, my question is why these new techniques are superior to the established techniques. Why would an AI solution be better? In what sense would it be different from any other method?
Again, I am not anti-AI or anti-ML but I'm interested in hearing opinions. If you could give me concrete examples where AI or ML is better in a way that is different from "faster computing" I would be sort-of satisfied.
For the ones pursuing the CTM line, forget that. I know it's BS and I hope nobody falls for that stuff. However, s/AI/TCM/g in the post would not make clear what the "edge" of AI is over conventional computers.
Thanks for the civil discussion.
Prior to 2012, this was done using traditional computer vision algorithms with hand picked features. The accuracy was often around 30-40%, until the first deep learning model beat the 2nd place competitor by near half the error. Today, almost every single competitor is deep learning based and accuracy is equal to human classifiers.
Since deep learning has so many applications, it's hard to generalize what the "edge" is against all other approaches. In my opinion, other approaches often require handpicking features (where a ANN learns them) and struggle to recognize patterns in many areas (where a ANN does).
If I look at the link you provide, I see an improvement which is great. However, the improvement seems to be somewhat linear year-on-year. In the original post, the first sentence is: "Some think the excitement around Artificial Intelligence is overhyped. They might be right. But if they’re wrong, we’re on the precipice of something really big. We can’t afford to ignore what might be the biggest technological leap since the Internet." Am I right in assuming that there is the expectation of AI/ML to have "hockey stick" growth in the (not-so distant) future? If so, what kind of methods could possibly make this real? If anyone could give me some papers, an essay or some words to search for this would help me a lot. Essentially, I am intrigued why there is this expectation of hockey stick growth. If my premise is wrong, please enlighten me too.
Just one random example: http://lmb.informatik.uni-freiburg.de/Publications/2016/TDB1...
Backpropagation. [1]
Are you familiar with the old Polaroid instant film? [2] You took a picture and pulled it out of the camera and it developed over the next few minutes. The picture went from dark grey nothingness to the image you took, like "magic".
Anyhow, so-called "neural" networks (they are no more neural than Kermit the frog is amphibious) are basically just giant polynomials with changeable coefficients. [3] The input variables are e.g. pixels from an image, and the polynomial is solved to generate a single output value indicating that it is e.g. a cat. (But before backprop this will be uncorrelated, random, useless.)
Now this image-to-yes/no-cat function is very difficult for a human programmer to write out by hand. I mean it's so difficult that no one can do it very well at all.
But using backprogagation, and a big pile of images (some of which are cats and some of which are not) and an index of which are cats for the computer to use (humans will have had to already sort and tag the images for this to work, TANSTAAFL), you can run a simple loop to develop a "function" (in the form of coefficients for the polynomial) that can "see" cats. The "training" phase adjusts the variable coefficients until the result correlates with the tagged "catness" of the images, and then you're done. After that, you can input new images to the function and it will be able to indicate whether they depict cats or not.
The function forms out of the data+backprop like an image developing on the film, projected there by the data set.
So yeah, this AI/ML stuff lets us generate useful functions that we have no other way of generating (literally no one knows how.) It's a big deal.
[1] https://en.wikipedia.org/wiki/Backpropagation
When you have 10M+ documents, it takes a long time for a team of lawyers to review them. So a company with AI/ML is at a huge advantage in identifying the documents a lawyer needs to win a case.
To get accurate results, you need a lot of data, but to crunch that data you need a lot of power, not just speed. A lot of the processing is done now with GPUs (thousands of cores)...traditional computers are CPU intensive and have limited number of cores. Think for example a multi-threaded application running on GPUs vs CPUs...the GPU can divide the problem in more tasks and the processing can be done more accurately not just faster.
http://www.nvidia.com/object/what-is-gpu-computing.html
PS: I think people overuse "Artificial intelligence" we're no where close...but it excites people compared to "machine learning" or "deep learning" or "convolutional network" or just statistics...It's just trying to find an equation that fits the data like linear regression.
In a sense, and correct me if I'm wrong, I get the feeling that the whole ML is just starting now because we have faster computers. To analyze such amounts of text and make some sense of it, you'd need a massive "database" or "fitted model" to decipher what the legalese means. Only recently has this been possible to create (thanks to Moore's law). It's not like there is a breakthrough in mathematics (statistics) right? The theory probably has been around for ages but is only now becoming practical.
That's absolutely correct. But to pooh pooh it on those grounds is like pooh-poohing the fourier transform around the 70s (which theory had been around for 200 years) and only had then "become practical" with the advent of transistor-based computer. Now the FT and relatives are used in everything from audio processing and compression, image filters, statistical analysis, pretty much every form of scientific spectroscopy, MRIs, etc. If you had generally made investments in "FT-related technologies" you'd be doing well.
No one knew you could do it, but yes, it build upon previous work. I was working in the field before and after and if anyone had asked me if one could represent all human languages in only 300 dimensions, and have vector composition be meaningful I'd have laughed at them.
Take using back-propagation to train deep neural networks. People had shown it worked in 1 or 2 layer networks, but despite years of work no one had been able to train anything deep enough to be useful. Then Krizhevsky, Sutskever and Hinton won ImageNet[2], proved it was possible and kicked off this whole ML craziness.
Neither of them is exactly because of more powerful computers, nor magical math breakthroughs. It was more lots of hard work by researchers trying many combinations of techniques until something worked.
These techniques, combined with huge volumes of data and - yes - more powerful computers are what have made the difference.
[1] https://papers.nips.cc/paper/5021-distributed-representation...
[2] https://papers.nips.cc/paper/4824-imagenet-classification-wi...
Please don't get me wrong: there is no future for CTM. You are 100% correct about this. However, quackery is still not a thing of the past. Therefore, I would ask of you not to debunk CTM or to point to its weaknesses (which has been done infinite times). I would like to know the AI-specific answers to your three questions (in your words or in a linked essay / paper). In a way, I don't want AI to be the "quackery of computer science": people use and believe it but it has no advantages. I would love it to be real.
I am all for a future in which we can have AI/machine learning/deep learning if it actually makes a difference. But I would love to know your opinion on what difference that could be and a little on how to achieve that. Obviously it seems I am missing something since YC is eager to invest in this field (or fields).
If I had general_ai.exe or worlds_best_nlp.py, what should I do with it? It isn't even clear to me how they would be useful.
Often I imagine I had a technology such as general artificial intelligence or world-class NLP. Perhaps I lack imagination, but I have not yet thought of an exciting application. These are tools, much like Perl scripts, and it is not clear to me which will displace more jobs during my life.
But seriously, narrower forms of AI are useful for things like self-driving cars.