Learning to Love the AI Bubble
sloanreview.mit.edu
sloanreview.mit.edu
It was right about the economic potential of the www. 20 years later and it is as big a deal as 1999 pundits predicted. It was right that the big winners would be very big, very fast. Google, FB Alibaba, amazon... Bigger than any tech company in 99. It was right about winning early and establishing dominant position... letting network effects and the scaling power of software and the web go to work.
Unfortunately for binary outcomes.. getting 4/5 things right is still a wipeout.
The bubble was slightly off about timing. More big winners were founded/determined in the 5 years after the bubble than during it. It slightly overestimated early mover advantage.. closely related to the timing mistake.
If that bubble is the model for this one... interesting times ahead.
Will the same happen to AI? Uses of AI and "AI" seem to take very well to today's world. Everybody wants a piece of the action be it in ad networks, big data, surveillance, cat ear filters or fake nude pics. People are much more technologically literate and the concept in any form will not land in barren land. It's sometimes even frustrating the way people expect a certain level of intelligence from basic applications (eg. try to implement a search function that is not error-flexible and you'll get angry comments).
Internet paved the way to having lots of redundant data we don't know what to do with. I think the world is all too ready to welcome advances in AI, and it's in fact ignored what it will do to financial systems or personal lives.
^great example, btw. auto complete and search captures it.
One thing is AI to the press and public, another thing is AI to investors, yet another thing for nontechnical workers, and not even a single cohesive thing to people building it all. Wherever you personally draw your lines between AI and not AI, the boundaries do keep expanding. Does that mean the bubble is growing? There are undoubtably more people doing machine learning, and there are more people doing statistics, and more people solving optimization problems, and each other thing that we call AI, but the "AI" label is growing faster than all that. It's a weird bubble. If it pops, does that mean there will be fewer jobs for people like me, or does it just mean people will just stop calling it AI? Or is this just a word's meaning changing and not a large bubble?
This story comes to mind: http://web.archive.org/web/20190626012618/https://gen.medium...
> Taking averages, grouped by something? That's AI now.
I think that is right. The algorithm that does the grouped averages is machine learning, and if you put error bars around it, it is stats.
To address your concern: I wouldn't worry about the relevance of applying math and logic to the world. It has always been growing.
Neural networks are shiny and new, but they are just an implementation for solutions from stats that have been around for decades.
Regression? MSE loss. Now with a neural network trained on MSE loss.
Classification? Logistic regression with cross entropy loss.
Anomaly detection? Feature extraction? Plenty of people still use PCA, which is nothing new. Autoencoders may get you more mileage, but conceptually work very similarly to PCA for these use cases.
Image data? Use methods from signal processing, also decades old. Convolutions are nothing new, you're just now implementing them with neural networks, and adding a loss function based on what you're trying to predict.
Time series data? You could be better off just sticking to ARIMA. Depends on your use case, but using RNNs may not even work here.
Reinforcement learning is more exciting, and is solving new problems that weren't even being approached before. Same goes for GANs, and unsupervised learning in general stays exciting and fresh.
But most of the applications of AI are ho hum. Just use decades old methods, now implemented with neural networks. At least, sometimes. What has really changed is the amount of data available now, and the ability to process it. Not necessarily the approaches to analyzing it.
You're still approaching stats problems with the same methodologies. Your just using NNs as your optimizer.
Some people say AI and mean "a thinking sorta-conscious thing that thinks like we do." Like C3PO or R2D2 or HAL or ourselves. That's their definition. People complain that that's not a rigorous falsifiable concept, so they say, "Okay here's a test that I think can only be done by a thinking sorta-conscious thing that thinks like we do." Then someone clever figures out how to do it without something that fits their definition. They respond, "Okay fine I guess my test was bad." It's kinda moving the goalposts, but their fuzzy conception of what AI means is still unchanged. It just remains hard to make concrete, especially since we can't really define things like consciousness in the first place.
Another camp is marketing. It takes anything it can sell by calling it AI and does so. The goalposts lower. Linear regression is branded as AI now, like it or not. This is the opposite direction of what you're talking about.
A final camp is what you talked about. I think these people are really in the first camp I mentioned. They don't move the goalposts on their concept, they just move the goalposts on their test. But there are also a distinct group who consider it as "something normally done by humans that most people don't think computers could do proficiently" or even just "computerized decision making/information processing."
I'm in the first camp. "AI" is no less meaningless than "consciousness" but it's equally hard to define. Some people have begun using the label "AGI" for it. Same concept whatever your word choice. I think of C3PO but understand that words are defined by usage and maybe linear regression is AI now.
While I certainly wouldn't call a technical discussion off topic, I'm not surprised that the majority of focus would be on the main topic of the article -- economics and the stock market.
I am going to be traveling for a machine learning convention in two weeks as well, but I'd love for a good place to find some background on this so I can maybe be successful in competing there.
If I only had 2 weeks of evenings and weekends to conjure up some ML knowledge, I would start there. Then you could move on to the courses from fast.ai (https://www.fast.ai/)
It walks you through all the basics of deep learning (with PyTorch) with a concept video, code video, and then suggested project for each week.
https://www.coursera.org/learn/machine-learning
Everything is done in Octave (ie - open-source matlab-like language); primitives are vectors and matrices - so you'll have to wrap your head around that.
But that course gave me the first explanation as to how neural networks actually worked that I could understand; I had been reading about neural networks for years from various sources - books, online, videos, etc - and nothing ever "clicked" for me (mainly around how backprop worked). For some reason, this did it for me.
Since then, I have taken other MOOCs centered around ML and Deep Learning, mainly with a focus on self-driving vehicles.
Oh - ML Class also led one individual to implement this during the course, as the ALVINN vehicle was mentioned in more than a few ways:
https://blog.davidsingleton.org/nnrccar/
While Singleton does mention its "vintage-ness", I still think it's a sound project for inspiration and learning how to apply a neural network to a simple self-driving vehicle system, not to mention the fact that it replicated a system from the 1980s using today's commodity hardware; I recall reading about ALVINN when I was a kid, with wonderment about how it "worked" - it was one of several 1980s projects in the space that got me hooked on wanting to learn how to make computers learn.
Unless we are using a different definition of "bubble" than it would seem (to me) most people intend when they use that word, this is a factually incorrect statement.
If widespread misallocation of investment capital does not "hurt" the economy, then what does? If capital is invested with a positive outcome for the wrong reasons, then that certainly would not fit the definition.
First applicable definition from a google search:
Bubble: used to refer to a significant, usually rapid, increase in asset prices that is soon followed by a collapse in prices and typically arises from speculation or enthusiasm rather than intrinsic increases in value.
Defining "bubble" can be surprisingly difficult and in some cases the illusion of a bubble persists after history proves the speculation or enthusiasm to have been correct despite an earlier collapse in prices. So one interpretation of "Not all bubbles have negative consequences for the economy" is that there are events that are widely perceived to have been bubbles that were not, in fact, widespread misallocations of investment capital.
Along the lines that Webvan may have tanked but we ended up with Google and Wikipedia. I imagine likewise that most of the present AI startups will tank but we'll end up with useful AI of serious value.
I am no economist, but virtually any economist I have read would respond to that statement with:
Compared to what?
By the way you can read the article with a trial account from O'Rilley but there's not much too it beyond what's in the summary https://learning.oreilly.com/library/view/learning-to-love/5...
I'm more worried about global economic crises than VCs losing their money.
Misallocation of limited human resources. Good people go from doing long term fundamental research into developing applications in startups.
Overinvestment and misallocation of capital resources during the bubble can lead to long period of underinvestment once the bubble bursts.