Ask HN: What will the state of AI be in 10 years?
Hey HN,
I'm a graduate student studying ML and Theoretical Neuroscience, and I've been thinking about what the state of ML will be a decade from now.
What are your thoughts?
I'm a graduate student studying ML and Theoretical Neuroscience, and I've been thinking about what the state of ML will be a decade from now.
What are your thoughts?
The math progress has not been substantial, frankly. The very same ideas were there before. Sure, there are some new ideas, but they too are the result of the real fundamental change, which was the scale-up in compute and in memory.
So where I see a good analogy is were computing was in the age of the mainframe and these huge computers that Bill Gates and his chums were fiddling with in assembly, compared with personal computing.
ML is going to be everywhere, the way databases are everywhere. To an extent, it'll be an extension of SQL, in the sense that if SQL is about "counting stuff on data" then ML is a more sophisticated take on that idea. Where you'll have data, you'll have an engine that extracts stuff from it the ML way. I am guessing transfer learning will play a substantial part in this, but I'm less certain on that statement.
Just like with personal computers, our expectations of what you can "just do" will adjust. But all of that will not stem from progress in ML, just that current technology becomes more widespread. As far as the cutting edge of ML, judging by the sort of papers I come across, I think we're currently at the stage of doing more-of-the-same only bigger.
Again, I find history a really useful teacher in that sense. Take a trip to a science museum to see that people have always first taken any technology they've had to its utmost boundary, alongside the diminishing gains, before something new came along and the curve restarts. We're already in diminishing-gains territory, but it's hard to say how long that kind of momentum lasts.
https://www.brookings.edu/blog/future-development/2020/01/17...
> steam engine, electric power, information technology (IT), and artificial intelligence (AI).
Just no. AI has not come, it hasn't even started. What people call AI today is literally faster IT infrastructure in implementing math. The proper term for what we see today is machine learning and it is based purely on statistics.
Also, I think interpretability and the introduction of better understood priors will lead to extremely large, sparse networks being used across multiple modalities in applications and software.
The only roadblock is I can’t think of any way to build in the three laws in such a way that they can’t easily be removed.
- Custom chips will perform the inference and probably the training. For example, Tesla managed to create a chip, which is 10x cheaper than a GPU for training.
- There will be some breakthrough that will combine logic and stat. This would eventually lead to AGI.
Similar to how languages that are 'designed for non-programmers' fared?
The ML dev process is an optimization process and not a decomposition process (I.e. it is stateless).
Instead, it is much more experimental (and hence declarative). I.e. you can define a search space, and substitute the data scientist time, with computation time.
This is the idea of "software 2.0"
https://www.youtube.com/watch?v=zywIvINSlaI
This is how Tesla operate today for its autopilot project.
Really? Where can I buy that chip?