4 years ?!?
That’s like a century in AI-dog years. Who knows how the world will be by then.
That’s like a century in AI-dog years. Who knows how the world will be by then.
When Big Blue beat Kasparov in Chess in 1997, I wonder if anyone would've guessed that it'd take almost 20 years for a computer to beat a master in Go.
IBM Watson was launched in 2010 and had many of the same promises as GPT. It supposedly fell flat in many cases in the real world. I think GPT and other models of the same level can succeed commercially on the same tasks within the next 1-4 years, but that shows it can easily be a decade from some kind of demonstration to actual game changing applications.
The advances that have come in the last few years have been driven first and foremost by compute and secondarily by methodology. The compute can continue to scale for another couple orders of magnitude. It's possible that we'll be bottlenecked by methodology; there are certain things that current networks are simply incapable of, like learning from instructions and incorporating that knowledge into their weights. That said, one of the amazing things about recent successes is that the precise methodology doesn't seem to matter so much. Diffusion is great, but autoregressive image generation models like Parti also generate nice images, albeit at a higher computational cost. RL from human feedback achieves impressive results, but chain of hindsight (supposedly) achieves similar results without RL. It's entirely plausible to me that the remaining challenges on the path to AGI can be solved by obvious ideas + engineering + scaling + data from the internet.
We've also gotten to the point where AI systems can make substantial contributions to engineering more powerful AI systems, and maybe soon, to ideation. We haven't yet figured out how to extract all of the productivity gains from the systems we already have, and next-generation systems will provide larger productivity gains, even if they are just scaled up versions of current-generation systems.
This is a different 'Leap' than the ones before it. It's a leap with an API. Now hundreds of thousands of company's can fine tune it and train it on their specific business task.
parroting your point, it will take years for the true fecundity of the technology in chat GPT 4 to be fully fleshed out.
Historically yes. Today, no way. It's a sprint and it's not slowing down.
I think the recent release of ChatGPT has skewed perceptions. There's no guarantee that there's going to continue to be as ground breaking shifts that have happened recently with llms and diffusion models.
To continue with the popular comparison, there were a lot of apps when the iphone first launced the app store before it tapered off. If you looked at just the first year, you'd think we'd have an app for every moment of our day.
Social impact of ChatGPT even in its current form is only getting started, it doesn't need to progress at all to be super disruptive. For example, see the frontpage story about the $80/h writer who was replaced by ChatGPT, and that just happened recently, months after ChatGPT's first release.
We (humans) are getting boiled like the proverbial frog.
GPT 3 is nearly three years old at this point, and was pretty capable at generating text. GPT 3.5 brought substantial improvements, but is also over a year old. ChatGPT is much newer, but mostly remarkable for the better interface, the extensive "safety" efforts, and for being free (as in beer) and immediately accessible without waitlist and application process. Actual text generated by it isn't much different from GPT 3.5, especially for the type of longform content you hire a $80/h writer for. ChatGPT was just launched in a way that allows people to easily experiment and create hype.
The parent is right. The success of ChatGPT in business is that it brought awareness of the capabilities of GPT that OpenAI struggled to communicate beforehand. It was a breakthrough in marketing, less so a breakthrough in tech.
You could have the best most magic product on earth and sell one of them versus the person that puts it in a pretty box and lets grandma use it easily.
This is something that many people on HN seemingly have to relearn in every big innovation that comes out.
This is such a primitive way of thinking. It's more of an instinct, where you consider by default that your sole value is in your ability to generate/work. Why the hell are we working for? Isn't it to improve our lives? Or should we improve them up to the point where we still have to work? Why not use the tech itself to find better ways of organizing ourselves, without needing to work so much? UBI and things like that. Why be such limited? Why only develop tech up to the point where we would have to work less but not at all, and who decides where that point is? There's so much wrong in this framework of thinking.