If it's true, then we're going to have something very similar to the Star Trek computer by the end of the decade at the latest.
If it's false, then we're probably going to enter another AI winter this decade.
followed by GPT95, 98, XP, etc
Even if there are a lot of great applications for these things (I think there are), it’s not clear they’re going to be truly transformative?
For me, it is already pretty transformative. I use ChatGPT at least as much as the regular internet - probably more.
Not just as a hobby/toy?
1) I wanted to make a github action to auto build, run tests and publish a package. I don't do a lot of this type of work so quite unfamiliar with it. ChatGPT gave me a very good start with some example yaml which I was able to tweak and get started with. When something wasn't working with my tweaked yaml file, I asked it what the problem was and it was able to pin-point the issue right away. I don't use it at work for anything but it is very useful for side projects and things like that.
2) I had some example json text from a service for which I wanted to create some serialization classes. This is easy to do but it is tedious work. ChatGPT was able to write most of this for me. There are likely tools, websites, services that can create this, but the experience in ChatGPT is pretty great - copy and paste in some json text and ask it to create the code. If it isn't quite right, give it some additionally instructions. When 80% OK, manually tweak it.
3) I use it to get ideas for things or to provide answers about things. For example, one of the commenters in this thread discussed the "AI scaling hypothesis". Google would have had a good answer for this but if you happen to have any questions(after the initial response is provided) this is a completely new search. ChatGPT, though not always correct, feels a lot more calming to use. Google seems very "noisy" to me now. Google is still very necessary but it feels like only 20% of queries need to go there at this point.
Obviously generative usecases are awesome (though IMO will be practically fewer and further between than people seem to think), and summarization usecases are awesome.
For example. I could have figured it out elseways, but would have taken >20x the time.
Eg https://www.cnn.com/2023/01/28/tech/chatgpt-real-estate/inde...
Also these types of use cases I think will not actually last very long. They’ll be adversarial’d out of value and then probably out of existence after that.
In a world where every listing is written by a bot, a reader’s first objective is to distill the human-written parts out of the unreliable narration and filler words of the bot.
> "Computer, locate Commander Riker's repo"
> "Commander Riker's repo is not currently on the ship"
Some graduate students I collaborate use it to help them revise and polish their writing (English is not their first language)
1) Short scripts in languages im unfamiliar with
2) Generating short, specific, well defined functions
3) Curing the tyranny of the blank page effect re emails, explanations, etc
4) Rephrasing that long email last thing on a friday when English just wont click
I suspect there will be a lot of resistance as it “succeeds.”
I kind of get it, too. If I made my money making art this would scare me. What I wonder is what's gonna happen when it's [ artists, accountants, copywriters, bloggers, designers, programmers, etc, etc, etc].
I see problems coming.
The way I’ve interpreted the scaling hypothesis is that we will see emergent intelligence with larger nets through automated training alone. If we want a model to learn images we throw a larger net at it with training data.
The way I’ve interpreted some of these newer techniques to multi modality is that they are stitched together models. If we want a model to learn images we decide this and teach one model images and then connect it to the core model. There’s not a lot of emergent behavior due to scaling in this scenario.
With that perception I don’t see how gpt4 says anything about the scaling hypothesis. However I am not in this field and would be grateful to learn more.
So the goal should be: we've created a "language module" (LLMs) and a "visual perception module" (computer vision), but we also need to add a "logic module", a "reasoning module", an "empathy module", etc, while continuing to improve each.
I just don't see how you could get an LLM, no matter how advanced, to recognize a car. Even if it can describe cars (wheels, windshield, doors) it doesn't know what any of those components look like. It's like that old joke about philosophers being unable to define a chair beyond "I'll know it when I see it".
these net can replicate at most the first step for now, even tuning by reingofrcement learning is more of a set up batch than an ongoing thing, and certainly not something they will be able to do in the context of a single problem but as part of a retraining
agi is still a fair bit away, I'm unsure if these super large architecture will ever get to replicate the second part of our brain, the flexibility while on the job, because of their intrinsic training mechanism.
Search engines run on AI, stock markets trade on AI, social networks distribute content by AI, your photos and content are sorted on your phone by AI, vaccines and medicines are produced by AI, precedent law is implemented increasingly via AI, our weapons are AI (drones) and so on and so on and so on.
And all this up there was BEFORE GPT and DALL-E.
No, no winter. We're in the singularity.
Are they composed of autonomous agents that exhibit: - diversity (in behavior) - high connectedness - interaction among agents - adaptation (agents changing their behavior based on the behavior of other agents)?
Because if current NNs don't fit that definition of complexity, they it seems unreasonable to expect them to be intelligent, no matter their scale. Intelligence will likely be an emergent property of the above four criteria.
If anyone knows of relevant books or papers, would love to hear it.
BTW: "AI winter" refers to a collapse in funding; but I think we're well past that point. Previous winters only happened because academia led AI research, and there were no commercial opportunities. Now, the economic benefits are far more tangible. And if funding doesn't dry up, there's no reason to think innovation will dry up either.
Once the continuous learning problem is solved, it'll be pretty trivial to just tell your compound model to "go learn everything you can about x" and have it autonomously do so.
If it were me, the first thing I'd ask it to do was learn how to make other AI, then give it some hardware and the directive to build the best thing it could and test it until it can build a model better than itself.
If science can neither have a better definition of intelligence, or analyze and understand the resulting black box of a neural net, I don't see how AI would improve or get nearer general intelligence.
I have much more curiosity about neurology and how psychology define intelligence, than how programmers are trying to build things that digest data.
Proper science requires understanding. ML is not about understanding, it's only "sophisticated statistical methods". It is still artificial, thus but genuine.