Seriously, a few years ago recognizing if there's a bird in a photo was an example of a "virtually impossible" task: https://xkcd.com/1425/
Yeah I mean, I agree. But in my opinion, it's a case of "doing the wrong thing right" instead of a more useful "doing the right thing wrong."
I grant that these automated models are useful for low-value classification/generation tasks at high-frequency scale. I don't think that in any way is related to intelligence though, and the only reason I think they've been pursued is because of immediate economic usefulness _shrug_.
When high-value, low-frequency tasks begin to be reproduced by software without intervention, I think we'll be closer to intelligence. This is just mimicry. Change the parameters even in the slightest (e.g. have this algorithm try to "learn" the article it created to actually do something in the world) and it all falls down.
Progress is often made with steps that would have been astonishing a few years ago. And every time the bar is raised higher. Rightly so, but characterizing this as doing the wrong thing is missing the point of what we, and the system, are learning.
Yes it's not intelligence. But then, it's not even clear that we ourselves can define intelligence at all… not all philosophers agree on this. Daniel Dennett (philosopher and computer scientist) for example thinks that consciousness may be just a collection of illusions and tricks a mind plays with itself as it models different facets of and lenses into what it stores and perceives.
I think you missed my point. I think we're going in the wrong direction for AI entirely, and these "advances" are fundamentally misguided. OpenAI is explicitly about "intelligence," and so we should question if this is in fact that.
It's clear that humans have fundamental intelligence much better than all of this stuff with 6 orders of magnitude less input (at least of the same data sort) on a problem.
Perhaps it would be better to say, "I think the ML winter is just around the corner" as opposed to "the AI winter is just around the corner." That said, this really is math, and these algos still don't actually do anything resembling true intelligence.
This direction has produced results that eluded 30+ years of research. What is the evidence that this is the wrong direction?
IIRC the following is attributable to either Margaret Atwood or Iris Murdoch:
"A writer should be able to look into a room [full of people] and understand [in breadth] what is going on."
https://www.youtube.com/watch?v=dXQPL9GooyI
Of course evolutionary algorithms are just one direction as well. But that doesn’t mean that nothing else is happening.
>6 orders of magnitude less input
That is utterly mistaken.
We have the input of millions of generations of evolution which have shaped our brains and given us a lot of instinctive knowledge that we do not need to learn from environmental input that happens during our lifetime.
Instead it was learned over the course of billions of years, during the lifetimes of other organisms that preceded us.
Our brain structure was developed and tuned by all these inputs to have some built in pretrained models. That’s what instincts are. Billions of years in the making. Millions, at the very least, if you want to restrict it to recent primates, although doing so is nonsensical.
>That is utterly mistaken.
I did say of data of the "same sort".
What's absolutely crazy is somehow we think of our DNA base pairs as somehow more important than the physical context that DNA ends up in (society, humans, talking, etc.)
We have the ability to be intelligent and make thoughts with 1 millionth the amount of textual data as this OpenAI GPT-3 study. Maybe... just maybe... intelligence is far more related to things other than just having more data.
I'll actually expand on this and throw this out there: intelligence is in a way antagonistic to more data.
A more intelligent agent needs less knowledge to make a better decision. It's like a function that can do the same computation with fewer inputs. A less intelligent agent requires a lookup table of previously computed intelligent things instead of figuring it out on its own. I think all these "AI" studies are glorified lookup tables.
Note in particular that "like a function that can do the same computation with fewer inputs" maps very well to GPT-3 - it can complete many interesting tasks by just having a few samples provided to it, instead of having to fine-tune it with more training.
The reason it doesn't need more training is because it's already trained itself with millions of lifetimes of human data and encoded that in the parameters!
Humans aren't born trained with data. The fact that we're throwing more and more data at this problem is crazy. The compression ratio of GPT-3 is worse than GPT-2.
You know what else is trained by the experiences of thousands of individual (and billions of collective) human lifetimes of data? And several trillions of non-human ones?
> Humans aren't born trained with data.
That's either very wrong or about to evolve into a no true scotsman regarding what counts as data.
AKA "why is it so hard to swat a fly?" because they literally have a direct linkage betweeen sensing incoming air pressure and jumping. Thats why fly swatters don't make a lot of air pressure.
Why do you yank your hand back when you get burned? It's not a controlled reaction. Where did you learn it? You didn't.
If you think the brain is much more than a chemical computer you are sadly mistaken. I would encourage you (not really but it's funny to say) to go experiment with psychedelics and steroids and you will quickly realize that these substances can take over your own perceived intelligence.
The most fascinating of all of this is articles/documentaries about trans people that have started taking hormones and how their perception of the world -drastically- changed. From "feeling" a fast car all of a sudden, to being able to visualize flavors. It's absolutely amazing.