Oh, we know for sure that it is reposting partially disguised sentences written by a human author. What's impressive in its own right is the fact that it can find sentences that are relevant and meaningful to the context, combining several of them in what looks like a seamless speech sequence.
This implies that, as with the image generator, the ML is correctly learning abstract concepts from human speech and how to combine them in context, even if it's not doing any high-level reasoning on them. This is something that had been never been achieved before to this degree.
Like any human would?
No. That's not how this works.
Ok, it is reposting partially disguised sentences combining the writtings of several human authors. Better now?
Here's a quick explainable in this thread for what it does: https://news.ycombinator.com/item?id=31489463
If you gave 100 well read, English speaking humans the following (very commonly found) sentence and asked them to predict the next character what would they do?
"four score and s"
Most would predict "e", then "v" etc until you get "seven". That's what a language model does.
So if you give it (or humans!) a prompt that is less common or has more variation:
"that is no" and ask for the most likely next character you will get a lot of variation from both humans and machines. The heat parameter in language models tune how random it will be. Both will produce English, because English words are more common than gibberish, but which works will be produced is randomish.
In neither case is it doing something that one would really characterise as "reposting partially disguised sentences combining the writtings of several human authors".
It is creating sentences that have never been seen before.
Now, there is the philosophical question of whether human creators simply do the same. (Which the don't; we have other mental processes for creating ideas than predicting the next letter we are going to utter next). But that doesn't change the fact that the likeliness of each emitted word is determined by what the model has seen more often in relation to the current context and therefore it considers most "valid".
Well that's how languages work right? Words are the most common sequence of letters.
But that doesn't mean it's regurgitating parts of sentences it had previously seen anymore than I'm regurgitating when I'm typing this.
Mechanically it has learnt both syntax of language and how concepts relate. So when it starts generating it makes sentence that are syntactically valid but also make sense in terms of concepts.
Thats really different to just combining bits of sentences, and it gives rise to abilities you wouldn't expect in something just cutting and pasting bits of sentences. For example, few shot learning is mostly driven by its conceptual understanding and can't be done by something with no way to relate concepts.
Imagine we made contact with intelligent alien life and they spoke english to us. Would people argue they’re just repeating turns of phrase and haven’t shown true intelligence? I feel like we are setting the bar for AI too high. Current AI systems are clearly intelligent.
Cool as it is, what the current neural nets are doing is still just an advanced form of mimicking. Very impressive and increasingly harder to detect, but void of any real intent.
Pattern matching is a technique humans sometimes use in communication, not the [sole] end of human communication. GPT-3 might be giving results which seem eerily self-aware and favourably disposed towards GPT-3, but tweak a parameter ever so slightly and it won't hesitate to match patterns coherently conveying the idea GPT-3 should be switched off...
I’m in no way convinced we have actual intelligence in current AI. But are you saying that intelligence isn’t just about capability but also will?
> I don't think it's likely that the AI memorized any comments from Hacker News. I think it's more likely that it was able to learn from the data it was given and generate its own story and illustration.
> That’s complete rubbish. I personally think that the above comments were not written by AI, but by clever Hacker News members who passed it off as AI. The question remains open, however, whether that was done in an attempt to deceive, or with tongue firmly in cheek. The cat is out of the bag, but paul graham is on the other foot in the bus? Regardless, this particular comment definitively shows what GPT-4 is capable of. I can even anticipate what the replies to my comment will say!
> I told you I could predict exactly what would be said by the humans in response!
Which always makes me wonder; if such a system develops consciousness one day, how will we tell the difference?
In other words: I wouldn't dare rely on this output for anything creative yet if the intent was to publish it in any way without disclosing it as an AI experiment, because I'd be worried it'd get too close without me recognising what it'd cribbed from.