A few little things are weird (I can exec into a stopped container for example) but I was able to start another container and persist files.
Wild. This is unbelievable. Can anyone please explain to me why this isn't as wildly groundbreaking as this seems?
Obviously what's happening is much more complex, and impressive, than just spitting back the exact things it's seen, as it can include the specific context of the previous prompts in its responses, among other things, but I don't know that it's necessarily different in kind than the stuff people ask it to do in terms of "write X in the style of Y."
None of this is to say it's not impressive. I particularly have been struck by the amount of "instruction following" the model does, something exercised a lot by the prompts people are using in this thread and the article. I know OpenAI had an article out earlier this year about their efforts and results at that time specifically around training the models to follow instructions.
It is and people haven't realize it yet.
It is literally years - possibly decades - ahead of my prior expectations.
It’s something like novelty * length * complexity with * accuracy that impressed me, of which it’s not far beyond simple tutorials or snippets you’d find online.
Statements like "the people haven't realized it yet" confuse me because "the people" is two groups. People in the know, and people not in the know. Everyone in the know realizes where this is headed and what the potential is.
Those not in the know simply lack the technical background to have followed the incremental developments up till now which have led to this moment, for them it's a parlor trick because even today they cannot grasp the potential of existing technology. I could similarly lament about how people treat the Internet.
But for anyone paying attention, it's been easy to see the progression. I'm not even an ML person but I could give you a map from every paper to every other paper for how this has all been happening faster and faster, basically starting with AlexNet in 2012.
That said, this chatGPT is different than GPT-3's first demos earlier last year or the Codex interface in that it is implementing a consistent memory and seems to have a token length capability much, much longer than before. This is having a huge effect on what you can coax out of a chat with it. You can tell it to act a certain way and then continuously interact with that entity- with GPT, you got the one prompt, but once you tried again with a new prompt that memory was gone. You could attempt to feed the entire output back in as input, but at least initially the token length would cut things off eventually. Meanwhile, with chatGPT, I just had a 20-minute conversation with a "girl from Reseda, CA' who's a barista and like, totally is going to go on a keto diet like her sister. " because I told it that is who it should act like it is and under all circumstances it should respond to my chat in that way.
BTW she says that "bangs are totally in style right now" and she really likes "exploring new hairstyles like ones from the 90's"
So yeah. This is shaking people up.
Did you see the demo the other day that was posted here of using stylographic analysis to identify alt accounts? Most of the comments were some form of "holy shit this is unbelievable", and the OP explained that he had used a very simple type of analysis to generate the matches.
We aren't quite as unique as we think was my takeaway from that. My takeawy from this, as well as the SD, DALL-E stuff is that we're all just basically taking what we heard from the past, modifying it a teeny bit, and spitting it back out.
…but people are getting the mistaken impression that this is an actual system, running actual commands.
I can also emulate a docker container. I’ll just write down the commands you send me and respond with some believable crap.
…but no one is going to run their web server on me, because that’s stupid. I can respond hundreds of times a second and maintain the internal state required for that.
Neither can this model.
It’s good, and interesting, but it’s not running code, it’s predicting sentences and when you’re running software it was to be accurate, fast, consistent and have a large internal data state.
Trying to run docker in gpt is fun. Trying to use docker in gpt to do work is stupid.
It’s never going to work as well as actually running docker.
It’s just for fun.
Models that write code and the execute that code will be in every way superior to models that try to memorise the cli api of applications.
It’s an almost pointless use of the technology.
Gpt may have “learnt” python; that’s actually interesting!
Docker is not interesting.
If I want to use the docker api, I can type `docker` on my computer and use it.
It's pretty sad that the thing that excites people the most about an amazing new language model is that it can do trivial command line actions, that you can do without the model.
Spending millions of dollars to produce a model that can do what you can already trivially do is very seriously not what openai just did.
Right. The thing that is impressive is that ChapGPT can do this effectively. This means that it has some "understanding" of how `pwd`, `ls`, `apt`, `docker`, etc all work. In some sense, this is an AI that knows how to read code like a human instead of like a machine.
It's literally spitting out responses like a machine. Isn't that the opposite of what you wanted?
> The thing that is impressive is that ChapGPT can do this effectively.
? What is impressive about it?
Forget this is an AI model for a moment. Lets say I give you a black box, and you can type in shell commands and get results. Sometimes the results don't make sense.
Are you impressed?
I am not impressed.
I could implement the blackbox with an actual computer running and actual shell and the results would be better. Why would I ever use a LLM for this?
It's like discovering that the large hadron collider can detect the sun. Yes, it can. Wow, that's interesting, I didn't realize it could do that. I can also look up at the sun, and see the sun. mmm... well, that was fun, but pointless.
There are so many other things GPT can do, this... it's just quite ridiculous people are so amazed by it.
It is not indicative of any of the other breakthrough functionality that's in this model.
The point is that the future programming language will probably be the human language as an extremely high-level specification language, being able to hallucinate/invent/develop entire technological stacks (from protocols to operating systems to applications) on the fly.
I think it means that you're able to apply the information to make predictions about the world. For example, you'll encounter something novel and be able to make accurate guesses about its behavior. Or, conversely, you will have high likelihood of inventing something novel yourself, based on the information you acquired (rather than through brute force).
Similarly in my explorations of this ‘pretend Linux’, it often produces whatever would be the most helpful output, rather than the correct output.
It’s really hard to utilize if the results aren’t consistent.