Hacker News Reader

Comment by TeMPOraL | Hacker News Reader

> If you leave chatGTP alone what does it do? Nothing. It responds to prompts and that is it. It doesn't have interests, thoughts and feelings.

A loop that preserves some state and a conditional is all what it takes to make a simple rule set Turing-complete.

If you leave ChatGPT alone it obviously does nothing. If you loop it to talk to itself? Probably depends on the size of its short-term memory. If you also give it the ability to run commands or code it generates, including to access the Internet, and have it ingest the output? Might get interesting.

View on HN

> If you also give it the ability to run commands or code it generates, including to access the Internet, and have it ingest the output?

I have done that actually: I told ChatGPT that it should pretend that I'm a Bash terminal and that I will run its answers verbatim in the shell and then respond with the output. Then I gave it a task ("Do I have access to the internet?" etc.) and it successfully pinged e.g. Google. Another time, though, it tried to use awscli to see whether it could reach AWS. I responded with the outout "aws: command not found", to which it reacted with "apt install awscli" and then continued the original task.

I also gave it some coding exercises. ("Please use shell commands to read & manipulate files.")

Overall, it went okay. Sometimes it was even surprisingly good. Would I want to rely on it, though? Certainly not.

In any case, this approach is very much limited by the maximum input buffer size ChatGPT can digest (a real issue, given how much some commands output on stdout), and by the fact that it will forget the original prompt after a while.

Wait, wait, this is not an accurate interpretation of what happened.

> I told ChatGPT that it should pretend that I'm a Bash terminal and that I will run its answers verbatim in the shell and then respond with the output. Then I gave it a task ("Do I have access to the internet?" etc.) and it successfully pinged e.g. Google.

It did not ping Google - it returned a very good guess of what the 'ping' command would show the user when pinging Google, but did not actually send a ICMP packet and receive a response.

> Another time, though, it tried to use awscli to see whether it could reach AWS. I responded with the outout "aws: command not found", to which it reacted with "apt install awscli" and then continued the original task.

You were not able to see whether it could reach AWS. It did not actually attempt to reach AWS, it returned a (very good) guess of what attempting to reach AWS would look like ("aws: command not found"). And it did not install awscli package on any Linux system, it simply had enough data to predict what the command (and its output) should look like.

There is an enormous semantic difference between being able to successfully guess the output of some commands and code and actually running these commands or code - for example, the "side effects" of that computation don't happen.

Try "pinging" a domain you control where you can detect and record any ping attempts.

I believe the op was being the device with the function.

The OP writes a script which asks chatgpt for the commands to run to check your online then start to do something. Then execute the script. Then chatgpt is accessing the internet via your script. It can cope with errors (installing awscli) etc.

The initial scout would send “build a new ec2 instance, I will execute any line verbatim and I will respond with the output”, then it’s a “while (read): runcmd” loop.

You could probably bootstrap that script from chatgpt.

Once you’ve done that you have given chatgpt the ability to access the internet.

> It did not ping Google - it returned a very good guess of what the 'ping' command would show the user when pinging Google, but did not actually send a ICMP packet and receive a response.

Yes, it did ping Google and it did receive an actual response. My apologies for not phrasing my comment as clearly as I should have. Here are some more details to explain what I did:

https://news.ycombinator.com/item?id=34835214

This is a huge misunderstanding of what happened. You gave it prompts, and it found examples of similar text in its database and extrapolated what was likely to follow. No ICMP packets were sent to Google.

Maybe I wasn't being clear enough:

I asked it to pretend that I'm a Linux terminal, ChatGPT gave me shell commands, and I then ran those commands inside a terminal on my computer (without filtering/adapting them beforehand), and reported their output back to ChatGPT. So, effectively, ChatGPT did ping Google – through me / with me being the terminal.

With long-running sessions, it helps to tell it to repeat or at least summarize the original prompt every now and then. You can even automate it - in the original prompt, tell it to tack it onto every response.

Same thing goes for any multi-step task that requires memory - make it dump the complete "mental state" after every step.

Oh, I am aware of that but emulating a terminal still proved to be difficult with the current buffer limit. After two or three commands with lots of output, you basically had to start a new session and repeat the prompt (and how far it got in the previous session) all over again.

You're giving it tasks, though. That's a bit different than "would it give itself tasks if it talked to itself instead of a human" by itself, to try to see what sort of agency it can or can't exhibit.

Absolutely! I was merely talking about the purely practical/technical issues of letting it talk to a terminal (or anything more complicated like the internet).

In any case, once there's a decent (official) API we can then have ChatGPT talk to itself while giving it access to a shell: Before forwarding one "instance"'s answer to the other, we would pipe it through a parser, analyze it for shell commands, execute them, inject the shell output into the answer, and then use the result as a prompt for the second ChatGPT "instance". And so on.

>If you leave ChatGPT alone it obviously does nothing. If you loop it to talk to itself?

then it degrades very quickly and turns into an endless literal loop of feeding itself the same nonsense which even happens in normal conversation pretty often (I've actually done this simply with two windows of ChatGPT open cross-posting responses). If you give it access to its own internal software it'll probably SIGTERM itself accidentally within five seconds or blow its ram up because it wrote a bad recursive function.

As a software system ChatGPT is no more robust than a roomba being stuck in a corner. There's no biological self annealing properties in the system that prevent it from borking itself immediately.

Reply on news.ycombinator.com