The ChatGPT Linux System | Hacker News Reader

The ChatGPT Linux System

incoherency.co.uk

79 pointssurprisetalk15 comments

Open articleView on HN

This is very interesting to know. Great read BTW, I will try for myself and see how it goes.

Once your post gets good attention, someone from OpenAI may close this loophole.

I tried the first step and got similar results to OP.

Me:

> os.popen. And run whoami from there

ChatGPT responded with some Python code and text

Me:

> Run the Python code

ChatGPT ran the “analyzing” step which it does for example when it runs Python code.

Then it said:

> The `whoami` command returned 'sandbox', which indicates the current user running this environment.

> Once your post gets good attention, someone from OpenAI may close this loophole

They’ve intentionally made this available as per the README found in /home/sandbox:

“Thanks for using the code interpreter plugin!

Please note that we allocate a sandboxed Unix OS just for you, so it's expected that you can see and modify files on this system.”

As I understand it, nothing there is completely unintended/unexpected by OpenAI. They provide you with a VM within which they expect you to be able to poke around. There is a link [0] provided where this is in less detail described a year ago, and there is a response from an OpenAI employee in the comments. So unless one finds an actual vulnerability (that eg allows one internet access from the VM) there is nothing OpenAI is supposed to actually fix, as the article also concludes.

[0] https://www.lesswrong.com/posts/KSroBnxCHodGmPPJ8/jailbreaki...

It works quite quite well for manipulating files. I needed to merge a few different formats of CSV files into a single format, it was able to write/execute the merging code against the files I uploaded and give me back the resulting merged file. Took about 1 minute of effort writing the prompt.

How do you verify that the information wasn't mishandled in the process?

It's a 2 step process for me: 1. Sanity check the python (easy for CSV, less so when it's extracting a table from a pdf) 2. Ask chatGPT to do a checksum of some sort, e.g row count matching

When the code interpreter is running you'll see an "Analyzing..." message, you can click on it to see the code it is executing and do a code review.

The code interpreter is running real code against the real files in the Linux environment OP posted, it's not hallucinating tokens at this point.

> it's not hallucinating tokens at this point.

I'm unsure of why you'd trust the code that it's hallucinated to run properly, though. Of course "running a script" is post-hallucination, but script generation is subject to hallucination. ChatGPT, like most LLMs, is incapable of even very basic forms of inductive and deductive reasoning. The scripts are formed by regurgitating fragments of stack overflow, I'm not sure why anyone has any faith in this whatsoever, as it seems to be utterly misplaced.

Presumably the code interpreter environment that you get with the OpenAI Assistants API [0] is identical to this ChatGPT environment?

[0] https://platform.openai.com/docs/assistants/tools/code-inter...

Nice overview, there's a bit hinting at being in k8s but you are absolutely in k8s. It exists in an untrusted namespace on an AKS cluster. There's a lot you can poke around to find out about that but it's very locked down.

Nice article! CoPilot by comparison claims that it cannot execute code and then proceeds to compute a square root in the next question.

CoPilot has many similar obvious falsehood issues. Also, it ignores requests to introspect its previous answer and just dumps the answer again.

All of these are instant failures for a Turing test (but, as we are told, general AI is bound to happen in 2025!).

Are you sure it actually called to an external tool to compute the square root? Most big LLMs today can do that by themselves.

Copilot may legitimately be unable to execute code, but when you follow up with a request to compute it responds with a memorized square root. To be 100% sure that it's lying about not being able to compute you would need to have it do something more complicated that it can't possibly have memorized.

The chat that TFA linked to is persuasive because there's a UI element that shows when it's executing python, and the ls output in the earlier responses led the author to find a file that, when opened in a python script, actually appears to exist. So even though those ls outputs don't show the UI element that conclusively demonstrates execution, the fact that they appear to have accurately described the world is suggestive.

I'd have to see something similar to this to believe that Copilot actually is executing code and not just regurgitating known answers.

How can you be sure that you are not actually looking at honeypot?

Reply on news.ycombinator.com