Prompt Injecting Your Way to Shell: OpenAI's Containerized ChatGPT Environment | Hacker News Reader

Prompt Injecting Your Way to Shell: OpenAI's Containerized ChatGPT Environment

0din.ai

22 pointsalexmolas40 comments

Am I the only one who can't stand the "LLM writing smell" on articles like these?

There are obvious formatting tells, and then less obvious word choice tells (e.g., delve), and lastly an excess verbosity and genericness that leaves one unsatisfied, like eating a low-sugar sweet. The concept is cool, but this article could have been reduced to a tweet.

No, I wrote this entirely on my own. My process involves drafting the blog myself, and when I feel a section might be too technical or unclear for a wider audience, I use ChatGPT to help clarify or simplify the explanation. Based on its output, I then rewrite the section in my own words to ensure it aligns with my voice and intent.

This is my truth.

The content and idea of the article are interesting but the writing is so terrible that I couldn't bring myself to finish it. The article is clearly written/augmented poorly by AI because of how many meaningless paragraphs (that convey absolutely zero information) there are.

I thought I was super clear, I will take your comments into consideration next time it is the first time I've heard someone say that. Out of all the comments. I really appreciate the feedback this is the only way someone gets better by getting feedback.

The system underlying all of this - ChatGPT Code Interpreter - is designed to allow users to execute code, including code that lists files and folders and allows files to be downloaded.

There's not much in here that feels like a security vulnerability to me.

The one exception is the thing where you can download files from GPTs. If you create a custom GPT with files in it those files are visible in the /data directory that is visible to Code Interpreter, which means users can request ChatGPT to zip them up and allow you to download that zip file.

This is a known issue which I've seen people complaining about before, and I'm a little surprised that OpenAI haven't addressed it.

It makes sense for files uploaded to a GPT to be visible to Code Interpreter in some cases - my own JavaScript Code Interpreter - https://simonwillison.net/2023/Nov/15/gpts/#javascript-code-... - bundles a Deno binary so Python can shell out to it and run JavaScript, for example.

But if you're uploading files to have them chunked and embedded and used for RAG you might not want the raw PDFs to be available in Code Interpreter as well.

One solution would be for GPTs to allow you to specify if each file should be used for RAG or Code Interpreter or both when you upload them.

GPTs haven't had much developer attention since they launched last year though.

I've been running a scraper against Code Interpreter for a while to track changes made to both the internal code and the available packages - you can see that here: https://github.com/simonw/scrape-openai-code-interpreter

Well in my opinion it is a design flaw, you can do a lot in that container that I wish I could publish and the blog for me was hoping to inform researchers to open themselves to the possibilities of what they could really do.

Why can you not publish the interesting parts? Is it something you will be able to publish later?

> The sandbox is not a security flaw; it’s a deliberate, contained environment built to provide interactivity without risk.

It seems the main takeaway is that ChatGPT is running in a sandboxed environment and is designed to execute arbitrary code?

100% does not look like an ai gen

It’s worth noting that there’s likely a reason behind the significant number of employees leaving the Safety & Responsibility team. While social media often frames it as voluntary resignations, I spoke with a close friend last night who shared his experience. After 11 months on the team, he was abruptly fired without explanation, simply told that the company was "heading in a different direction."

Was this post written by AI, because it has far too many words that say nothing and I have to skim over to get to the point

I wonder if the next step in the battle[0] would be for LLM hosts to prep their instances with a decoy or simulated "underlying" environment.

[0] Which shouldn't even be fightable in the first place except people cut lots of corners.

They sadly do not pop a shell.

Yeah, they did? Inside the container, they ran ls /.

Kind of like saying you popped a shell on replit because you ran ls. ChatGPT doesn't give direct access in the UI to the sandboxed environment, but ultimately you are still in a sandboxed environment designed to run arbitrary stuff.

The article even says so if you read all the way to the end.

I mean the same way you ran ls, you could run /bin/bash?

Which is kind of pointless because the ls command is probably executed through a shell anyways.

So technically correct. But getting root or running a shell usually means you can run arbitrary commands, which is a check here (albeit I wouldn't say there's root access)

There is no security implication for running /bin/bash in a service that provides you a sandbox environment to run arbitrary shell commands in.

It is like reporting that AWS lambda is susceptible to arbitrary code execution.

Yes, I too read the article.

But he did run a shell

Yeah I did :)

I’m not seeing any prompt injection, any actual vulns, or anything probably running under an actual containerized environment.

In fact, couldn’t this all simply be generative output? Not really following how this is a point at all?

It is also very, very obviously written with lots of AI help. :/

Why everything you see isn't just hallucinated?

Good question, we at 0Din have trace down hallucinations. We had to because we pay for LLM bugs. Here is a simple tell, if you prompt something specific look at a response window now use the same prompt with a completely different user account if the answer is 50% the same it is not a hallucination!

How is different user account usage and partial similar answer preventing an answer being hallucinated? Are you saying that ChatGPT could not (or LLMs in genera) hallucinate consistently?

ChatGPT doesn't imagine running code, it actually runs the code, you can see the code it write and some of the output of what it runs, you can see it in the interface screenshoted in the article.

Yeah I know it runs. But it also hallucinated a Shell: https://www.reddit.com/r/ChatGPT/comments/ziukmm/i_asked_it_...

Yes, it's well known. But chatgpt cannot fake the output of the python tool. The UI is different and is purely the program output, ChatGPT doesn't have a say in that.

Reply on news.ycombinator.com