Open-interpreter: OpenAI's Code Interpreter in your terminal, running locally | Hacker News Reader

Open-interpreter: OpenAI's Code Interpreter in your terminal, running locally

github.com

117 pointstranspute58 comments

Wow, ok, bring on the downvotes, cause I've got plenty of nice stuff to say about Open Interpreter. I came on here to voice some criticisms but since everyone else is already on that bandwagon, I'm switching teams!

I had a great experience with Open-Interpreter. Watching it work with GPT-4 as LLM vendor is nothing short of magical. In fact it was the closest thing I've ever seen to pure science fiction in my lifetime, esp since I hooked it up to a microphone and I was controlling it with voice. "Computer, bring up Hacker News. Summarize. Make it so.".

And it WORKS. Good times! That is a trivial exercise for GPT-4!

No, it's not going to brick your computer. You should know by now that GPT-4 is so well aligned at this point that it's yawn inducingly boring.

Before Autogen, before LlamaIndex, before LangChain, etc, Open Interpreter was the first thing to convince me that LLMs had a use beyond mere novelty -- that they could be the engine for useful agentic software robotics. Up until that point it was all theory, I had never seen it.

In fact, the paranoia that it could brick your system or extract information from your machine requires you to concede that it could generate and run code. Which it can -- except it in fact tries to be helpful. If you tell it to brick your computer, I doubt it would even comply.

My criticism comes not from Open-Interpreter but from the fact that there are many products like this that only work with GPT-4, and that's a shame.

But it was extremely inspiring to, rather than sit around and complain about it, get to work building some models and systems that can actually be used to do useful stuff and deliver on the automation promised, and it has become a sustainable small business model for me. So, thank you killianlucas, for inspiring me and making me a lot of money.

So put that in your pipe and downvote it.

What's so useful about Code Interpreter? I'm getting a lot of value from normal Chat questions and DALL-E. I'm also using the chat interface to generate code sometimes. But I don't see much point in the chat bot itself running the code for me without my environment, credentials, data, etc.

Am I missing an interesting use case here?

Everybody knows GPT hallucinates - so occasionally it suggests code, you try it, and discover the functions or arguments it’s suggesting don’t exist.

Code interpreter takes you out of that loop - GPT can find and fix its own mistakes.

I’ve not found this to be a beneficial feature. Aside from being prone to erroring out, it tends to write code that will run, but is not very well-organized. You then have to revise. In the standard UI, I can include instructions to structure everything as functions or classes, alongside whatever other constraints, review and provide feedback, until it’s actually correct. This also almost runs for me on the first try.

Let's say you have a complex question that requires some math or difficult logic to solve.

The idea of Code Interpreter is that ChatGPT, instead of inventing/hallucinating the answer to your complex question, can write code to solve it.

ChatGPT is better at writing code that matches the logic of the problem, than it is at coming up with a straight-up reply.

So it comes up with the code, it runs it with Code Interpreter, and then reads the answer out loud to you.

GPT4 knows it sucks at math, but it's awesome at writing simple code, so it answers math problems by writing and executing simple scripts.

It’s handy for uploading a dataset and having it play with it.

Cool stuff. You can upload sensitive customer data, employer proprietary code. Sounds good to me.

Need to think twice before using any tool that isn’t running locally. OpenAI isn’t unique here.

Op wrote "uploading a dataset and having it play with it.", not that they would use it locally.

actually, this post is about a local replacement for code interpreter. the point is for you to not have to upload anything, but for you to download the model and run it locally on your computer.

I doubt this to be true openAI does not allow you to download models (with few exceptions)

This lets you use OpenAI’s models or local LLMs.

The API is governed by a different policy than ChatGPT and won’t be used for training data.

The original post clearly states that you may use any number of models, even a local LLM it tells you how to run using a simple CLI flag.

openai does not have a monopoly on large language models

It might not, but the post has "OpenAI's Code Interpreter" in the title

The readme in the linked repository explains what they mean:

> An open-source, locally running implementation of OpenAI's Code Interpreter.

> […]

> Open Interpreter lets LLMs run code (Python, Javascript, Shell, and more) locally. You can chat with Open Interpreter through a ChatGPT-like interface in your terminal by running $ interpreter after installing.

> […]

> Comparison to ChatGPT's Code Interpreter

> OpenAI's release of Code Interpreter with GPT-4 presents a fantastic opportunity to accomplish real-world tasks with ChatGPT.

> However, OpenAI's service is hosted, closed-source, and heavily restricted:

> - No internet access.

> - Limited set of pre-installed packages.

> - 100 MB maximum upload, 120.0 second runtime limit.

> - State is cleared (along with any generated files or links) when the environment dies.

> Open Interpreter overcomes these limitations by running in your local environment. It has full access to the internet, isn't restricted by time or file size, and can utilize any package or library.

> This combines the power of GPT-4's Code Interpreter with the flexibility of your local development environment.

> […]

> Change your Language Model

> Open Interpreter uses LiteLLM to connect to hosted language models.

> You can change the model by setting the model parameter

> […]

> Running Open Interpreter locally

> Open Interpreter uses LM Studio to connect to local language models (experimental).

> Simply run interpreter in local mode from the command line

It is OpenAI's model (although you can switch to another LLM). It's just that your data and the actual run environment for the code you got from the ChatGPT API, runs locally.

So AFAICT your prompt is still "leaked" to OpenAI, but not your data.

Probably splitting hairs and I'm not arguing one way or another, just wondering - why isn't the prompt itself considered leaked data?

Well I'm saying the prompt is leaked.

But that's not the biggest issue in most cases.

Prompt: "I have a file of 315 customers with their IP, behavior, religious affiliation and previous purchases plus medical history. I want to figure out if I have any customers whose religious affiliation pre-dates a post-surgery stay in hospital."

Data: a huge CSV file which, as you can tell, contains incredibly sensitive/legally impactful information.

With this system, you send the prompt to GPT-4 or Claude, and it doesn't see your data at all. It just writes a python program that can do the analysis.

You run the program locally on C:/Users/passwordoops/SuperSensitiveData/Confidential.csv

You get the result.

OpenAI has never seen your data.

It's a win for confidentiality.

(of course, that's if the LLM didn't give you a program that would exfiltrate your prompt, and if the CSV contains no prompt injection to exfiltrate the data, etc. This system is a security nightmare.)

chatgpt is bad at math.

Wait... so this is basically every sci-fi movie's nightmare scenario.

Cool idea on paper and there was no way people wouldn't do it.

Security-wise though, this seems like a pretty epically catastrophe-prone concept.

The safeguard is your diligence before approving its actions:

    Open Interpreter will ask for user confirmation before executing code.

That's security theater though.

It will protect you against an "honest mistake" like if you asked for code to delete all files with an "x" in the name and the command is deleting all files with a star "*" and you catch that. Great.

It will not protect you against actual malicious code. LLM inputs should be treated with the same caution and suspicion as unregistered-user inputs submitted over Tor.

I think maybe you haven’t played around with ChatGPT very much? Generating malicious code is possible, but unlikely unless you’re prompting for it.

I have. And I have already identified that it can generate, unwittingly, code or string values that break things or cause code injection if you don't treat its outputs as "untrustworthy" and just run them as SQL/insert to DB/display to user.

Also, it is very likely that a further GPT model might behave, with non-malicious prompt, like a malicious actor, simply because such an fairly-unlikely event is almost guaranteed at the kinds of scales we are talking (100MM weekly active users is a lot of rolls of the dice).

Yes, it could have security holes and it all needs to be reviewed. At scale, a lot of mistakes could pass code review.

But people make mistakes, too. The code that people write themselves or copy from Stack Overflow will have lots of bugs at scale.

For the individual programmer who knows what they're doing and uses it responsibly, it's unclear that the risks are any higher than writing code on your own.

unless you set the flag that lets it execute the code it gets

Why doesn't it mention the lmstudio dependency in the local docs here https://docs.openinterpreter.com/language-model-setup/local-...?

And the linux build must be requested from discord: https://lmstudio.ai/

For shell commands, there is a tool from Github that does it quite well (and it's included on the copilot plan)[1].

It's perfect to run fast bash commands that I forgot but are simple enough that the first Google search result would solve it anyways.

[1] https://www.npmjs.com/package/@githubnext/github-copilot-cli

I wrote a similar thing for myself: https://github.com/ricklamers/shell-ai

It currently has 882 stars and I think a few people at least are enjoying using it.

Got a laugh out of the portion of the demo video that had “shai do something crazy”

Coming back to say - thank you and awesome work!

Thanks! Appreciate it

I am so looking forward to someone releasing a version of this that runs code in a proper sandbox.

One easier option which would likely work: run Python code entirely inside a WebAssembly sandbox directly in the browser using Pyodide. https://pyodide.org/

Anyone seen an attempt at that yet?

I've built exactly this as an experiment (code interpreter with pyodide sandbox). Should probably tidy it up to open-source it.

> Let language models run code on your computer.

No, I don’t think I will.

For anything remotely complex, even gpt-4 hallucinates libraries and commands.

> Note: You'll be asked to approve code before it's run.

I don't know about you, but my malicious code detector isn't anywhere close to 100%.

You're probably fine with code generated from direct use of GPT-4 - it's clearly not smart enough to be dangerous. But it's not going to be the case in the future, so best not get into the habit.

> it's clearly not smart enough to be dangerous.

The two most dangerous types of programmers are experts and novices.

I think you're forgetting the latter. It's the reason there's so many "how to exit vim" jokes and why every linux noob (that actually gets into linux and not just "oh, I installed ubuntu and only use the GUI) deletes shit they needed and learn the importance of backup the hard way. Hell, even being on systems like this for over a decade and knowing them really well I sometimes make mistakes. I have a bunch of aliases and routines literally to prevent accidents.

You’d need another LLM to check the code before running! LLM squared

This, but unironically...

You can run it in a container

How is this different from using OpenAI's Function Calling API: https://platform.openai.com/docs/guides/function-calling

... and running exec() on the resulting code?

Open Interpreter had one of the first working implementations of a very good way to interact with your computer just by typing prompts. Their canonical example was changing your desktop background, which, while a simple example, is a cool demonstration of something that previously was hiding behind AppleScript incantations...

changing backgrounds, cool… what else that is useful?

for those of us too unimaginative to draw the rest of the owl

The rest of the owl is all of computer interaction, as a human that usually has to do the whole interaction themselves

did you try watching the demo video?

If you’d rather run code in a Jupyter notebook, I wrote a VS Code plugin for that:

https://marketplace.visualstudio.com/items?itemName=skybrian...

Yeah, this is how ASI will get us.

Next logical step: make it recursively self-improve.

"Here, read the current directory. See that app.py? Yeah, that's you. The sqlite is in the parent directory. I gave you 16 tools. Go forth!"

Im confused. Is OpenAI open source?

No... this library calls gpt (or claude, or another hosted LLM) and asks it to write the code.

The code is then retrieved to your computer and run locally.

This way you can run the code on local files or give it access to the internet. Basically, this is Code Interpreter with none of the security features.

Why do they call their company “OpenAI” then? I mean what’s open about it?

Feels like very dangerous, if I ask it to delete my home folder I guess it will delete it right away

This looks like a great way to brick your machine. I'm not sure how everyone gets high quality code from GPT, but I can't get 3.5, 4, Falcon, LLaMA, or whatever to even do things like make ridge plots. Maybe I suck at prompting, but even assuming that's true doesn't mean this doesn't have a good chance of bricking your machine. I hope people properly sandbox this, but the first thing in the demo is switching the UI to dark mode, so I'm assuming people aren't going to do that.

Reply on news.ycombinator.com