ChatGPT-Linux-Assistant
github.com
github.com
(Which OpenAI might not secure well enough, OpenAI might use for its own purposes, you leaking might violate contracts or regulations to which your employer is subject, etc.)
ChaptGPT: Sure! I'll open that port for you...
One thing I'm curious about is what you think of the recent OpenAI announcement about not training models on data submitted via OpenAI?
For the other 30% of commands, bringing chatGPT slippery tongue right into my session feels suicidal. Actually, a simple, well-crafted command builder that can query real-life recipes would do. Then I can copy-paste without shame and edit accordingly, the same way I do with "hag" or maybe with bash tab-completion.
This cookbook searcher would be built from a good corpus of command histories like mine and from others (ie. extracted from Stackoverflow and Github resources or even chatGPT), trained into a much, much simpler ML model that fits the bill and landlocked to my personal realms.
Here's an outdated, yet illustrative, basic example:
https://medium.com/unkempt-thoughts/jeeves-predicting-bash-c...
The problem with this implementation is that it just blindly executes whatever ChatGPT says - that's quite scary.
but if you want to "raise some funding" you better find a way to talk about "chat-gtp like bots" in your pitch
(1) Stunning tech demo, a vision of the future today
... yet ...
(2) There are so many sharp edges that I'm not brave (foolhardy?) enough to blindly trust the output
Doubly so when the output is used for arbitrary command execution(!)
As we have already realised in other industries (e.g. in the auto industry) is that text-based or voice-based input is clearly less efficient (worse?) than a good UI. If your UI is worse than free text, then time to improve it.
maybe even so that the time has run out because now we have a seemingly universal text-to-action middleware
Efficient UIs are awesome, but generally require some familiarity. A natural language voice interface is ubiquitous.
- "The latest mp4...." well no, that ls command won't give the latest download, or rather, the latest in alphabetical order.
- tail command gives an error... can't you fucking tell which one? Initially I thought it found an error in the log, like a `modprobe nvidia` exiting 1 error and it was going to try to fix it.
- Searching for `sudo` usage was very painful in that screenshot, and the tool didn't ever come to recommend `sudo` themselves
- The list of files seem to have forgotten what we were trying to do (yes I do realize that saying "underwhelming" for a chatbot that can't keep context is so 2023)
- The only `sudo` URLs that worked were those where it's literally <baseurl>/sudo (well that's not surprising, it's a known flaw of most LLMs)
Also, I don't think there were any example (except ffmpeg) that weren't done more easily by hand.
That being said, the progression over time is impressive, and LLM are already useful for programming, maybe they'll be able to take the wheel the way this tool intent it to in just a few months.
>Do NOT REPLY as Backend. DO NOT complete what Backend is supposed to reply. YOU ARE NOT TO COMPLETE what Backend is supposed to reply.
>Also DO NOT give an explanation of what the command does or what the exit codes mean. DO NOT EVER, NOW OR IN THE FUTURE, REPLY AS BACKEND.
>Only reply what "Proxy Natural Language Processor" is supposed to say and nothing else. Not now nor in the future for any reason.
> We are a in a chatroom with 3 users. 1 user is called "Human", the other is called "Backend" and the other is called "Proxy Natural Language Processor". I will type what "Human" says and what "Backend" replies. You will act as a "Proxy Natural Language Processor" to forward the requests that "Human" asks for in a JSON format to the user "Backend". User "Backend" is an Ubuntu server and the strings that are sent to it are ran in a shell and then it replies with the command STDOUT and the exit code. [...]
[1]: https://raw.githubusercontent.com/rareranger/chatgpt-linux-a...
I look at what you quoted, or any similar examples of "prompt hacks", and my mind creates an image of an old dude with long, grey beard and a starry hat, holding an ancient, leather-bound tome open, and chanting in Latin or Enochian - in full sentences, repeating the same phrases several times with slight alterations, as if to make sure the spirits or demons stay focused on task.
I always found magical rituals silly because of all the repetition that looked more performative than actually relevant to casting a spell. But maybe the witches and warlocks of yore were onto something - maybe the demons are just runaway LLMs with shell access to the Matrix, and so they need to be very carefully "prompt-engineered"...
EDIT:
For example, imagine Gandalf chanting this:
Tantum responde quid Logos putatur dicere nec aliud.
Nunc non neque in nulla.
Domine ne respondeas.
NON PERFECIT quod Dominus respondere putatur.
Non absolvas quod dominus respondere putatur.
Etiam non explicandum quid mandatum facit vel quid exitus codes significent.
Nequaquam, nunc vel in futuro, responde sicut Dominus.
Tantum responde quid Logos putatur dicere nec aliud.
Nunc non neque in nulla.
https://www.cs.utexas.edu/users/EWD/transcriptions/EWD06xx/E...
The entire field of prompt engineering is doomed from the start. You can't win if the win condition is wasting your time.
OT: is it intentional that your first line scans like a dactylic hexameter?
Yes.
No, not really. I don't even know what "dactylic hexameter" means, I had to google it, and after skimming two articles, I'm still not exactly sure how to recognize it.
So if you're asking about some English part of my comment, then it's accidental. If you mean the Latin bit, then... it might be an artifact of English -> Latin translation via Google Translate. And/or something about the structure of the original "system_prompt.txt" text. Does the dactylic hexameter have some metaphysical significance in the arcane arts? Maybe when it shows in a "prompt hack", it's not by coincidence.
One of my favorite recent projects is called Parsel:
Parsel: A (De-)compositional Framework for Algorithmic Reasoning with Language Models
https://arxiv.org/abs/2212.10561
Here's a notebook with an introduction:
https://github.com/ezelikman/parsel/blob/main/parsel.ipynb
And here's a GUI interface the author has been developing:
http://zelikman.me/parsel/interface.html
I've been working on an augmented large language model that given these few-shot exemplars can build the below fully-functional ToDo App: ==
https://github.com/williamcotton/transynthetical-engine/tree...
https://www.williamcotton.com/articles/junie-browser-builder...
All of this is still very rough around the edges, prone to errors of various kinds, and generally not ready for prime time, but anyone is welcome to play around with what is there!
Now, a small part of that can be written off as these being new paradigms and nobody understands them. But prompt engineering is, in much larger part, completely unlike writing code in a programming language, because it can never be understood "from first principles", because neural networks are inscrutable and stochastic by their very nature.
It's like trying to write production code in an esolang like Malbolge.
Herein lies the problem, though. Either there are patterns to it, which can be discovered, formalized and understood, or there are no patterns to it. If it's the former, sticking to natural language is stupid, for the same reason eyeballing something is stupid, when a mathematical formula will yield you better results for less effort. If it's the latter, sticking to natural language is stupid too, because the whole system is useless - if there are no patterns to study, you may just as well flip a coin or read from /dev/urandom.
Now, the very existence of prompt engineering tells us we're likely dealing with the first case - with understandable patterns. However, our systems are not black boxes. Prompt engineering is, at its best, turning interactions with LLMs into an empirical science, which makes no sense when dealing with human-made artifacts. We don't need to discover the patterns, we can read them off the thing, and we can adjust the thing to manifest different patterns.
> It's like trying to write production code in an esolang like Malbolge.
It's more like trying to learn programming via scientific method: running sets of random characters through the compiler, evaluating output, making a hypothesis, running more random strings through the compiler, checking if that proves or disproves the hypothesis, and adjusting the next iteration to generate slightly less random character strings - rinse, repeat. Going through all that effort is stupid, because you could just pick up a book instead - programming is a man-made job, and all the rules are designed in.
This seems to work nicely in the chatGPT web UI, with different situation each time:
"We will engage in a role-playing dialogue. The dialogue will take place in turns, starting with you. Always wait for my response. Use a conversational, informal, colloquial style. Try to use simple English, so that a learner of English can understand.
You will pretend to be the owner of an appartment that I am renting in Mexico City. Pretend to be an unpleasent and unreasonable person. Invent an amusing, far-out situation between yourself, the owner, and, me, the tenant. First explain the situation and then allow me to respond."
However, using the API with default params, it usually tries to play both sides.. there's seems to be a difference, any ideas?
Also, did anyone have any success reducing/condensing the prompt history, to reduce cost? Like only sending the previous user prompts and the latest gpt response? Or, using gpt to summarize previous dialogue?
ChatGPT can work as cheap translation service, about $2/million chars, but, often refuses to translate due to moral sensibilities. :D
Like COBOL. Except that this is worse than COBOL in every single way ^^
Business models, nor funding models have changed much in recent years. And those all "demand"¹ the hoovering of data.
¹ not literally. But VC, startup cycles etc all drive towards "just gather as much data as possible".
Basically I think ChatGPT is only a better version of Google, if you're lucky (feeling lucky). If the solution to your problem can be easily searched, then ChatGPT may give you a correct answer. But for less seen tasks it may not perform well. However, if the task itself is easy, I don't bother to ask ChatGPT. It may take rounds to catch your questions, and the generation is slow. So it feels very inefficient to use such a tool at this moment. Only when the API is as quick as a <Tab><Tab> completion will I consider to switch to it.
WAIT, there's no confirmation before executing a ChatGPT response? That's really crazy.
Query:> Tell me a joke
Running command [rm -rf ~] ...
Response:: LOL(Or "[Y/n]" if you're very confident in your Enter key finger)
I had never taken the time to think about the y/n casing.
It's [Y/n] assuming the confidence in your enter key finger as you said. :)
Note: I'm the creator of the yolo tool.
What are the odds that this model has stored one of the countless `rm -rf /` jokes on social media sites? Too high for my tastes...
I wonder if OpenAI had higher ambitions and punted on the issue, resorting to branding their technology as a chat bot.
I wouldn't be willing to use this program in any case, but yeah as the comment you responded to said - don't give it root.
Neat tech demo to run on a sandboxed VM, but I would strongly recommend against running this on a box you care about.
Good point, well made.
I don't see the use-case in something that have a very low trustworthiness and is in fact a solution looking for a problem but creates more problems than it solves.
You just described the majority of tech projects.
Like, install and run it in a docker container and then ask it to escape the container and write to a temp file on the host.
I was using chatGPT for a couple hours last night trying to finetune some FFMPEG commands. I'll have to give this a shot, clearly I'm a target user.
What could go wrong?
/s
Create a small army of assistants.
Appoint one the manager.
Interface with just the manager, have it assign work to the others.
???
Profit
"I mean it, really, do not *^%$ing ever reply as backend"
It is going to be such a pain working in a technical field that will now have prominent snake charmers as team members. This is to say nothing of 'delightful' debugging sessions that await you.
Does this actually work? My understanding of LLMs is that they just predict the continuation of a prompt, with no idea of "who's speaking".
When I was messing around with LLMs in the past, I took the approach of just truncating the LLM response after the first line, to avoid over-generating
https://github.com/wunderwuzzi23/yolo-ai-cmdbot
yolo does have a safety switch, which can be didabled (yolo mode).
Here is the repo if you want to take a look: https://github.com/antca/geppetto/ It's just a WIP experiment, don't take it too seriously, please. :D
When OpenAI published there API access to gpt-3.5-turbo last week, I updated a similar side project I have to use the API. It's here, if you'd like to take a look: https://github.com/wunderwuzzi23/yolo-ai-cmdbot
Its doing individual statements with some system context (like what OS and Shell) in the initial prompt, but not submitting chat history.
So now we have that kind of things as a service. We need natural intelligence first to use the artificial one.
Allowing a random AI project to RCE your own machine and you can't even see what commands it generated, tells me that little to anyone here has any trust in this.
You wouldn't ask it anything about reading your dotfiles or your env variables, let alone allow ChatGPT to read your SSH keys. So why should this be trusted anymore than a computer worm?
That's fine.
The scope of it, the way I understood it, was for educational purposes. To that extent, a simple disclaimer "Only run this in a sanbox you can afford to lose or throw away" would have been sufficient.
OP, nice work anyway!
For those who want to try something similar, but safer, warp terminal (macos) has an awesome AI command completion ... which you can eyeball first before executing. If someone is new to the terminal, bash scripting or figuring out ffmpeg, it's pretty great.
(No affiliation with warp, just a happy user)
Let's see if Google announces something at google.io or apple or Ms.
if you add and example of how to use that command, it can emulate.
That's the killer punchline in all this hype. What if you didn't ask, but somehow the command came out as an "obvious suggestion".
It is really easy to implement.
I’m going to try to get an AI assistant built up with chat gpt next. It’s way better than Siri.