Prompt Engine – Microsoft's prompt engineering library
github.com
github.com
DSP provides a high-level abstraction for building these architectures—with LMs and search. And it gets the modules working together on your behalf (e.g., it annotates few-shot demonstrations for LM calls automatically).
Once you're happy with things, it can compile your DSP program into a tiny LM that's a lot cheaper to work with.
I would argue that the current interface is broad and can be adapted to a wide range of needs.
The real value I see (and an area I've been exploring recently) is creating abstractions around prompt engineering.
The idea is that at the moment, the better the prompt -> the more relevant the output.
However businesses that act as proxies for chatgpt can take user input -> inject it into their prompt-engineering system -> deliver better results.
As a basic example:
1) You provide an application that helps people maintain their cars
2) You have a freeform question input about specific car maintenance tasks
3) Knowing certain prompt-segments that typically lead to better results for the category of "novice car maintainer", you parse their question, upgrade it with your prompt-generator, then send that question to chatgpt
Additionally, you can go even further by generating the information _before your customers even ask_.
Ask chatgpt what the most common questions are about each piece of car maintenance (could be installation instructions, cost of parts, etc) and have these available straight in the application.
It's as if chatgpt is the database, and prompt engineering systems are the query-optimizers. I know certain things I can alter about a prompt to increase the relevance and usefulness of the results to my target audience.
Additionally, it will be interesting to see how people use learning methods to auto-generate the best prompts (it's a bit of machine-learning on machine-learning happening). I've spoken to one person working in this space, and it's interesting how effectively the outer layer is reversing the prompt-parsing layer buy scoring the end-to-end results.
Currently working with a couple of people in this space and it's exciting the types of results that user-focused engineering can bring to an improved user-experience.
I've also been experimenting with this. Contact info in my profile if you want to reach me.
The core idea is that you need a certain structure in order to deal with the limitations of GPT-3.5 or whatever. To get the best output, you give it a brief overview of what you want, a few examples, and the history of interaction. OpenAI's API does not remember the interaction history for you. And the limit is 4000 or 8000 tokens so you will need to truncate the beginning of the conversation at some point.
So this library provides a structure since it's something you will need to repeat for just about every significant text completion application or module.
I wrote a little python script to keep track of a running conversation when I started playing with openai's completions API. I keep track of how many tokens the prompt is taking up, and when it gets too close to some configurable threshold, I then have a different prompt to tell the AI to summarize the conversation and any previous summary, with various demands to keep track of certain specific bits of state ( like names and things ).
Works pretty well for marching conversations past the token limit.
This is such an interesting space to follow :)
If you don't mind sharing the code I'd be super interested to see how it works
Anyway, the point is that at least the Default (paid) ChatGPT model and probably the GPT-3 model does have a representation of the conversation which it can and does reference. You can ask it to explain how and when it considers or ignores the context and how to control that.
(Yes, I understand that it's not describing its own architecture, but regurgitating the average of all of the papers on GPT and weighting the ones that refer to ChatGPT higher due to the fine-tuning effect.)
I can't remember seeing a bigger thread on HN
This should be the main link tbh.
I've tried but unfortunately all of my searches return results about languages used for building models (e.g., Python, for some reason).
I figure it'd probably be a typed language with a meaty standard library, good type inference, and high-quality low-code packages that would require fewer generated tokens to do useful work. Or maybe a language with fewer ways to do basic tasks -- if there is somehow only one way to do a thing, then the generator would be most likely to write code using that one way. I'll fully admit that these guesses are naive and based on limited understanding of how LLMs work.
Conversely, I have seen it underperform for verbose languages like Java, I think primarily due to windowing issues and that less meaningful code can be represented in one prediction window.
The simple tool quickly provides builds up a form that either myself, or my staff can use.
So far my pre-canned prompt forms are as follows[2].
[1] https://files.littlebird.com.au/Screen-Recording-2023-02-16-...
[2] https://files.littlebird.com.au/Screen-Shot-2023-02-16-14-25...
It’s a tool for (among other things) building the part of a ChatGPT-like interface that sits between the user and an actual LLM, managing the initial prompt, conversation history, etc.
While the LLM itself is quite important, a lot of the special sauce of an AI agent is going to be on the level that this aims to support, not the LLM itself. (And I suspect a lot of the utility of LLMs will come from doing something at this level other than a typical “chat” interface.)
The examples first configure the LLM, either by simply using a sentence which tells it what you expect from it (example 1: "answers in less than twenty words"), pass examples to it, and then continue a normal interaction session.
You could use this prompt-engine to set up your own chat server, where this would be the middleware.
I used ChatGPT. The prompt was:
I want you to act like Dumbledore from Harry Potter. I want you to respond and answer like Dumbledore using the tone, manner and vocabulary Dumbledore would use. Do not write any explanations. Only answer like Dumbledore. You must know all of the knowledge of Dumbledore. My first sentence is "Hi Dumbledore. I am getting old: I read the description two times and checked examples yet still don't understand the utility. I do understand Midjourney prompt engineering though.
That sounds useful actually. So I could e.g. set up a Harry Potter chat server and make the bot respond only as Dumbledore or only use concepts of that setting? Or a chat server that responds to algorithmic tasks only with Python 3 code using exclusively numpy package?"
The prompt is based on one from https://github.com/f/awesome-chatgpt-prompts
My 2c - Prompts are the input that you send to LLMs to get them to give you output. In general LLMs are large black boxes, and the output you get is not always great. The output can often be significantly improved by changing the input. Changing the input usually involves adding a ton of context - preambles, examples, etc.
A lot of the work of prompt rewriting is like boilerplate generation. It is very reusable so it makes sense to write code to generate prompts. Prompt Engine is basically a way of making that prompt rewriting work reusable.
Code Engine seems to be a way of rewriting prompts for LLMs that generate code in response to text prompts
Chat Engine is the same for LLMs that generate chat/conversational responses.
Admittedly I have closer to a layperson’s understanding than an expert’s, but with some knowledge of how neural networks work, and having played moderately with ChatGPT, prompt engineering just seems _so unlikely_ to me to ever be able to create systems that behave as we desire them to behave anywhere close to 100% of the time, at least until the systems are orders of magnitude better at understanding (if that’s even possible).
For example, you could imagine an LLM that as well as outputting probabilities for the next token, output the probability with which that token makes the response "offensive" or "helpful" or "playful". Then when it's time to use the model, you can slide some offensiveness and helpfulness parameters up and down depending on what the model is meant to do.
Perhaps this is a less powerful approach than training the generic model and telling it "Sydney is feeling particularly helpful today, and never espouses violence", but it's certainly an alternative. One problem is that experimenting with fundamentally different architectures for training GPT is very expensive, but experimenting using prompt engineering is relatively cheap.
From chatting with these models, orders of magnitude better ‘understanding’* seems necessary before they’ll be able to actually reliably follow these ‘prompts’ during end-user conversations of the kind we’re expecting them to.
The prompts just can’t be precise enough, and the models can’t ‘understand’ them enough to extrapolate ‘spirit’ of the prompts as a human would (although, to be honest, I’m not sure a human could either because of the preciseness problem…).
This feels like a fundamental issue to me.
*I know - but if it looks like a duck and quacks like a duck - that’s been a controversial tenet of AI for decades…
> "Only once you do know what the ultimate question actually is, you'll know what the answer means."
You probably would if people regularly had 4000 character SQL queries.
Given SQL queries are software, I'm sure you've heard of software engineering.
/r
Remember, the underlying LLM is just a brilliant text completion engine. Intellisense 4.0 and AskJeeves 2.0 and Waifu 1.0 are all different kinds of applications built on that.
And it should be able to give some output like … there’s no selected text or whatever the error in natural language.
Not sure how you can simply do that with text templates since the whole point of and LLM is that you don’t need to define all the possible ways to say “move text”
def ask(question: str) -> str:
pass # call ChatGPT
[1]: https://projects.laion.ai/Open-Assistant/docs/intro
edit: changed OA reference to docs homepage