The whole prompt engineering thing feels like a temporary stopgap. Because ideally language models should be able to understand prompts from anyone without them trying to craft in a way that works for the model.
I'm definitely not calculating the path of a projectile in a similar manner when I catch a ball.
I'm definitely not "computing" a sentence when I read it in the same way that I compute the multiplication of those two three digit numbers!
You're not calculating it in a traditional sense, but there's definitely some systems of partial differential equations being solved in real-time.
I was increasingly frustrated with all the NLPness and operator deprication in google which has been accelerating since at least the 2010s.
But with Kagi it reallys makes me feel like I am back at the wheel. I think for product search it still has some way to go but for technical queries it is just on a whole other level of SNR and actually respects my query keywords.
Isn't this just an intrinsic problem with the ambiguity of language?
Reminds me of this: https://i.imgur.com/PqHUASF.jpeg
edit: especially the 1st and last panels
Prompt: "Calculate the average price of Milk"
This is far too vague to be useful.
Prompt: "Calculate the average Price of Milk between the years 2020 and 2022."
A little better but still vague
Prompt: "Calculate the average Price in US Dollars of 1 Gallon of Whole Milk in the US between the years 2020 and 2022."
Is pretty good.
For more complex tasks you obviously need much more complicated prompts and may even include one-shot learning examples to get the desired output.
This argument is weak. Undefined behavior does exist and „high level programming language „is a moving target
Other than complicated requests like “if object is of type A, include fields ABC but not D. If object is of type B, include only D but not other fields”, it gets this right 99% of the time.
It also works for CSV, but it’s trickier. It seems like it “knows” how JSON works to a much better extent.
And as for parsing JSON? I’ve not truly pushed it to its limits yet, but so far it’s had no issues understanding any of it.
It’s mind-boggling. Yes, it’s inefficient, but it can basically parse, generate and process valid JSON with just a brief set of instructions for what you want to do. For exploring ad-hoc data structures or quick mocking of API backends, this is great.
2. the action of working _artfully_ to bring something about. "if not for his shrewd engineering, the election would have been lost"
(https://www.google.com/search?q=define%3AEngineering)
Merriam-Webster has:
3 : calculated manipulation or direction (as of behavior)
giving the example of “social engineering”
(https://www.merriam-webster.com/dictionary/engineering)
Random House has:
3. skillful or artful contrivance; maneuvering
(https://www.collinsdictionary.com/dictionary/english/enginee...)
Webster's has:
The act of maneuvering or managing.
https://github.com/dair-ai/Prompt-Engineering-Guide/blob/mai... (this is claimed for LLMs, not proven)
It only seems like a trick until enough papers get written about these kinds of findings.
This includes asking questions, and trying to direct someone to effectively complete a task.
Prompt engineering is just communications skills as applied to AI instead of meat-minds.
E.g when talking with my accountant, she usually ask me a bunch of clarification questions and numbers, instead of just making a best effort, but confident sounding response with whatever initial context i gave.
I myself don’t have to have the depth of knowledge to direct someone to complete a task step by step to get the right results.
Perhaps a big step forward for chat AI interfaces is to make the AI capable of knowing when it needs to ask follow-up questions and then having it do so. Essentially, it helps you along with your prompt engineering.
Perhaps this is what the prompt engineering tools are doing anyway.
What we need to do is integrate the prompt engineering tools with the chatbot itself, so it can both help extract a good prompt from users and then answer that prompt in the same process.
I think this is where we'll move towards relatively soon; it seems obvious when you say it.
They were recently given $300M by Google, so certainly you have a promising idea there
If you did have to send google-esque one shot queries to humans you'd probably settle on short hand like "polish shiny" or you'd opt to use "poland" specifically to avoid it. For most known-ambiguous terms you'd do this, the same way we say out loud "M as in Mancy" because we know the sound of the letter can be ambiguous sometimes. We have lots of these in English that you use all of the time without knowing it. In a smaller audience the local language gets even more specific, you probably disambiguate people with your friends like "programmer Bob, not carpenter Bob". It's not at all crazy that we'd develop another local language for communicating with computers even if that's not a traditional programming language
If I were talking to a python programmer I could assume they knew what a for loop was, so could phrase a question requiring the context of one differently than I would for the layperson. Just like if I were designing inputs to one language model I could assume it's capabilities that are different from another.
I don't think it's relevant to the performance of language models for that same reason though, we already have to design out queries with the thing being queried in mind so I don't see why we wouldn't for llms.
But they don't.