Show HN: Prompt Engineering Jobs
prompt-engineering-jobs.com
prompt-engineering-jobs.com
However, I think prompt engineering will evolve to be an actual technical role, akin to Data Engineering (the people who make the systems, pipelines, ETL jobs, etc for the data).
Prompt engineers will build systems that facilitate prompt generation. Meaning that prompts will be dynamically generated or at least partially generated with modifications or additions to the raw user prompt.
It's the difference to being able to write HTML vs being able to do all the backend work to dynamically generate the HTML for my Amazon homepage (including the performance benchmarks and other strategic requirements), for example.
No pressure. My email is in my profile.
1. Take user task
2. Pass it to a prompt that requests a Product UI description of a component
3. Pass 1+2 to another that asks for which npm packages to use
4. Pass 1+2+3 to a templated prompt to write the code in a constrained manner
5. Run 4 in a sandbox to see if there are errors, if so pass it back to #4, looping
It’s currently quite slow, but that’s an implementation detail I think.
I see a fresh new generation of supply chain attack, or more prompt engineering to hopefully filter out malicious packages
Although, 'script(/injection) kiddie' will be an interesting phenomenon in the future...
Already, chatGPT and bing can both give great on-topic answers to 3 word queries. and the fact that you can infinitely refine it is great
OTOH i think there is space for developing GUIs for prompts. Makes them more engaging
You’re completely ignoring the system prompts that OpenAI/Bing have already set up so your “3 word query” works as you intend. These system prompts are what prompt engineering is all about.
If I have an LLM in a video game that generates NPC dialog, I might want to feed more information to the prompt based on things my character has done in the game so the NPC dialogue is more relevant to me.
Maybe I want to inject the user’s location or the current weather in Sunnyvale, CA for the specific query. Or maybe I want to inject that the user is currently at Disneyland.
Maybe the user really likes a specific tv show and I want to let the model know that. Or maybe the specific TV show was DMCA’d by an IP owner and I need to put that in the prompt.
Do I need to detect that the user is below 13 yrs old and use a different prompt (or a different model altogether)?
Maybe I want the model to only ever respond with JSON without the user needing to specify. It needs to be clarified in the prompt with no way for the user to override it.
Models can be simplified to: input to output. Prompt engineering will be engineering the inputs.
Can you share something similar for "prompt engineering" ?
What engineering is required for Amazon to deliver me an HTML page? Are the backend engineers just “injecting” HTML? Of course not. It’s not the same as generating the HTML for a personal blog.
As far as what skills are need, it depends on how any given team wants to modify the prompt. It’s up to the ingenuity of the engineers. Maybe they need to hit 5 other models before generating the prompt. IDK. The prompt may change for any given scenario, which is why you need engineers to build the system.
https://www.promptingguide.ai/
https://lilianweng.github.io/posts/2023-03-15-prompt-enginee...
These clearly demonstrate that there's engineering skills beyond "I want you to act as a Linux terminal."
Also note that there's a difference between the prompt that you use on ChatGPT and the one that you use in the GPT API or other LLMs. In the first, you're already dealing with the prompt that openai supplied to the LLM in the first place.
Is that what you discount them for?
The reason was what I stated, which has nothing to do with the provenance or capabilities of that person.
The stuff you say it will evolve in already exists and has names everyone uses.
It is a nightmare for me, and I do somewhat regret taking the contract even with the sizable hourly rate.
In my eyes, programming makes sense. Even if I introduce bugs, I can sooner or later track down the issue, facepalm, resolve it, and move on with my day, with some semblance of accomplishment and lessons learned.
My prompt engineering work offers no such rewards, and is a total time-suck. This is because, as another commenter wrote here, it is "throwing shit at the wall and hoping it sticks."
While you can treat it as a scientific endeavor, testing hypotheses against a black box, you will never find a prompt that works consistently, even with a low temperature, solely because these models were not built to give consistent results. There is no end-all solution, there is no "correct prompt."
Companies employing prompt engineers are looking for such consistency. Prompt engineers are therefore stuck in a system where they simply cannot succeed. They can hope and pray that the testing done by managers produces fruitful results upon every test, but the results are, for all intents and purposes, random.
Are the companies enamored with the productivity gains of prompt-based systems sufficiently that they can ease up on the requirement for consistency?
Let me give a real example (cant go into too much depth but this is generic enough). I was told to create a prompt that would make ChatGPT generate ten of the hottest real estate markets:
"You must always format this data in exactly the following format: ${index (as a single numeral)}: ${name of location} ${median housing price} ${YoY increase} ${link to internal system} ${number of available locations}"
You would think this would be a pretty easy task for ChatGPT/Davinci to handle, but like I said, consistency is completely missing.
- Sometimes it will say "Market ${index}" instead of just the number, even with the single numeral qualifier.
- Sometimes it will comma separate the data, even if there are explicit instructions not to do so
- Sometimes it will provide the prices/percentages/numbers as regular integers, sometimes it will format them, again, even if explicit instructions are provided
- Sometimes it will place an empty space in between the lines, even if there are explicit instructions not to do so
- Sometimes it will place random punctuation at the end, even if there are explicit instructions not to do so
It's entirely random, you could have a 4000 token prompt or a 40 token prompt, and the results will be of the same quality.
Usually, this is fine for content generation. But content formatting? Anything remotely specific? Forget about it. All of the models (I haven't been able to try ChatGPT 4 yet, however) will simply ignore things, even if (and I promise it's the last time I'll say this) there are explicit instructions not to do so.
I would claim that using examples do improve its chances, but it is still too random to assume the edits were what improved the response, or whether it just happened to be a lucky roll of the dice.
Perhaps ask for json, to simplify the output space?
I imagine in the near future we'll be able to prompt with a json schema, and have it enforced.
When I assign a task to a (human) developer, the results depend on two things: First, how good the developer is, second, and more importantly, how well and clearly I am expressing the requirements. And this is also true for ChatGPT. With very precise requirements I get very good results.
So prompt engineering is like writing good requirements, and that also requires understanding the problem domain.
Software Engineering is kinda fake, especially in industry, but at least that's an actual discipline.
"AI monkey" is a better description
A more charitable description might be "You're employing the scientific method to extract value from GPT-like systems." Just like in science, with time you're developing intuition for how the underlying system works, but you still have to run the experiments.
Using LLMs to solve real problems is not easy. Making sure that you don't introduce regressions while making improvements is difficult, and requires building and evaluating a dataset, and the necessary pipelines. It may also include diversification of LLM providers, and creating the necessary abstractions. A fundamental understanding of how LLMs work, ability to compare different architectural approaches, along with typical data engineering and software development skills would be required.
What if you want to use the LLM for Question/Answer systems that requires working with embeddings? What if you want to find a way to process data locally without sending sensitive data to the LLM provider?
This requires real engineering skills.
Yes it is.
That is exactly why products like ChatGPT have been taking off as quick as they have.
What you're talking about is not using LLMs as a product but using them as a component within a broader system. And so of course that requires engineering skills.
It's highly unlikely that anyone taking this effort seriously is copying and pasting from ChatGPT, rather than using the API and building pipelines as part of a broader system.
Instruction-fine-tuned LLMs like ChatGPT require creating, validating, and maintaining prompts. Finding ways to use them safely is also not easy - prompt injection and hallucination are just 2 potential pitfalls - there are many more.
Denigrating this effort as "AI monkey" is myopic at best, but really just comes across as a signal that someone is terrified of being replaced by this new tech. With that attitude, they will be.
>What if you want to [do software engineering]?
Then you're a software engineer. Writing prompts isn't engineering. Building systems is engineering. Just because I use keyboards to program doesn't mean I'm a keyboard engineer, does it?
Otherwise, the job title would be "prompt writer".
Your point is what, that existing engineering titles cover this effort? Sure, you can just call all of it software engineering, but sometimes it's useful to be more specific. The LLMs are so powerful now that this new, more specific title makes sense to me, and clearly those using this new title. We'll see how it pans out over the next few years.
"Google for Dummies"
I want to be a Cannabis Prompt Engineer now https://www.xing.com/jobs/muenchen-remote-cannabis-ki-prompt...
I'm afraid these kind of job postings asking for experience longer than technology will always be there.
I have removed the post now.
They seem pretty determined to find that one candidate.
In that perspective, I understand why many people think it is useless. However, if you tried to make a chain of functions/calls, or worked with a tool like LangChain [0] you will see its importance.
Ex: "Which stock had better performance in the last 6 months, Tesla or Microsoft?"
A question like this would check:
- Understanding this is a financial question.
- Get the stock ticker (symbol) for each one.
- Use an API to get their performance history in the last 6 months.
- Compare.
- Return the answer.
Given the wild popularity of posts about prompt engineering "jobs" paying +$300k, it was only a matter of time for an indie hacker to create a job board specifically for this type of job.
I have a BSc and Master's in Computer Science, 14 years of experience, I have slowly climbed the ladder, got a FAANG job 5 years ago, and only last year I managed to break 300k salary for the first time (working in ML of all things). Am I losing my sanity or these reports are greatly exaggerated?
Please share an example. I've only seen articles claiming this might exist, based one attention-seeking job ad from one company, which didn't claim "no experience or technical skills".
I believe that a critical part of software engineering is the ability to trace and resolve an issue through many different systems and code paths (what most people call debugging)
There is no way to debug these models, there is no way to systematically fix your prompts. It is therefore completely guesswork on what words in what orders might produce results closer to what your stakeholders expect.
Software engineers, when presented with an issue, can dive down as deep as they want (hell, even to the machine code layer if it comes down to it) in order to fine-tune, fix bugs, or do whatever else a stakeholder might expect.
Two very different coins in two very different universes.
I think if you spend some more time with these models, or any similar model where you have X input > Y output, you notice that you can in fact "debug" those as well, without knowing 100% of the internals. Some fixes are better than others. There are better and more reliable ways to steer these models, compared to other ways. It is not 100% guesswork, some people are better at it than others.
With a google search you are getting citations and stuck finding the truly good ones that truly match, with chatgpt you are getting an answer from someone who read all those citations and treats them all as equally good.
I think the real problem was to get the most perfect citations given your specific question. We can forgive the error since it took the web a few years to discover that best is rarely a fresh bit of blog spam by a moron who has read everything and refuses to cite sources. So GPT is convincingly as good as a human blogger yet not as useful as a less human emulating tool.
What does "smart" mean to you in this context?
I don't speak french but it clearly says 20 years of experience. Talk about trolling.
I think Prompt Engineering Jobs could become popular
We've started calling it LLMing (llemming).
Edit: Specifying prompts is leaning towards specification. I am not saying googling is that. I'm saying that, like googling, it will just be a part of the job in a not distant future.