The fun app I'm building right now is yourdrink.is, it takes the name of a drink (real or imagined) and generates a plausible recipe, complete with a photo. I'm using a prompt that asks for the model to complete a JSON object like this one:
{title: <user submitted>,
description: <str 150 chars>,
product_photo_prompt_for_dalle: <str 150 chars never includes the title>,
ingredients: [<str like '1.5 oz whiskey'>,...],
instructions: [<str like 'add whiskey and soda to a rocks glass'>,...]
mixologists_notes: <250 char, public-facing, promotional tone>}
Those are my arm-chair definitions of prompt engineering. Other sites I've built that involved this kind of 'prompt engineering':
poetic.am (poems from your pictures)
support.rip (type what you want to say, get out perfect Support Tone)
uplift.boo (uplifting friend that tells you you look great or otherwise showers you in positivity)
How many templates do you tend to test out per use-case prior to settling on a specific one? Do you just use the first one that works or try to "fine-tune" the wording to get it just right?
What sort of things do you tend to spend the most amount of time while prompt engineering?
I'll implement that prompt in code and build the front end around it, and then the mindset shifts: now I start trying to break the prompt and will get adversarial with it, finding weaknesses and making tweaks to patch them. If you've ever done a bug bounty program this is a lot like trying to turn a vuln into an exploit, it feels like a really creative process and I find it very engaging, I tend to get into a flow state with the model when I'm tweaking prompts. You kind of quickly switch between red team and blue team thinking.
It's very helpful to log all of your prompts and results in a database. I use nltk to do some simple analysis of my results, counting occurrences of ingredients is how I verified my impression that certain ingredients were being highly over-represented in the output. I also use prompts here to help me explore the data ("I have a pandas dataframe of named entities and their raw counts from a corpus, I want a sorted list of the top 20 entities and a float from 0-1 that...")
Once you launch the project another shift happens, you can start observing how people are interacting with the model and make changes as needed based on real user input. There are a lot of people poking at hastily-built AI tools these days, so I've even learned about some prompt attacks by finding them in my own logs.
The most interesting example of seeing unexpected problems with a prompt post-launch that I've personally experienced was with my 'poems from your photos' app https://poetic.am, very early on a user got a bunch of poems that mentioned their wheelchair, praising them for overcoming obstacles and whatnot. That's fine, but for someone in a wheelchair, the chair is far from the most interesting thing in the photo of their holiday gathering. Someone who uses a wheelchair should be able to get poems about their photos just like everyone else, not an endless string of "hey, look at you, overcoming obstacles!" So I tweaked the prompt to account for these kinds of cases.