You probably don't know how to do Prompt Engineering | Hacker News Reader

You probably don't know how to do Prompt Engineering

gist.github.com

59 pointsDer_Einzige82 comments

Is it just me or are the self proclaimed "prompt engineers" on Twitter just a new iteration of crypto bro grifters? Dont get me wrong, I think LLMs and generative AI are really cool and useful and have already improved my productivity to some degree, I just get no-talent, self-promoter, vaporware vibes from people I see pushing "prompt engineering" the hardest

(Edit: I agree with the premise of article, I just wanted to rant about about AI bros)

It reminds me more of a social media manager, but having having them say they work in "engagement engineering".

Would it be a proper trend without it's attendant grifters?

Anything that feels "techy" but doesn't require practice, study, or math is always going to be a magnet for grifters who were doing beer bongs while others were learning to code.

(Edit: I'm not trying to imply anything about the linked author of this piece. They actually do have code examples!)

the idea that learning to code is virtuous and having fun is bad needs to go away. Lots of people learned to code AND did beer bongs.

also lots of people didn't learn to code and did not do beer bongs... and despite knowing those two attributes, we know nothing about them

I think the point is some people didn't do the former

Then clearly I'm not talking about them.

FWIW, I don't think learning to code is inherently virtuous. I do think putting in the time to be good at something is more virtuous than trying to find the easiest way to latch onto whatever the hot new industry is, which is my impression of most prompt engineering at this point.

It's not about fun. Nuance is the key differentiator. Conscientiousness is (correctly, imo) valued because correctness matters (a lot).

You guys aren't having fun coding?

> a magnet for grifters who were doing beer bongs while others were learning to code

My experience as a test dev is that testers, especially the ones who can't code, are into credentialism and overly-technical taxonomies that describe their field. I claim people who can't do get certified (and those who make money from the certifiables teach).

more like influencers? or more properly attention seekers

I hate to say it, but LLMs/AI are entering the "solution in search of a problem" phase. The tech is crazy powerful, no doubt, but it's going to be gradually adopted by companies and woven into products we use. It will not immediately lead to hundreds of successful billion dollar startups as "AI bros" on Twitter will have you believe.

Agreed, it's kind of really useful for a lot of things people have been using NLP to solve. In some ways the human interaction piece expand the scope but overall it doesn't mean NLP suddenly solved all problems

Surprised it took this long! In my last job in academia, people started trying to turn their ho-hum ideas into gold by attaching "deep learning" or related terms around the same time other people started doing the same thing with "blockchain." Some people had great ideas and some of them turned out to be useful. But it was painfully obvious to everybody but the person talking when they were trying to obfuscate mediocrity with fancy words.

Maybe they were there all along and people just stopped ignoring them? Maybe it's all of the hucksters whose absurd coin schemes dried up or never panned out looking for a new hook?

Yeah, there is a pretty annoying pattern right now on Twitter of such accounts trying to get some engagement. It usually goes like this:

"Everything is about to change. You are being left behind. Read this thread to know more"

“99% of users are stuck in beginner mode”

I agree with you, but I think you should read the article. The author makes this exact point: "Much of it is coming from individuals who are peddling around an awful lot of "Prompting" and very little "Engineering"."

Basically, the author is arguing that just fiddling around with the words you enter in a ChatGPT window is not prompt "engineering". Instead, he talks about some specific techniques for e.g. blending prompts, indicating some words deserve more attention, etc.

Yeah my comment isn't about the substance of the article (I read it and don't have any complaints) it's just a general complaint about the concepts mentioned in the title.

And yes, I know this is a classic hacker news trope, I just couldn't help myself

It’s the new agile. I bet there’s already prompt engineering books, courses, consultants, and coaches.

It's not just twitter, YouTube is also a cesspool of such stuff. Some channels are straight up reading the docs in the video with no insight provided. And the thumbnails, why are the optimising like a Mr Beast video with huge head in the thumbnail.

Siraj Rawal the infamous ai influencer is back to being an AI influenza again. I get an almost accurate estimate of overhype by following tech influenzas now.

Yes, absolutely. There's a lot of people moving in to capitalize on the space between what people imagine they can do with LLM's, and what is currently possible. The difficulty in estimating what the LLM is likely to do with an output, the actual randomness that is involved in getting there, and the variability of the output creates a situation where you might as well put up an "INSERT GRIFT HERE" sign.

Even this post (which I think is making some pretty well informed and intentioned suggestions) exists largely because getting exactly what you want out of an LLM can be pretty difficult. Even fairly static tasks like data extraction can have aggravatingly variable outputs. I don't think that most of these are the _right_ way to get to the goal but are rather, largely clever hacks that can help a user try and nudge the LLM towards the desired latent space when adjusting the instructions fails.

I'm a dev, obsessed with ai, I'm building a startup around it now, I think of prompt engineers like spaghetti coders writing very insecure and buggy WordPress plugins and themes and calling themselves an engineer.

I've worked in PHP most of my career, but there's more engineering in an mvc than using an auto install script and installing and configuring, plugins.i guess my point is, I kinda just roll my eyes anymore.

I feel the same way with prompt engineers who don't know how to use langchain or llama index etc and aren't working on some kind of cognitive architecture to milk gpt4 for all it's worth.

tldr: I think prompt engineering is a great, legit field, but half the people in it are pretenders, although, if they can use gpt half as good as they claim, then gpt can handle most of the coding etc.. so it's probably easier to blur the line a little.

I just spent some time tweaking image generation parameters; 0.1 that way, 0.01 back again. I finally managed to eliminate the weird artifact while leaving everything else mostly the same.

Then I realized, I'm the one doing gradient descent now. I'm the one navigating a high dimensional space looking for that local optimum. I'm the prompt engineer.

You are the Oracle. How far are we from a "gpt bisect" that shows you images kittenwar-style until you pick one that is good enough?

I don't understand the proposed "Prompt Weighting". The method of prompt weighting in StableDiffusion doesn't seem to have any direct analog in the context of an LLM. What exactly is being proposed here?

Generally you use different words instead of prompt editing for an LLM, but I guess it can be useful in certain situations.

There is an equivalent to this in full text search where you can give different weights to different parts of the query (solr).

There is a free course from deeplearning.ai with intro from Andrew Ng

https://www.deeplearning.ai/short-courses/chatgpt-prompt-eng...

I don't even care about it. I just write natural language. I don't have time or the motivation to keep up with esoteric incantations and that too with a limited shelf life, that may be version or release specific.

I do think there is some value in putting the AI in the correct context role if you're using the API for some app.

This is some really cool stuff. I wish there was a sensible way to work on this, that didn't involve either using an Nvidia GPU or using cloud GPUs.

Strategy: try to get hired at an AI company that lets you play with this stuff for money!

Context: https://news.ycombinator.com/item?id=35640310#35642665

Contribute to Microsoft's ONNX runtime, it's helping accelerate non-Nvidia hardware for all sorts of ML goodness: https://onnxruntime.ai/

Prompt engineering is like being the dog whisperer but for computers.

> Thank you Stable Diffusion Community for showing us in NLP the way

Oh… so… very mostly porn. Not the first time, not the last.

Hey I have an idea. If writing prompts for the AI is hard, why not have the AI write the prompts?

> written by yours truly

Who else would have written it?

Well the whole article gives me the feeling of exactly what the author detects in others calling themselves prompt engineers. "You're not prompt engineers, /I'm/ a prompt engineer" and then goes on to suggest blending prompt results and performing some basic calculations.

True "prompt engineering" is knowing how to get a desired output, reliably, from an input. How to do things like information extraction, inference, expansion, transformation, instruction, etc. But sure, we can zipper two outputs together and write tetchy gists too.

True "prompt engineering" is not a thing. its just a misuse of the word engineering by some tech bros.

this is not engineering

Sure, in the same way that software engineering isn't engineering, if you want to be pedantic about the term "engineering".

No, software engineering is. Writing prompts to a chat bot is just being a user of that software

My personal definition of "prompt engineering" is more about building new software on top of LLMs - the kind of stuff in https://github.com/openai/openai-cookbook

I see using ChatGPT directly as prompting, not prompt engineering.

Software engineering is a type of engineering, but very few programmers actually do or even understand software engineering. Believe me, I know: I have hired and fired a lot of them.

Software engineering is. The problem is most people think they are software engineers simply because a company makes such title for a job. If that is accepted, then why not call janitors as senior principal custodial enginner.

Reply on news.ycombinator.com