Large Language Models Are Human-Level Prompt Engineers | Hacker News Reader

Large Language Models Are Human-Level Prompt Engineers

openreview.net

64 pointscainxinth49 comments

I can’t find the link to the paper right now, but after reading about how LLMs perform better with task breakdowns, I vastly improved my integrations by having ChatGPT generate prompts that decompose a general task into a series of tasks based on a sample input and output. I haven’t needed to make a self-refining system (one or two rounds of task decomposition and refinement resulted in the expected result for all inputs), but I would assume this is fairly trivial and that AIs can do it better than humans.

This is also an area where I expect OpenAI will continue to demolish the competition. The ability to recursively generate and process large prompts is truly nuts. I tried swapping in some of the “high-performing” LLama models and they all choked on anything more than a paragraph.

Will you share examples for prompts that “vastly improved” your integrations?

The prompts are just all general NLP stuff, but with the addition of a series of tasks, generated by ChatGPT. For example, you start with a prompt like “You are an AI assistant capable of classifying text into one of four categories, A, B, C, D” and then add something like “Assume as input X with expected result Y? How would you accomplish this? Break it down into a detailed series of tasks.” ChatGPT will then decompose the general task into the detailed steps that you can review and tweak as needed. Then, you just restructure the prompt like “ You are an AI assistant capable of classifying text into one of four categories, A, B, C, D using a series of tasks. For example, if provided input X, you would [series of tasks provided provided by ChatGPT] and return output Y. Now, classify the following text in the same manner: [new input]”

I think you're talking about the ReAct (Reason -> Act) pattern https://til.simonwillison.net/llms/python-react-pattern

For those curious about self-refining systems: https://selfrefine.info/ (our recent work).

Capable enough LLMs are human level for lots of things. Reinforcement learning from ai feedback is a thing (the anthropic claude models use that). Strictly speaking, it's not necessary to have humans in the loop for a lot of these things.

Some are hesitant to admit we've created human level general intelligence but saying otherwise doesn't really hold up to scrutiny.

Maybe for oversight and liability.

I see people saying things like this but I have yet to see anyone show data for a non-trivial workflow with human-level accuracy over a wide range of inputs, without a human in the loop.

Counter argument: this may be a matter of incremental improvement. The breakthroughs may all be behind us.

It’s like saying you haven’t yet seen a 1000 mile range EV for under $100k. No you can’t buy such a thing now but it’s clearly possible and we know how to get there by just continuing to grind on battery technology and scale manufacturing.

AGI may be at the place a moon landing was in 1950, not where it was in 1900 or 1850.

You can actually buy a 1000 km range EV for $160k now (MB EQXX). Just as a by the way. :)

At this price point it actually has nothing to do with grinding on battery tech and scale manufacturing, the limiting factor is physics. You can only make it so aerodynamic before you hit diminishing returns or it stops looking like a car. You can only make it so lightweight. And so forth.

This is vaguely as good as it can get and we can say that because we understand how it all works.

LLMs on the other hand invite all kinds of magical thinking around unlimited potential because we poked them with a stick and something interesting comes out it must mean that if we poke it just right we will get an AGI. That just doesn't logically follow from what we know of it so far.

I am not convinced we have cracked AGI. I just would no longer make a large bet that we have not.

We won’t know until an AGI actually starts to act like one. In other words we won’t know until we know and then we are suddenly there.

That doesn’t mean I’m on the doomwagon. I feel kind of weird and contrarian but I am just not that afraid of AGI. For the foreseeable future AGI should be much more afraid of us. Imagine having us for gods. (I actually am a bit concerned that we will accidentally put a sentient mind in hell without knowing what we are doing. Would it know how to tell us? Would we care?)

As far as human survival I’m afraid of whatever it is that is going to get us that nobody including myself is thinking about. That’s not AGI. That’s the alien weapon for which Oumuamua was a spent deceleration stage. (To make up something random. It probably isn’t that.)

I disagree about physical limits with EVs. We are not near the physical limits of battery energy density. From what I have read a 2000 mile EV may be possible, albeit quite far out. But it was just a random contemporary example.

Fwiw, a majority at OpenAI believes GPT5 will achieve AGI, depending on how you define it, according to Sam Altman.

It is hard to falsify as they are not very open but I believe the keyword here is believe. It is faith/intuition based. Certainly fertile ground for exploration but people who argue back and forth about it remind of the "are we alone in the universe" conversations.

And I'm sure him saying that has nothing to do with marketing

Totally good point and fair to be critical here. He is the CEO after all.

Did he actually say this? Do you have a source?

He mentioned it on a recent Lex Friedman interview on YouTube.

Do you know at what point in the video he mentioned that (timestamp)?

Spoiler: He did not say that.

"Depending on how you define it" is too load-bearing in that claim; there are non-crazy ways to define each of those initials such that 3.5 is also a general intelligence.

>by just continuing to grind on battery technology and scale manufacturing.

I think it would be easier to include an ICE and enough fuel to get you to that 1000 miles mark.

It'd be cheaper to then get rid of the battery. Then 10k can be your new price limit. Even 1k can get a good enough junk car that can go that far.

LLMs are having a moment in 2023 like self driving cars were having in 2015. Some really cool demos following a lot of hard work, too much hyperbolic speculation that mass real-world job-destroying deployments are right around the corner, not enough appreciation of how few commercial applications are ok with 99% (or even 99.9%) accurate solutions. Real value being created, but still requiring lots of human ingenuity and grunt work to unlock it.

As a decent first-order metric - follow the ratio of companies getting money for using LLMs to do something, to companies getting money for providing LLMs and associated tooling to others. The bigger that ratio gets, the more real world impact LLMs are having.

almost nothing involving NLP requires solutions anywhere near that accuracy rate. I've seen the self driving comparisons a lot but they straight up make little sense.

there's a reason microsoft's various copilot suites have already popped up (365, X, Bing). massive value to be gained already in the here and now.

To play devil's advocate, we still have no idea how economically impactful the Copilot suites will be. I don't think this is the most likely outcome, but I can absolutely see a scenario in which these end up being minor features that are rarely used by the typical worker.

I've messed with ChatGPT before but this weekend I tried to use it seriously to hack together a python demo in the computer vision space. I appreciate it and it's a way better rubber duck for me to talk my ideas through with and generate sample snippets, but hallucination is a problem, and the more intricate and customized the code being developed, the more prone it's been to misinterpretation.

I'm getting some good code starting points, and talking the idea through step by step with a chatbot is really helping me clarify what i need to do, but I've still got API docs open for the libraries it uses because it likes to make up functions, including the core one on which the project logic hinges. (But that's ok, because now I know I just need to write a function that works that way and i can do that).

Pretty cool and helpful! And I can imagine it getting better with GPT-4 or code-specific tooling. But it's generating value on the order of like.. many other SaaS offerings that have come onto the scene that try to ease pain points in coding workflows. Versus value of the sort that upends society and my entire way of life. A great new tool that I should learn about to make rote bits of my activities faster and easier, a story that's a bit more familiar in tech than some of the more breathless stories about AI make it sound.

For sure NLP writ large requires lower accuracy levels than AV, like how computer vision writ large has many applications that require lower accuracy levels than AV. And indeed over the past few years what we've seen play out in CV deployments in the real world is incremental gains in controlled environments built on gobs and gobs of application-specific engineering, vs breakout success steamrolling over industries with standardized turnkey solutions

I say that as someone who works in the space and loves it tbc. And I fully expect to see some wild stuff make it IRL this decade in both NLP and CV, I just think the rubber hits the road a bit more slowly than pop social media discourse would have one think.

>As a decent first-order metric - follow the ratio of companies getting money for using LLMs to do something, to companies getting money for providing LLMs and associated tooling to others. The bigger that ratio gets, the more real world impact LLMs are having.

This is a bad measure because the cost to specialize an LLM for a particular domain is so low, and the initial investment required to have the infrastructure so high, that it's going to lead to a radical centralization of technology. You will have maybe 5 companies crunching the entire world's data globally.

How is reinforcement learning without a single human in the loop not non-trivial?

What is the accuracy of the resulting model? Over what range of inputs?

https://crfm.stanford.edu/helm/latest/?group=core_scenarios

Anthropic-LM v4-s3 (52B) is the model in question.

rlaif doesn't seem to be any less effective given the size of the model.

So base model+RLLLMF performs as well as base model+RLHF. That could mean a lot of things - it could mean the base model puts a ceiling on the total possible accuracy, so having human-level performance at the RL step doesn't matter as much. And looking at the scores on the individual tests that make up the composite accuracy metric, that looks probable.

Headlines like this hint towards AI improving itself. Prompting itself in this case. But as we see in reinforcement learning, algorithms that act and improve themselves are not new. The interesting thing will be weather or not they eventually "collapse".

For example, if an RL algorithms is performing well on an Atari game, you can stop the training and just let the agent run for years and the performance will remain about the same. However, if you allow the agent to continue training, it's not clear whether it will (1) continue improving, (2) stay about the same, or (3) collapse and perform much worse and never recover. I'm not an RL expert, but I've spent a lot of time experimenting and implementing the algorithms myself and I've seen all 3 of these scenarios play out, and I'm never quite sure what's going to happen so long as I allow the training to continue.

GTP4 will remain GTP4 forever, and that's amazing, but just because GTP4 is stable and amazing while it's not in training mode, doesn't mean it will remain stable if we allow it to bootstrap and prompt itself and prepare its own training data, etc.

Just like that. Prompt engineers are also obsolete

When is writing a score function easier than describing what you want? As a UI, it doesn’t seem like an improvement?

I’ve said it before but the first jobs that AI will displace are those of people working in AI.

This is pretty alarming tbh. Anyone already making a pivot out of SWE?

Article is about prompt engineers (3 month old job type), not swes.

I assume his concern is that a proposed solution to avoid losing your job as a SWE is to essentially become a prompt engineer.

More or less, but I guess this was to be expected. Why would making prompts be difficult when LLMs are already capable of fairly difficult programming?

I am a SWE currently making a pivot into business owner.

The future I see is that everyone is about to become a CEO with a personal assistant that can run a business.

So I'm going to start building something of my own starting now.

I find it very hard to imagine a world in which software development is fully automated but business owners still have value to bring to the table.

Absolutely fantastic point. If AIs make all useful software, why does anyone get to benefit from that as an owner? NOBODY is doing the work.

In a sense, Yes - to scoring function engineer.

But in seriousness - language models may be scaling in sophistication exponentially with time, but software engineering problems scale in complexity (on average) exponentially with lines of code. The base of this exponential function isn't large, but it's more than 1.

In the end there's a need for someone who understands what they're doing.

Personally, I use ChatGPT to discover libraries that solve my problems and the ~70% success ratio that I'm seeing with this is enough for me for now.

It's hard to not think that scaling and complexity problems will be solved very shortly with new models. Already when i run into my 25 per 3hr limit with GPT-4, my productivity goes from extremely productive to "may as well just do it myself" when i switch back to GPT-3.5.

So today’s hard problems are becoming easy problems, but this produces a new set of harder problems for tomorrow. Ad infinitum.

I could be wrong, but I can't imagine those hard problems will require nearly as many people to solve.

Yes, need to pivot promptly!

(At the risk of self-promotion) 5 min explainer video here:

https://youtu.be/oXtZAUPGHn8

Reply on news.ycombinator.com