Hacker News Reader

Hacker News

Comment by nopinsight | Hacker News Reader

Full thread

nopinsight

“Reflection-Based GPT-4 Agent is State-of-the-Art on Code Gen

Iteratively refines code, shifting “accuracy bottleneck” from correct code gen to correct test gen

HumanEval accuracy:

-Reflexion-based GPT-4 88%

-GPT-4 67.0%

-CodeT 65.8%

-PaLM 26.2%”

with link to code in the Tweet:

https://mobile.twitter.com/johnjnay/status/16393620718075494...

21% improvement after adding a feedback loop and self-reflection to GPT-4, which just went public 12 days ago. (The approach is based on a preprint published 4 days ago.)

Human coders often need a feedback loop and self-reflection to properly “generate” code for problems novel to them as well.

-----

A larger question: Are we hurling ourselves toward a (near) future of unaligned AGI with self-improvement capabilities?

View on HN

Someone asked GPT-4 to build a complete app from scratch. It's now on the store. He seems to apply good prompt engineering techniques.

Screenshots, video demo, and process here: https://mobile.twitter.com/mortenjust/status/163927657157489...

It seems plausible to me now that a junior developer position would be hard to find in 2-3 years (I thought it would be ~5 years).

So this is very impressive and looks like a solid lowering of barrier to entry which is great….but that app is around 300 lines of code in one file that fetches data on 5 movies, screenshots are not cropped correctly and the pager doesn’t swipe back to the first dot. I am bit surprised it made it through the review process. Not hating I think it is great to make this stuff more approachable but not convinced junior devs are in danger yet.

GPT-3 was released less than 3 years ago and it was far from capable of this.

Sure and it is impressive but the upper bounds of AI products are hard to predict. That said things are changing fast I don’t know what tomorrow will bring.

Exponential curves are theoretical constructs: all actual phenomena are S-shaped.

The question is only when does the "exponential regime" turn into the flat; and the answer is often fairly obvious if you don't begin from the "time = magic" premise.

There's an entire industry of public (pseudo)-intellectual who's schtick is to draw logistic phenomena with exponential curves and then cry, "the sky is falling!".

On the contrary, few experts expected this performance from an AI this soon too.

If you can identify one or two aspects of the human “general” intelligence that an AI cannot ever possess, even in principle, I think a lot of people would be grateful.

In animals, propositional knowledge is built from procedural knowledge; and it can't really be otherwise.

What AI does at the moment is approximate propositional knowlegde with statistical associations, rather than take the procedural route. But this fails because P(A|B) doesnt say whether A causes B, B causes A, A is B, A and B are causally unrelated, etc.

What is the procedural route? To perform actions with your body so as to disambiguate the cases. Animals have causal models of their bodies which are unambiguous and their actions are intentional and goal-directed and effectively "express hypotheses" about the nature of the world. In doing so, they can build actual knowledge of it.

There's at least some good reasons to suppose that "bodies which express hypotheses in their actions" require organic properties to do so: becuase you have to have adaption from bottom-up to top-down to really have "the mind" grow the body in the relevant ways.

In other words, every action an animal performs isnt clockwork: in acting, it's body and mind change. Every action is a top-down, bottom-up whole change to the animal.

This is a very interesting hypothesis that could be quite true for living beings. What I disagree with is that having an animal-like body is necessary for the process of forming a world model. A simulation could be sufficient. And there is already work on that front. (Also, I would not characterize deep-learning-based AI as trying to form propositional knowledge. In fact, its great performance partly stems from not dealing with propositional knowledge directly.)

If a body is in fact necessary, PaLM-E could be paving a way toward it as well. https://ai.googleblog.com/2023/03/palm-e-embodied-multimodal...

You might be interested in this thread by a DeepMind’s research scientist. https://mobile.twitter.com/AndrewLampinen/status/16396602197...

>Exponential curves are theoretical constructs: all actual phenomena are S-shaped.

This. I've been thinking much the same recently, although not expressed so succinctly. Nothing in nature is ever exponential (for very long).

the paper: https://arxiv.org/pdf/2303.11366.pdf

it reminds me of the thinking-fast vs. thinking-slow dichotomy. Current llms are the thinking fast type. Funnily people’s complains about its errors are reminiscent of this. It answers just to quick and only with its instant response neural net. A thinking slow answer would be more akin to a chain of thought answer. Allowing the llm a more flexible platform than CoT promptin might well be the next step. Of course it would als multiply compute cost. So it might not be in your 20$ subscription

A narrower question: can we perhaps stop putting AGI and ChatGPT in the same paragraphs as if they are somehow relevant to each other? Intelligence has very little to do with a glorified Google search trained by statistical crunching of superhuman amounts of data; there is not even a chicken-sized trace of intelligence in ChatGPT that is not a reflection of the training set or of the embedded human-designed models it uses to mimicry problem solving and conversation.

Several necessary ingredients of human intelligence are present in GPT-4: complex pattern matching, abstraction from concrete examples and apply the abstract patterns to new examples, pattern interpolation, basic reasoning.

This is evident by its ability to generalize from the training set to new problems within many domains.

It's still unable to generalize as well as a smart human beyond the distribution it was specifically trained on, which is evident by its poor performance on AMC, Leetcode medium and hard, and Codeforces problems. But most humans are not great at these kinds of problems either.

Benchmark and test results: https://openai.com/research/gpt-4

> A larger question: Are we hurling ourselves toward a (near) future of unaligned AGI with self-improvement capabilities?

We are running towards a brick wall and people are not paying attention. Setting up self-reflection loops today is actually fairly trivial and can be done programmatically, all the model needs is to produce a solution, invoke the evaluation and keep iterating.

so how do we know chat gpt didn't have humaneval in training data?

The point is adding a couple components can improve GPT-4 significantly, as shown above. The data it originally trained with is presumably held constant in the evaluation above.

the point is if humaneval was in gpt training data, then this component improved memorization from mediocre to Ok-ish, and actual coding skills still not tested.

From other people’s and my experiences, GPT-4 can do more than simply memorizing. It can at least interpolate and reason a little bit too.

A few other tests show that GPT-4 would achieve much better results than 67% for something it has sufficient training data on like GRE Verbal and AP Macroeconomics.

https://openai.com/research/gpt-4

Yes, it still can’t generalize properly outside its training distribution. However, when armed with feedback and self-reflection, it seems better at that too.

yup. we're (we=the public) far from getting access to the full model. that said one of the commentors in the twitter thread brings up how openapi isnt being fully forthcoming about their methods.

AI Explained has a good summary of many of these topics

Reply on news.ycombinator.com