Teaching Large Language Models to Self-Debug
arxiv.org
arxiv.org
Explain to them how to debug and fix the code they've written.
Which is pretty much what you would do with an inexperienced human software developer.
Looking at this with fresh eyes, it's both shocking to me that this sort of thing is even possible, and yet also completely unsurprising as yet another emergent capability of LLMs.
We live in interesting times.
You can teach GPT-3 arithmetic - https://imgur.com/a/w3DAYOi
Basically 100% accuracy up to about 13 digit addition and >90 after that.
What else can you teach GPT without changing weights ?
The graphs you just posted do not support that, they'd support at most 100% accuracy up to 4 digits.
This is such a circular thing, that I feel like it is amazing to see it.
The reason LLMs use a NN is because they're trying to encode a probability function for generating the passage.
And now, you are encoding another n-gram follower exercise (i.e 1+1 = 2) on top of it :)
Beware of bugs in the above code; I have only proved it correct, not tried it. - Knuth
It usually does notice inconsistencies between A and B when asked this. But its ways of reconciling inconsistencies can be bizarre and suggest a very superficial understanding of concepts.
For example, it once reconciled an inconsistency by saying that, yes, 2 * 2 = 4, but if you multiply both sides of that equation by a big number, that's no longer true.
I will be super impressed the day we have a model that can read an arithmetic textbook and come out with reliable arithmetic skills.
> 2 * 2 = 4, but if you multiply both sides of that equation by a big number, that's no longer true.
You know, the very same scenario I pulled from your comment?
Fair enough, have you explained it the axioms of arithmetic? It only has memorized examples that it has seen, it has a right to be skeptical until it's seen our axioms and proofs about what is always true in mathematics.
When I was a child I was skeptical that an odd number + an even number is always odd etc for very large numbers until I saw it proven to me by induction (when I was 6, I think, imo this was reasonable skepticism).
Now, ChatGPT probably has seen these proofs, to be fair, but it may not be connecting the dots well enough yet. I would expect this in a later version that has been specifically trained to understand math (by which I really mean math, and not just performing calculations. And, imagine what things will prove for us then!)
I think one problem with these models is that all their knowledge is soft. They never learn true, universal rules. They seem to know the rules of grammar, but only because they stick to average-sounding text, and the average text is grammatical. At the edges of the distribution of what they've seen, where the data is thin, they have no rules for how to operate, and their facade of intelligence quickly falls apart.
People can reliably add numbers they've never seen before. The idea that it would matter whether the number has been seen before seems ridiculous and fundamentally off-track, doesn't it? But for GPT, it's a crapshoot, and it gets worse the farther it gets away from stuff it's seen before.
But if anyone wants to give it to me for free, I would happily make a $1000 bet that I can get GPT-4 to make the same mistake.
If you provide the inputs for some specific task where you expect GPT-4 to fail in this manner, I can give it a try.
Like as far as the model is concerned, how can it distinguish between the task being "do your best but if you do make an error, correct it" and "make some mistakes like in this example and then fix them".
For decades in reinforcement learning we've had Q learning, which promises to solve any optimization problem if only we can build a powerful enough function approximator. It can even learn off-policy, meaning it can just watch from the sideline and find the optimal solution. It works for toy problems, and it works in theory, theres even formal proofs that it will work given infinite time and resources, and yet in practice it often becomes unstable and collapses.
Supervised learning is one thing, having a model remain stable while bootstrapping through a complex environment is another. GTP is supervised learning, so far, let's see if it can bootstrap.
Putting aside the incongruity of Google researchers using the OpenAI model, I'm curious how GPT-4 would do in this situation. Probably its zero shot attempts at coding would be better, and maybe its self criticisms would be better too.
FWIW if you ask it how transformers could be improved, it basically gives a recap of various relevant papers that it knows: Longformer, BigBird etc. Amazingly, all the papers cited actually exist and have the specified titles, names, and dates. But then if you ask it for some ideas of its own:
> Can you think of other novel improvement ideas that haven't been investigated yet, to the best of your knowledge? What makes the most sense based on your understanding of the subject matter.
As an AI language model, my knowledge is derived from the text data I have been trained on, which includes research papers and articles published up to September 2021. While it's difficult to guarantee the novelty of any idea, here are a few suggestions for potential improvements that may not have been thoroughly investigated yet:
Context-Adaptive Layers: Instead of using the same set of layers for all input sequences, consider using context-adaptive layers that are dynamically selected or generated based on the input's characteristics. This could allow the model to adapt its processing strategy to different input types, such as formal vs. informal text, or narrative vs. expository text, which might improve its performance on diverse tasks.
Multi-modal Fusion: Develop techniques to better fuse information from multiple modalities, such as text, images, and audio, within the Transformer architecture. Improved multi-modal fusion could lead to enhanced performance on tasks that require understanding and generating content across different modalities, such as visual question answering or image captioning.
Meta-Learning for Architecture Search: Employ meta-learning techniques to search for the best Transformer architecture for a specific task or domain. By learning to optimize the architecture itself, the model might be able to discover more effective configurations or components that have not been considered before.
Transformer Modules for Specific Tasks: Design specialized Transformer modules tailored for specific NLP tasks, such as syntactic parsing, named entity recognition, or sentiment analysis. These task-specific modules could be combined with the generic Transformer architecture to create a more powerful and adaptable model.
Hierarchical Transformers: Explore hierarchical architectures where multiple layers of Transformers work together to process information at different levels of abstraction. This could help the model capture both low-level patterns, such as local dependencies, and high-level patterns, such as long-term dependencies and semantic relationships, more effectively.
Enhanced Tokenization Strategies: Investigate novel tokenization strategies that better capture the structure and meaning of the input text. This could involve using character-level, subword-level, or even syntax-based tokenization to better represent the input for the Transformer model.
Regularization Techniques: Develop novel regularization techniques specifically tailored for the Transformer architecture. This could help improve generalization and prevent overfitting, especially when training large-scale models with billions of parameters.
I’ll admit that I only have had time so far to read the abstract, and I’m not sure what their baseline is, but a 2-3% improvement doesn’t sound like a quantum leap forward that you’d expect from the title. Heck, I’d think that’s likely within expected sampling errors.
I’m not sure about others’ experience and, while I keep reading articles showing impressive seeming examples, my few forays into attempting to get ChatGPT to write code were actually completely useless. Even with follow on prompts to correct itself.
The other day I asked it what covid case fatality rates were in 2020. After all the various opinions at the time, I was curious to see what it was pre-vaccine. It would alternately tell me that it couldn’t give me data for 2020 because it only had data up to Sep. 2021, and then give me wildly varying numbers.
Is this a Rocko’s Basilisk trying to lure me into a false sense of security… haha.
With respect to GPT etc. as a copilot, the current dialogue seems to focus on "ask for GPT to generate code to do X" then "just paste in the error message to fix bugs in the code GPT generates"
A.) Why is GPT generating code that results in simple compiler errors (that is why GPT probably shouldn't be used to generate any code / replace devs for real projects yet), and
B.) error messages are (just guessing here) probably <1% of the actual errors in most codebases.
I personally know of a few large companies laying off devs over this.
IMO, the tech debt we're going to see in 6 months will probably be huge. Good now to start a staffing agency of human experts who can come in and fix this type of problem (extricating massive amounts of code generated by GPT without starting from scratch) because there will be a bunch of fires to put out and those fires will be worth $
Neither of those three would function nor would they throw an error. Prompts to correct itself would not improve things.
So I did the natural thing and started to debug myself. At which point, I couldn’t help but ask myself why I was debugging machine generated code when I could not be lazy and actually build it from first principles.
They’re laying people off and replacing them with chat gpt generating code? That seems... aggressive. Or are they laying off devs who copy-pasted gpt-generate code?
The way I see it, if you code without it, you won't compete with the speed and value.
And they are not going to back fill those roles
I’ll admit my bias of having seen enough vacuous industry hype over the years to be naturally skeptical. Heck, I worked in a marketing department once where I helped manufacture the stuff (forgive me father for I have sinned, in my defense they did fire me after three months in a layoff). But my few personal experiences with ChatGPT were pretty disappointing and I’m actually looking for someone to tell me otherwise.
My personal use of gpt4 (also daily) is: correct, rephrase spelling from my brain dump, make python plots (stylize, convert, add subplots, labels, handle indexing when things get inverted), makw short shell scripts (generated 2FA, login vpn through console using 2fa, make script of disabling keyboard etc), and help debug my code (my situation is this, here's some code, what do you suggest?).
The last part is an interesting one for people with attention deficiency disorders, like myself, where procrastination can be conquered when there's an assistant that keeps you on track.
Aha, I’ll def have to give it a whirl. My procrastination ability is world class.
You can only make employees more productive.. this in turn could, in theory, lessen the need for developers in the long run, but it assumes the company will not bother to use the extra bandwidth for other projects.
And exactly like this some future brain implant thing will also put another layer of pressure. People will get it as it'll give them an edge on certain fronts.
This is exactly what would happen if ChatGPT was actually a productivity boost for senior devs. I don't know why some idiots on here keep insisting businesses want to get rid of people when that's not how the game works at all. Extra work capacity will always be used. Regardless of what payroll costs, what's always more important is the ROI.
Nah they deserve to eat shit and the staffing agencies hired to fix the bad AI code will undoubtedly be people abroad who barely speak English and will only tangle it up worse. I would actually pay to be a fly on the wall in those meetings listening to people lose their minds in frustration.