Learning something with GTP-4 is so much more fun than using google/Stack Overflow.
It's like having a tutor that knows your code and gives you hints and solutions based on your actual project. It's a real game changer for learning
It's like having a tutor that knows your code and gives you hints and solutions based on your actual project. It's a real game changer for learning
If they embrace a chatGPT interface as a first line of defense for users questions, it may improve the quality of real questions, reduce repetition, and filter through actually good and compelling questions. LLMs still need systems that encourage high quality data generation from humans to improve its model. I would like to imagine a world where there is a win-win for AI and the website we've relied on so much over the years.
Step 2: They implement GPT-4 responses to questions
Step 3: The community + context provide the same feedback
Step 4: GPT-5/6/7 gets progressively better
It has seen those Stack Overflow responses and if you ask it to give you more context about a certain approach, it will.
I've learned when asking GPT-4 for help to push past the first answer, or ask it to offer more solutions. Also to ask why it choose a certain approach.
Quite often it will give me a response that doesn't fit well with my code, and I'll say "I don't want to do it this way, please give me another option, keeping in mind that [some important aspect of the code base]."
It then spits it out and it's usually way closer. Iterate on this and it's often quite good.
I think you're completely right if a person were copy pasting the first response every time, but that person is simply using it wrong in my opinion.
I ran into this the other day and it's made me very leery about using chatGPT for learning things I don't already know. I moved on from toy examples and tried getting it to write a program in a domain I was already intimately familiar with: I immediately realized that this thing is going to help people churn out all sorts of code that is a poor fit for the surrounding system or ecosystem.
Basically I had it write a program in Elixir as a GenServer. After that was done I asked it how I could integrate that solution into a Phoenix web application. I had to hold chatGPT's hand, so to speak, as we evolved the solution from:
(1) Spawning (and linking!) the GenServer to a controller i.e. in response to a web request.
(2) Spawning and linking the GenServer to the application but outside the supervision tree.
(3) Spawning the GenServer inside a supervision tree.
Now technically the application could have worked, more or less, if you ran it at any one of those intermediate steps. However only the last solution is the robust and idiomatic one. It felt almost like a constraint solver to me: we iteratively arrived at the right answer by adding more and more constraints to the solution. The problem, as it relates to a novice, is that they don't have a list of constraints rattling around in their head. (Language idioms, best practices, framework knowledge, system architecture knowledge, etc.)
The insidious thing here is that 1 or 2 are syntactically valid programs. If you try to beat 1 and 2 into submission you're going to have a subtly broken system, one that makes a lot of Elixir programmers very sad.
---
I'll be watching the Khan Academy guys pretty closely, because they seem to have put a lot of thought into how students could use these systems safely and effectively.[1] I think the answer is going to be we need AIs that are aimed at professionals, and AIs that are aimed at education. The hard part will be that somehow we need to direct people to use one appropriate to their skill level.
[1]: https://www.ted.com/talks/sal_khan_how_ai_could_save_not_des...
One minor point: In your demo, you state that "Today, almost 50% of code is written by AI, so why shouldn't you learn how to code with AI?"
Citation? I thought this might be true of developers using Copilot (based on their recent publications), but even that seems a bit of a stretch. I guess you could claim this because it's so easy to generate LOTS of code with an AI model. How much of that code is used, though? Really 50% of ALL code written now?
"Today, GitHub Copilot is behind an average of 46 percent of a developers' code across all programming languages -- and in Java, that number jumps to 61 percent." - https://visualstudiomagazine.com/articles/2023/02/15/copilot...
> Case in point: When we first launched GitHub Copilot for Individuals in June 2022, more than 27% of developers’ code files on average were generated by GitHub Copilot. Today, GitHub Copilot is behind an average of 46% of a developers’ code across all programming languages—and in Java, that number jumps to 61%.
I'm not sure what this means. GitHub does not have the ability to measure Copilot usage across all code written anywhere in the world, so surely it can't be that. Do they mean code generated by Copilot as a fraction of code produced by someone using an IDE with the Copilot plug-in? That number still seems high to me.
[1] https://github.blog/2023-02-14-github-copilot-now-has-a-bett...
So it wouldn't surprise me. For repetitive code it's pretty good. And even with non repetitive code, once you start typing a bit it is good at finishing it.
Especially in a verbose language like Java. How much of copilot use is something that autocomplete would have handled anyways?
BTW, in some countries, you learn binary in grade school :)
But still, I treasure the old ways I learned how to program. It was such a fulfilling experience by making newbie mistakes and getting corrected in hard ways, and you really only get the authentic insight by going through that process, and by reading all those important books chapter by chapter. It's probably not as "fun" as using GPT-4 though.
At this point, i am in a place where GPT gives me good solutions that might not work vs having to go through all the noise Googling stuff to find a working solution.
But it is great just to learn different concepts at a high level. Helps me clear my mind when coming up with designs
Decide for yourself whether that's worth it based on other people's experiences. Personally, I'll pass.