Hacker News Reader

Comment by lyair1 | Hacker News Reader

I am happy that so many people get to experience the feeling me and other software engineers feel when we write code. OpenAI democratize some of this feeling and now people that never knew how to code can experience the rush of creating something new using code.

In a world where a lot of AI Researchers that develop these amazing models are slowed down by their lack of "normal" engineering skills I think it's very apparent that software engineering goes well beyond writing a snippet of code.

Like the internet, stackoverflow, forums and other advances in technology that opened new doors and opportunity and democratized the industry - this is another one, a big one! the smart ones will use this push as an entry point to the software development world rather than looking for an oracle function for software development (which does not exist yet)

View on HN

> the smart ones will use this push as an entry point to the software development world rather than looking for an oracle function for software development (which does not exist yet)

Very much agree. I think mentorship becomes even more important now. I have a friends that's switching careers and starting to learn software engineering. Just for fun, I back-seated him while he tried to build a really simple app using GPT-4.

Even with really fine-tuned prompts, some of which I helped with, the code that was generated was either barely readable, had performance issues, and/or in some cases was just completely broken. At the time, I made the quip that for junior/entry-level folks it's like getting cybernetic implants in your legs before you know how to walk or run.

People who use ChatGPT to spit out code as a method of learning software engineering are going to be seriously deficient in fundamentals like performance.

I've already worked with way too many engineers who copy/paste code without understanding what it does.

I've seen similar results with teams I work with. The strongest engineers who can spot the deficiencies or errors in LLM generated code generally get a huge productivity boost, as they can compare the "shape" of the output to what they would expect for a given problem.

The junior devs, lacking the experience that provides the context or knowledge to analyze the "shape", let all manner of minor and major issues from the generated code slip past.

I really notice this particularly with say less popular frameworks or code libraries, where many LLM code generators appear much more likely to hallucinate invalid code. I see similar issues with DSLs too, where the LLM starts to just guess at the DSL syntax. I really hate when an engineer new to a framework just goes to the LLM first, without at least spending a little time learning the framework for the task.

All said, I suspect these code generation tools will get good enough that the rate of errors goes down enough it maybe won't matter in the longer term.

I think this is currently an open question.

It may turn out that ChatGPT results in a new generation of developers who produce terrible code, full of flaws, with no understanding of how it actually works.

Or... it might be that ChatGPT helps newcomers get over that horrendous initial learning curve, start by writing bad code... and then get better at it.

I'm currently cautiously optimistic that the latter scenario is going to win out.

In my experience newcomers and junior engineers don't get better by writing more bad code. They learn mostly by 3 methods. 1) finding flaws in their code the hard way (slow) 2) getting careful feedback from more experienced people (has to be done well) and 3) reading better code for a problem they already understand.

I'm not sure GPT helps with any of those yet, as it seems to be more likely to spit out incorrect-to-sortof-okayish so far.

It goes the other way too. I've had it generate code using better methods or features new to me, when I thought I knew the language well.

I'm the hallucinating LLM? And Other Such Crises it has given me.

I'm on year two of my journey to become a game developer, i am focusing on programming and if i realize i don´t want to work with games in the future at least i have solid knowledge on how to deal with a broad area of software problems.

I don´t rely on ChatGPT to write any code for me but i do use it whenever there´s a concept i don´t understand and if there´s any problem blocking me from getting forward in my many programming books. Having a tireless teacher/mentor with extensive knowledge and who does not care however stupid question you ask is a very good tool for learning.

I share your optimism.

ChatGPT could also empower individuals who may not have considered learning programming to create practical programs that enhance their daily routines, similar to how VB6 significantly reduced the barrier to entry for developing various applications.

I don't think it will help people learn as much as having to write everything yourself. Sure it will get them productive 10x faster, as a shortcut. You can only get so good at riding a bicycle by watching others ride a bike. Remember that caching issue that cost you hours of debugging? You don't forget that. Maybe the generated code doesn't produce a bug this time, but the 5th or 6th time? It's hard enough to debug code you wrote yourself sometimes, let alone generated code.

The good news is that those copy/pasters are probably going to be copy/pasting higher quality code if they’re using GPT4.

I don't think this is good news, it's at best not-bad news. We're already inundated with content, most of it copycatting other content, and it's replacing meaningful things, like archetypal stories and philosophy. Look at how people describe Marvel movies and Star Wars shows - it's content, not stories. Scaling the ability to just repeat things without an understanding of those things will at best just add more content. It will likely contribute to the further creation of complex systems people don't understand and can't control.

"Um, I'll tell you the problem with the scientific power that you're, that you're using here. It didn't require any discipline to attain it. Ya know, you read what others had done, and you, and you took took the next step. You didn't earn the knowledge for yourselves, so you don't take any responsibility for it. You stood on the shoulders of geniuses, uh, to accomplish something as fast as you could..." (Emphasis mine).

https://www.youtube.com/watch?v=4PLvdmifDSk&t=61s

Just about any original scientific text or undertaking begins with a literature review.

But the awesome part is that you can ask it to explain parts and it usually does pretty well.

I was using it for debugging WINE - where I have no DirectDraw4 / DirectX6 experience, and it's much nicer than trying to trawl through ancient MS documentation myself.

If performance is a concern, just re-prompt GPT if there are any performance concerns, and to propose fixes for them if there are.

This is called reflexion and has been found to be an effective way to improve response accuracy and usefulness (also reduces hallucinations).

https://arxiv.org/abs/2303.11366

I don't like the use of the word "democratize" in this context, there's no secret authority of programmers preventing people from learning. You need only a computer, which is practically a commodity in first world countries; the users really have all the power and resources they could ever possibly need. I don't see how it could get more democratic at all.

The reason people can't code isn't because an external force is preventing them, it's because they aren't putting the work in.

I don't know how to paint, is the solution to "democratize" painting by waiting until an AI can do it for me?

You don't get that feeling around solving complex logic puzzles in your brain. You do get to copy and paste

I use GPT-4 for helping with my code all the time and I still get that feeling, because once the code base is sufficiently complex there's no avoiding the logic puzzles.

Maybe one day it can look at your entire code base and just fix it / writea new feature, but it's definitely not there yet.

The models AI researchers develop are cutting edge, and mathematically involved; the kind of thing I would expect an LLM to fare poorly at. And today's researchers are capable of implementing their models in pytorch and the like.

Reply on news.ycombinator.com