Hacker News Reader

Comment by CapmCrackaWaka | Hacker News Reader

This depends entirely on _how_ the code is wrong. I asked chatGPT to write me code in python that would calculate SHAP values when given a sklearn model the other day. It returned code that ran, and even _looked_ like it did the right thing at a cursory glance. But I've written SHAP a package before, and there were several manipulations it got wrong. I mean completely wrong. You would never have known the code was wrong unless you knew how to write the code in the first place.

To me, code that is 95% correct will either fail catastrophically or give very wrong results. Imagine if the code you wrote was off 5% for every number it was supposed to generate. Code that is 99.99% correct will introduce subtle bugs.

* No shade to chatGPT, writing a function that calculates shap values is tough lol, I just wanted to see what it could do. I do think that, given time, it'll be able to write a days worth of high quality code in a few seconds.

View on HN

The thing about ChatGPT is that it warning shot. And all these people I see talking about it, laughing about how the shooter missed them.

Clearly ChatGPT is going to improve, and AI development is moving at a breakneck pace and accelerating. Dinging it for totally fumbling 5% or 10% of written code is completely missing the forest for the trees.

Sure, it will improve, but I think a lot of people think "Hey, it almost looks human quality now! Just a bit more tweaking and it will be human quality or better!". But a more likely case is that the relatively simple statistical modeling tools (which are very different from how our brains work, not that we fully understand how our brains work) that chatGPT uses have a limit to how well they work and they will hit a plateau (and are probably near it now). I'm not one of those people who believe strong AI is impossible, but I have a feeling that strong AI will take more than that just manipulating a text corpus.

I'd be surprised if it did only take text (or even language in general), but if it does only need that, then given how few parameters even big GPT-3 models have compared to humans, it will strongly imply that PETA was right all along.

Anyone who has doubts has to look at the price. It’s free for now, and will be cheap enough when openai starts monetizing. Price wins over quality. It’s demonstrated time and time again.

Depends on the details. Skip all the boring health and safety steps, you can make very cheap skyscrapers. They might fall down in a strong wind, but they'll be cheap.

After watching lots of videos from 3rd world countries where skyscrapers are built and then tore down a few years later, I think I know exactly how this is going to go.

It does depend on the details. In special fields, like medical software, regulation might alter the market—although code even there is often revealed to be of poor quality.

But of all the examples of cheap and convenient beating quality: photography, film, music, et al, the many industries that digital technology has disrupted, newspapers are more analogous than builders. Software companies are publishers, like newspapers. And newspapers had entire building floors occupied by highly skilled mechanical typesetters, who have long been replaced. A handful of employees on a couple computers could do the job faster, more easily, and of good enough quality.

Software has already disrupted everything else, eventually it would disrupt the process of making software.

Yeah, but people were also saying this about self-driving cars, and guess what that long tail is super long, and its also far fatter than we expected. 10 years ago people were saying AI was coming for taxi drivers, and as far as I can tell we're still 10 years away.

I'm nonplussed by ChatGPT because the hype around it is largely the same as was for Github Copilot and Copilot fizzled badly. (Full disclosure: I pay for Copilot because it is somewhat useful).

I wonder if some of this is the 80 20 rule. We're seeing the easy 80 percent of the solutions which has taken 20% of the time. We still have the hard 80% (or most of) to go for some of these new techs

Replacing 80% of a truck driver's skill would suck but replacing 80% of our skill would be an OK programmer.

Considering the deep conv nets that melted the last AI winter happened in 2012, you are basically giving it 40 years till 100%.

Tesla makes self-driving cars that drive better than humans. The reason you have to touch the steering wheel periodically is political/social, not technical. An acquaintance of mine read books while he commutes 90 minutes from Chattanooga to work in Atlanta once or twice a week. He's sitting in the driver's seat but he's certainly not driving.

The political/social factors which apply to the life-and-death decisions made driving a car, don't apply to whether one of the websites I work on works perfectly.

I'm 35, and I've paid to write code for about 15 years. To be honest, ChatGPT probably writes better code than I did at my first paid internship. It's got a ways to go to catch up with even a junior developer in my opinion, but it's only a matter of time.

And how much time? The expectation in the US is that my career will last until I'm 65ish. That's 30 years from now. Tesla has only been around 19 years and now makes self-driving cars.

So yeah, I'm not immediately worried that I'm going to lose my job to ChatGPT in the next year, but I am quite confident that my role will either cease existing or drastically change because of AI before the end of my career. The idea that we won't see AI replacing professional coders in the next 30 years strains credulity.

Luckily for me, I already have considered some career changes I'd want to do even if I weren't forced to by AI. But if folks my age were planning to finish out their careers in this field, they should come up with an alternative plan. And people starting this field are already in direct competition to stay ahead of AI.

I'm doubtful - There's a pretty big difference between writing a basic function and even a small program, and that's all I've seen out of these kinds of AIs thus far, and it still gets those wrong regularly because it doesn't really understand what it's doing - just mixing and matching its training set.

Roads are extremely regular, as things go, and as soon as you are off the beaten path with those AIs start having trouble too.

It seems that in general that the long tail will be problematic for a while yet.

I was of the impression that Tesla's self driving is still not fully reliable yet. For example a recent video shows a famous youtuber having to take manual control 3 times in a 20 min drive to work [0]. He mentioned how stressful it was compared to normal driving as well.

[0] https://www.youtube.com/watch?v=9nF0K2nJ7N8

If you watch the video you linked, he admits he's not taking manual control because it's unsafe--it's because he's embarrassed. It's hard to tell from the video, but it seems like the choices he makes out of embarrassment are actually more risky than what the Tesla was going to do.

It makes sense. My own experience driving a non-Tesla car the speed limit nearly always, is that other drivers will try to pressure you to do dangerous stuff so they can get where they're going a few seconds faster. I sometimes give into that pressure, but the AI doesn't feel that pressure at all. So if you're paying attention and see the AI not giving into that pressure, the tendency is to take manual control so you can. But that's not safer--quite the opposite. That's an example of the AI driving better than the human.

On the opposite end of the social anxiety spectrum, there's a genre of pornography where people are having sex in the driver's seats of Teslas while the AI is driving. They certainly aren't intervening 3 times in 20 minutes, and so far I don't know of any of these people getting in car accidents.

> [...] Copilot fizzled badly. (Full disclosure: I pay for Copilot because it is somewhat useful).

In what sense did Copilot fizzle badly? It's a tool that you incorporated into your workflow and that you pay money for.

Does it solve all programming? No, of course not, and it's far from there. I think even if improves a lot it will not be close to replacing a programmer.

But a tool that lets you write code 10x,100x faster is a big deal. I don't think we're far away from a world in which every programmer has to use AI to be somewhat proficient in their job.

The thing is though, it's trained on human text. And most humans are per difinition, very fallible. Unless someone made it so that it can never get trained on subtly wrong code, how will it ever improve? Imho AI can be great for suggestions as for which method to use (visual studio has this, and I think there is an extension for visual studio code for a couple of languages). I think fine grained things like this are very useful, but I think code snippets are just too coarse to actually be helpful.

Improve itself through experimentation with reinforcement learning. This is how humans improve too. AlphaZero does it.

The amount of work in that area of research is substantial. You will see world shattering results in a few years.

Current SOTA: https://openai.com/blog/vpt/

This is magical thinking, no different than a cult.

The fundamental design of transformer architecture isn't capable of what you think it is.

There are still radical, fundamental breakthroughs needed. It's not a matter of incremental improvement over time.

Excellent summation. Majority of the software developers work on crud based frontend or backend development. When this thing's attention goes beyond the 4k tokens its limited to, there will be very less number of developers needed in general. Same way less number of artists or illustrators will be needed for making run of the mill marketing brochures.

I think majority wouldn't know what hit them when the time comes. My experience with chatgpt has been highly positive changing me from a skeptic to a believer. It takes a bit of skill to tune the prompts but I got it to write frontend, backend, unit test cases, automation test cases, generate test data flawlessly. I have seen and worked with much worse developers than what this current iteration is.

I experienced ChatGPT confidently giving incorrect answers about the Schwarzchild radius of the black hole at the center of our galaxy, Saggitarius A-star. Both when asked about "the Scharzchild radius of a black hole with 4 million solar masses" (a calculation) and "the Scharzchild radius of Saggitarius A-star" (a simple lookup).

Both answers were orders of magnitude wrong, and vastly different from each other.

JS code suggested for a simple database connection had glaring SQL injection vulnerabilities.

I think it's an ok tool for discovering new libraries and getting oriented quickly to languages and coding domains you're unfamiliar with. But it's more like a forum post from a novice who read a tutorial and otherwise has little experience.

My understanding is that ChatGPT (and similar things) are purely language models; they do not have any kind of "understanding" of anything like reality. Basically, they have a complex statistical model of how words are related.

I'm a bit surprised that it got a lookup wrong, but for any other domain, describing it as a "novice" is understating the situation a lot.

Over the weekend I tried to tease out a sed command that would fix an uber simple compiler error from ChatGPT [0]. I gave up after 4 or 5 tries - while it got the root cause correct ("." instead of "->" because the property was a pointer), it just couldn't figure out the right sed command. That's such a simple task, its failure doesn't inspire confidence in getting more complicated things correct.

This is the main reason I haven't actually incorporated any AI tools into my daily programming yet - I'm mindful that I might end up spending more time tracking down issues in the auto-generated code than I saved using it in the first place.

[0] You can see the results here https://twitter.com/NickFisherAU/status/1601838829882986496

Who is going to debug this code when it is wrong?

Whether 95% or 99.9% correct, when there is a serious bug, you're still going to need people that can fix the gap between almost correct and actually correct.

Sure, but how much of the total work time in software development is writing relatively straightforward, boilerplate type code that could reasonably be copied from the top answer from stackoverflow with variable names changed? Now maybe instead of 5 FTE equivalents doing that work, you just need the 1 guy to debug the AI's shot at it. Now 4 people are out of work, or applying to be the 1 guy at some other company.

Does anyone remember the old maxim, "Don't write code as cleverly as you can because it's harder to debug than it is to write and you won't be clever enough"?

Or the company just delivers features when they are estimated to be done, instead of it taking 5 times longer than expected

> Sure, but how much of the total work time in software development is writing relatively straightforward, boilerplate type code that could reasonably be copied from the top answer from stackoverflow with variable names changed?

It may be a significant chunk of the butt-in-seat-time under our archaic 40hour/week paradigm, but it's not a significant chunk of the programmer's actual mental effort. You're not going to be able to get people to work 5x more intensely by automating the boring stuff, that was never the limiting factor.

Reply on news.ycombinator.com