Hacker News Reader

Comment by digitalsushi | Hacker News Reader

If I had a little robot riding in the passenger seat that could tell me whether to go left, straight, or right, and it was correct 90% of the time, I'd think that was pretty great. I'd get where I needed to be, even with a couple mishaps.

ML code suggestions are the same thing to me. If I don't know where I am going, I can just ask it for suggestions. And it's probably going to be what I want.

In both cases, I am annoyed with myself for having started before I knew where I want to end up.

View on HN

The problem with ML is that it's pattern recognition, it's an approximation. Code is absolute, it's logic that is interpreted very literally and very exactly. This is what makes it so dangerous for coding; it creates code that's convincing to humans but with deviations that allow for all sorts of bugs. And the worst part is, since you didn't write the code, you may not have the skills (or time) to figure out if those bugs exist, especially if the ML is extremely convincing/clever in what it writes. I would argue that this overhead is even worse for productivity over just writing it yourself.

Expand your view. AI can write tests, read error messages, find bugs in your code, we just need to give this task.

Let's think about tests. You write a function, the AI writes a few tests for you. maybe you need to add a few more. But it's better to have tests, and you might have missed one of them.

Error messages - we rely on error messages to make this leap from "code parrots" to "bug free". Most of our codes fail the first time we run them. We're just fancy pattern matchers too, but we have a runtime. So the AI could also fix its bugs all alone, given the opportunity.

Finding bugs - we can train AI to spot bugs. It can become an excellent tool to check not just AI code, but also human code. Having a bug detector running in the background would be great, even if is not perfect.

All those things are susceptible to the same issue: the ML model can generate test cases and error messages that are convincing to you and me regardless of whether they're actually right. Don't get me wrong, ML will reach a point someday where it catches up to humans in this respect; this is merely a shortcoming of today's ML, not tomorrow's, and it highlights where ML's weaknesses currently lie (in contrast to more vaguely defined goals like artwork).

Yup the next iteration of ML needs to focus more on proofs. Okay you know this but how do you know it, show me the proof

It's about refining the approximations it makes until those approximations are too tiny/negligible to create failures in code.

It doesn't work in "closed book" form, the only way you can do that now is to have good tests and let the language models try many attempts.

Your example hinges on at least two things

1) How many turns do you take on a particular trip

2) How do those wrong turns end up? if it's "travel time extended by 30 seconds" or "My car, the car I hit and the side of this building are all in shambles" changes what a 10% failure rate means a lot.

Right? Took a turn the wrong way down a one way street. Hit a lady with a stroller who was looking the other way. She is a surgeon. Her husband is a lawyer. You killed the kid. Your life is over.

On the other hand, you did just describe the early days of both consumer GPS navigation and things like mapquest. There were countless stories and gleeful news articles about people turning the wrong way down a one way street, driving into ponds, getting stuck on logging roads and all sort of other chaos caused by inattentive drivers and blind faith in flawed systems. But the key takeaway I think is that in the end, consumer GPS succeeded. Bugs were fixed and now everyone does have a little robot in their pocket that can tell them how to get somewhere.

90% right is superficially impressive but in actual practice is abysmal.

Voice recognition software needed to get to 99.9% accuracy to be actually useable.

The difference is that with code suggestions, you don't necessarily notice errors, even extremely important errors.

> If I had a little robot riding in the passenger seat that could tell me whether to go left, straight, or right, and it was correct 90% of the time, I'd think that was pretty great. I'd get where I needed to be, even with a couple mishaps.

Pretty sure you wouldn't when it starts telling you to make illegal turns, or tell you to turn where there are no roads etc, that is the state of language models for code. You'd likely run over a person or get stopped by police at some point if you listen to it, then it is easier to just turn it off so you can focus on driving. A bad assistant is much worse than no assistant.

Edit: And these models are more like Teslas autopilot than an assistant giving you directions, since you have to oversee what it does instead of letting it tell you what to do. An autopilot that does the wrong thing 10% of the time is horrible.

One in ten turns being told to do the wrong thing sounds incredibly annoying. Maybe if you’re driving on the highway for long stretches, but otherwise that would be awful.

Reply on news.ycombinator.com