Hacker News Reader

Comment by LeanderK | Hacker News Reader

I love those ChatGPT projects! Of course it's silly and nobody is really replacing somebody with a program that confidently get half it's answers wrong. But it's fun to just naively let ChatGPT solve the problem.

But I wonder what it's going to look like in a few years. Currently, it's really just a demo that got surprisingly huge traction. I think the most pressing problem is not to get ChatGPT smarter but to get it more reliable. I think more realistic use-cases would emerge if we could build systems that have a better understanding when they are out of breath. I don't think this needs a revolutionary breakthrough just more science.

View on HN

> nobody is really replacing somebody with a program that confidently get half it's answers wrong

I gotta tell you, I use copilot to help with my coding, and it still sends a shiver down my spine when it writes the entire database migration based off of the filename, or writes 4 pages of automated tests that work right on the first try.

Which programming languages do you use?

> writes 4 pages of automated tests that work right on the first try

Not sure that’s the right criterion to assess the quality and adequacy of tests.

But my GitHub badge says I have 105% test coverage! That means there are negative bugs, right...?

I'd be impressed if AI wrote a test suite that immediately found a bunch of valid bugs. I'd be nonplussed if it wrote a test suite that "passed" on the first try. The presence of passing tests does not imply an absence of bugs, and the hard part of developing a test suite is the test design, not the implementation.

lol, fair enough, but I would never commit a line of code I hadn't carefully read myself. my new class was basic, so the new unit test was basic as well, but it was thorough and correct. And I exaggerated a bit; I did write 3 or 4 comments to give the ai some idea of where I was heading.

I've had it freak me out with how prescient can be, but it also gets a lot of things subtly wrong, like its programming against an API similar to the one in my application, but not identical.

Do you check that those tests can fail at all? That seems like a common human error, might be present here?

Reply on news.ycombinator.com