Hacker News Reader

Comment by nibbleshifter | Hacker News Reader

I don't use them either.

I've played around with ChatGPT and Copilot a little, and found that they often are subtly, but very confidently wrong in their output when asked to perform a programming task.

Sure you could spend ages refining the prompt etc, but its going to be faster to just write the fucking code yourself in the first place most of the time.

Then there's the privacy/security concerns...

View on HN

I really doubt it would be faster to write code manually, even with the state of AI tools today. Even with very sophisticated keyboard macros and traditional autocompletion, someone using GPT would outperform anyone who doesn't. Think of the amount of boilerplate and tests you write, and tedious API documentation lookups you do daily; that all goes away with GPT. The amount of work to double check whether the generated code is valid, and fix it, is negligible compared to the alternative of writing it all manually.

Of course, I'm saying this without actually having used it for programming, so I might be way off base, but the feedback from coworkers who rely on even the now basic GitHub Copilot is that it greatly improves their productivity. I'm envious, of course, but I'm not willing to sacrifice my privacy for that.

People who are downvoting this: please set up and use GitHub copilot once, maybe for some auxiliary thing not connected to your main task.

It is not just a tool for students to write assignments. In experienced hands it can easily double your productivity.

I agree with this statement a lot. Using Copilot saves you a lot of tedium if you are comfortable with the language already. If you are new to the language, then it might trip you up a bit (at least in its current incarnation).

Here is an example where it helps. I tried to initiate a connection to a Mongodb server using Python. While i have used many databases before, I have never used Python and MongoDB together. So, i knew i would have to have some kind of MongoDB library, a connection Factory and a connection string. I could have googled all of these things.

I did the following in VS Code using CoPilot. def get_db():

   """Initialise a MongoDB connection to a local database"""

It then automatically filled in the rest.

   db = getattr(g, '_database', None)

   if db is None:

       db = g._database =

MongoClient('mongodb://localhost:27017/')

   return db

Notice above, that it knew i was using a flask environment and added the line getattr.

Why this is a productivity boost is that i did not have to alt-tab to a browser, search for "pythong mongodb tutorial example" and then type it out. I was able to do the whole thing from VS Code and since i use vsvim, i could do this without taking my fingers off of the keyboard.

This is the next jump since autocompletion. I like it.

And you will have no idea whether the solution it presents to you is idiomatic or recommended or contains some common critical flaw or is hopelessly outdated. How can you find out? Back to alt-tabbing to the browser.

Sure it may take a bit more time to get going, but then you'll get it right the first time and learn something along the way. Your copilot example is just another iteration of copy-and-paste some random snippet from StackOverflow in the hope that it will work, but without having seen its context, like from when is the post and what comments, good or bad, did it get.

I'd actually be pretty afraid of a codebase that is created like that.

Watch the demos where they provide GPT-4 with an API for performing search queries and calculations. These tool integrations are the next step and they will include specialized ones for using language and library docs. They could also be given access to your favourite books on code style or have access to a linter that they could use to cleanup and format the code before presenting it. The model is capable of using these tools itself when it is set up with the right initial prompt. Even now Copilot is pretty good at copying your code style if there is enough code in the repo to start with.

You have no idea if the alternative code you would have written would have been idiomatic or had some critical flaw.

We have 50+ years of software engineering wisdom to deal with these issues. Testing, Fuzzing, version control, code reviews, the whole gauntlet.

I mean... I certainly know which languages I can write idiomatic code in and which I cannot.

I can't know that my code will be free of critical flaws, but I do understand the common sources of flaws and techniques to avoid them, and I'm quite confident I can build small features like this that simply aren't vulnerable to SQL injection, on the first try and without requiring fuzzers or code review: https://infosec.exchange/@malwaretech/110029899620668108

You don’t have to accept the suggestions as-is. It’s just code, you can edit it as much as you like. Getting a good idiomatic starting point is a great boost.

I'm confident enough in most languages I write in to recognize correct code. But I am not usually so familiar that I can conjure the exact syntax for many specialized things I need. Copilot is just a much quicker way to get what I need without looking it up.

> You have no idea if the alternative code you would have written would have been idiomatic or had some critical flaw.

But I have a feeling for both, which is one of the key components of the skill in our trade.

For idionmatic code, I know the degree to which I'm following how things "should" be done or are "usually" done in a given language. If I'm uncertain, I know that. GPT won't tell me this. Worse, it will confidently claim things, possibly even if presented with evidence to the contrary.

For critical flaws, I know the "dark corners" of the code. Cases which were not obvious to handle or required some trick, etc. I'll test those specifically. With GPTs code, I have no idea what the critical cases are. I can read the code and can try to guess. But it's like outsourcing writing tests to a QA department. Never donna be as effective as the original author of the code. And if I can't trust GPT to write correct code, I can't trust it to write a good test for the code. So, neither the original author of the code (GPT) nor somebody external (me) will be able to test the result properly

"Back to alt-tabbing to the browser."

Yes because tutorial info on some rando's webpage is never out of date. /s

It is. I can see that it was written in 2003 and discard it. GPT won't tell me if its answer is based on an ancient lib version.

Essentially, GPT is that rando's webpage but with the metadata stripped away that allowed me to make judgement calls about its trustworthyness. No author, no time, no style to see if somebody is trolling.

My experience was always that most of my time isn’t deducted to writing code. Maybe 10%, the rest us thinking about how the code I write will fit into the existing architecture or accommodate future features.

In very near future your IDE will send the whole codebase as context to LLMs. Then instead of thinking up all possibilities you can just ask. LLM will suggest multiple alternatives and you can select the best when and ask it to implement it.

Don't make stuff up. There is no indication that will happen.

This seems very plausible to me. The context window improved a lot between GPT-3.5 and GPT-4, and OpenAI clearly see value in increasing it further.

Explain how it improved. Yes, it has improved accuracy (test taking) but it still gets stuck in the same loops over and over:

"response has issue A" -> point out issue A to GPT "response has issue B" -> point out issue B to GPT

GPT replies with the response that had issue A ...

This is not a tool that is going to be good at performing generic tasks. It just isn't.

I said that the context window improved. I mean that it is larger. GPT-3.5 is 4k tokens, GPT-4 is 8k tokens (standard) or 32k tokens (only API access atm). This is the number of tokens that GPT-X can take into account when producing a response.

Specifically, I was using this to support the statement "In very near future your IDE will send the whole codebase as context to LLMs." I'm not talking about loops or accuracy.

https://platform.openai.com/docs/models

It's true, but there is no indication that GPT can explain larger concepts for you, and negative indication it will be able to do it accurately.

It can't even explain small code to me unless it is something that it has been trained on. Often it gets even simple things wrong, either obviously, or worse, subtly wrong.

I agree that this is the part that needs more work, and is most uncertain. Increasing context windows seems like a fairly straightforward computational challenge (albeit potentially expensive). On the other hand, whether or not we can scale current models towards "true understanding" (or similar), is a total unknown atm.

I still think we will get useful things from scaling up current models though. I've already got a lot of value out of Copilot, for instance, and I'm looking forward to the next version based on GPT-4. Recently, I've been using the GPT-3 Copilot to write a lot of pandas/matplotlib code, which is fairly straightforward and repetitive, but as mainly a Java developer, I just don't have the APIs at my fingertips. Copilot helps a lot with this sort of thing.

> can scale current models towards "true understanding" (or similar), is a total unknown atm.

Right, but it's no more known than before GPT models IMO. It's the same unknown.

I don't mean to imply these language models are not impressive. They are pretty impressive.

GPT-4 has a model capable of using around 50 pages of written text worth of tokens (32,000) not sure exactly how many lines of code that translates to but it’s a lot. GPT-3 can use 4k so that’s a huge increase, the next version could be even larger and there are other ML techniques that allow for massive context lengths. Copilot already does a good job of refactoring code and knows enough about your code base to use your functions and methods. So what the other commenter said does not sound impossible to me.

Is there a proof that context length increase will result in a better result? It’s a possIbility, but not certain.

It isn't anywhere near being able to diagnose anything more than off-by-one and other common errors.

Identifying hose problems will bring a LOT of value - but it isn't going to program and do general problem solving for you! It just has no signs of being able to do that.

Yesterday’s Steve Yegge post talks about it. You can provide text as context but you can also provide a dense representation in the form of text embeddings that capture the context. Today you can manually do it by something like LangChain but in future it will be part of our text editors.

Yes it will be able to give amazing feedback to us as devs and quickly identify common problems (that I still make all the time, even after developing for a decade!) which will bring a ton of value to programmers.

But, it will be a tool - it won't be something that will solve general problems for you. It won't make an average programmer a great programmer.

It has already happened:

- https://news.ycombinator.com/item?id=35236275

- https://news.ycombinator.com/item?id=35248704

- https://news.ycombinator.com/item?id=35228808

Absolutely not what was suggested, but ok.

The moment that happens we can all forget about working and just do arts, space exploration, and acid orgies. But that “future” is somewhere between full self driving and thermonuclear reactor.

> I really doubt it would be faster to write code manually,

Not faster or slower but at what quality?

At least every time I've tried to ask GPT 3 & 4 to write anything it's always missing things or not even close to the optimal way that I have to look up the docs and fill in the gaps, which often takes just as long as starting from scratch.

> now basic GitHub Copilot is that it greatly improves their productivity

Perhaps it depends on what you're working on. If it's quick iterations that isn't that concerned about what code goes in and whether it's maintainable then sure.

It's impressive but for now still has lots of gaps. However it is over confident and often misleading.

Which language is it struggling with?

I've had similar experiences when testing it out with Rust, Java and Go. Once I got beyond basic stuff, very little of the output was of a quality that I would consider remotely acceptable, and the work to bring it up to standard was basically equivalent to just writing the code in the first place (which, come on, typing is not even the time-consuming part of engineering).

It makes sense that it wouldn't be very good at Go or Rust since both are rather rare languages in the open source world that copilot is trained on. When I did my first go project a few years ago, I had the hardest time finding even basic examples like how to parse a json string. Rust is even newer and less used. But java is the 3rd most popular language for GitHub projects so I would think it would do better with Java.

The failure mode seemed similar in all three languages. If you were doing toy things, or writing boilerplate stuff, it did perfectly fine. If you were writing something that wasn't a slightly-modified copy of some code that already exists out there, it fell apart. I don't think the issue is the language in this case — Go and Rust are common enough, and it rarely had trouble with the syntax — I think it's that the model doesn't go very "deep", so it's able to reproduce common patterns with minor variations but is unable to conceptualise.

Java is ultimately worse. It's old. It's gone through A LOT of change over the years. How does it even tell what's good and bad? Java also has lots of convention over configuration and "magic" in most frameworks, which it doesn't exactly understand.

When I tried it I'd often have to go back to it and keep telling it to use a different way of doing things because the world moved on. By that time there wasn't much point.

I see praises by those that have never coded in a language. They try ChatGPT, see it produce "ok" output and call it a day. If that's where we're going the web will be even more bloated than Electron and everything will be 10x worse. It's like low code but even lower (in quality).

There are over 50,000 and over 40,000 repositories just on GitHub that contain Go and Rust code, respectively. [1] Among them, some truly massive projects like Kubernetes for Go or Servo for Rust. I will freely admit your argument for new and/or obscure languages like Hare, but Go and Rust are not "rare" under any reasonable definition of the word.

[1] https://github.com/topics/go https://github.com/topics/rust

> Think of the amount of boilerplate and tests you write, and tedious API documentation lookups you do daily; that all goes away with GPT.

At work we have really worked hard to minimise boilerplate and manually-written/repetitive tests, so I don't write much of that. Getting GPT to write it would certainly be worse: we would still have the deadweight of boilerplate/repetition even if we didn't have to write it, and some of it would be incorrect. Maybe this varies a lot by company — if you're often writing a lot of repetitive code, and for whatever reason you can't fix the deeper issues, then something like GPT/Copilot could be a godsend.

About documentation lookups, I don't know if this varies by language, but I've had very little luck with using GPT for this. For the languages I use regularly, I can find anything I need in the documentation very rapidly. When I've tried to use GPT to answer the same questions, it occasionally gives completely wrong answers (wasting my time if I believe it), and almost always misses out some subtlety that turned out to be important. It just doesn't seem to be very good for this purpose yet.

> At work we have really worked hard to minimise boilerplate and manually-written/repetitive tests, so I don't write much of that.

There's boilerplate in any codebase, even if you make an effort to minimize it. There are always patterns, repeated code structure, CI and build tool configuration, etc.

If nothing else, just being able to say "write a test for this function", which covers all code paths, mocking, fuzzing, etc., would be a huge timesaver, even if you have to end up fixing the code manually. From what I've seen, this is already possible with current tools; imagine how the accuracy will improve with future generations. Today it's not much different from reviewing code from a coworker, but soon you'll just be able to accept the changes with a quick overview, or right away.

This may be highly dependent on problem domain or programming language (see the other article about GPT tending to hallucinate any time it is given problems that don't exist in its training set). My experience has mostly been that the output (including simple stuff like "test this function", though we generally avoid unit tests due to low benefit and high cost) is consistently so flawed that the time to fix it approaches the time to write it.

If you eliminated 100% of my code typing time with perfect effectiveness I think that'd make me maybe 10% or 20% more productive? Turning ideas of what the code should be doing into code just isn't a bottleneck for me in the first place. Are there people who just add net 1000 lines of code to whatever they're working on every single day or something?

I keep seeing this point made, but AI tools don't save you just typing time. They save you time you would previously use to lookup documentation, search the web and Stack Overflow answers. They save you time it takes to navigate and understand a codebase, write boilerplate code and tests, propose implementation suggestions, etc.

Dismissing them on the basis that they just save you typing time is not seeing their full potential.

Navigating and understanding a codebase is the only one of those which would excite me, but it's also something I've never even someone propose using ChatGPT for. Do you have an example of what that would look like?

There have been a few announcements here just in the last week:

- https://news.ycombinator.com/item?id=35236275

- https://news.ycombinator.com/item?id=35248704

- https://news.ycombinator.com/item?id=35228808

And this is with the GPT-4 limitation of 32k input tokens. Imagine what will be possible in the next generation that increases the context size.

there are people who code the whole thing in their head and have perfect recall of all language/api docs/references/syntax/features as that is how their brain works. ie they don't even type code until this step has occurred for them.

I think that is a small percentage of the dev community, so for me and people that don't operate like that, these tools are a game changer as you point out. I don't take what ChatGPT says at face value, I've got 15years of experience I'm weighting results against as well...

chatgpt's version of the above: Coding entirely in one's head is rare. Most developers need external resources, making development tools invaluable. While ChatGPT's input is valuable, it should be balanced against personal experience and expertise.

I've only toyed with ChatGPT, but what I like is that it knows about stuff I don't. I'm reasonably informed about the tools, practices etc. in my field, but I don't know everything, and it's been trained on all kinds of stuff I've never heard of.

In practice the stuff it will suggest to me is sort of random, it may or may not be the best choice for the task at hand, but it's a form of discovery I didn't have previously. The fact that when it tells me about e.g. a new library it can also mock up some sample code that might or might not work is a pleasant bonus.

Copilot is a huge time and typing saver for manipulating data, richly autocompleting logging messages, mocking out objects and services in tests, etc.

If you're only expecting it to solve your hard problem completely and from scratch entirely from a prompt that's probably not going to succeed, but I can't see how you're possibly faster typing 80-90 extra characters of a log statement than a Copilot user who just presses tab to get the same thing. Those little things add up to significant time savings over a week. Same for mocking services in a test, or manipulating lists of data or any number of things it autocompletes where you'd previously need to author a short script to perform or learn advanced vim movements and recording macros to emulate.

Yes unless you understand the problem well it is hard to fix it. Might as well code it yourself.

I suspect the people who find this amazing tech don't program much or are using this very differently than we are. Or program very differently than us.

If you share your prompts then we could help. The most generally applicable thing I can think of is that you have to drop your old habits from using search engines, those types of queries will not get you very far. You have to talk to it like you would talk to someone in your company Slack/Teams/whatever chat and explain what it is you're trying to do and what tools you want to use to do it. Then ask it to refine its answer by telling it what it got wrong or clarifying the request you made by adding more details. Also, always keep in mind that it is fundamentally a text completion engine. You can drastically alter the type of output you get by adding relative context up front. That can be anything from snippets of code to requests for it to write in the style of some famous person to even just a chunk of your own writing so it can get an idea of the style that you use.

I'm not talking about informational stuff.

Cool, if you don't want to talk about it that's ok. In that case I'd suggest looking up one of the various prompt libraries and learning from there.

I'm just not talking about trying to fish an accurate answer. I think you can do that. I'm talking about getting an answer about something that requires an interaction that gets the model to "understand" the problem (like you would a person) - which GPT can't do.

Something akin to: Here is a niche mathematical problem that does not fit any existing solutions/publications, we need a working but not necessarily optimal solution to do this. ?

Sure, or even an existing algo or method that is applied to something very niche that hasn't been done much (at least in the data GPT is trained on, I guess).

In my experience these things need a little nuance communicated as a part of the problem, but GPT can't get there. It just starts looping over incorrect solutions (rather than modifying them slightly to get it right like a person would).

These aren't crazy advanced things, either. I'm not a genius. I solved the problems when GPT couldn't. I was just trying out GPT4.

Consider how long our comment thread is and I still don't have a clear idea of what you're talking about or how to help you.

I'm not looking for help, so I don't think we're really trying to have the same conversation.

Then use it as autocomplete to write things you were going to put anyway but faster, it will still speed things up

Reply on news.ycombinator.com