Hacker News Reader

Comment by f6v | Hacker News Reader

My experience was always that most of my time isn’t deducted to writing code. Maybe 10%, the rest us thinking about how the code I write will fit into the existing architecture or accommodate future features.

View on HN

In very near future your IDE will send the whole codebase as context to LLMs. Then instead of thinking up all possibilities you can just ask. LLM will suggest multiple alternatives and you can select the best when and ask it to implement it.

Don't make stuff up. There is no indication that will happen.

It has already happened:

- https://news.ycombinator.com/item?id=35236275

- https://news.ycombinator.com/item?id=35248704

- https://news.ycombinator.com/item?id=35228808

Absolutely not what was suggested, but ok.

This seems very plausible to me. The context window improved a lot between GPT-3.5 and GPT-4, and OpenAI clearly see value in increasing it further.

Explain how it improved. Yes, it has improved accuracy (test taking) but it still gets stuck in the same loops over and over:

"response has issue A" -> point out issue A to GPT "response has issue B" -> point out issue B to GPT

GPT replies with the response that had issue A ...

This is not a tool that is going to be good at performing generic tasks. It just isn't.

I said that the context window improved. I mean that it is larger. GPT-3.5 is 4k tokens, GPT-4 is 8k tokens (standard) or 32k tokens (only API access atm). This is the number of tokens that GPT-X can take into account when producing a response.

Specifically, I was using this to support the statement "In very near future your IDE will send the whole codebase as context to LLMs." I'm not talking about loops or accuracy.

https://platform.openai.com/docs/models

It's true, but there is no indication that GPT can explain larger concepts for you, and negative indication it will be able to do it accurately.

It can't even explain small code to me unless it is something that it has been trained on. Often it gets even simple things wrong, either obviously, or worse, subtly wrong.

I agree that this is the part that needs more work, and is most uncertain. Increasing context windows seems like a fairly straightforward computational challenge (albeit potentially expensive). On the other hand, whether or not we can scale current models towards "true understanding" (or similar), is a total unknown atm.

I still think we will get useful things from scaling up current models though. I've already got a lot of value out of Copilot, for instance, and I'm looking forward to the next version based on GPT-4. Recently, I've been using the GPT-3 Copilot to write a lot of pandas/matplotlib code, which is fairly straightforward and repetitive, but as mainly a Java developer, I just don't have the APIs at my fingertips. Copilot helps a lot with this sort of thing.

> can scale current models towards "true understanding" (or similar), is a total unknown atm.

Right, but it's no more known than before GPT models IMO. It's the same unknown.

I don't mean to imply these language models are not impressive. They are pretty impressive.

GPT-4 has a model capable of using around 50 pages of written text worth of tokens (32,000) not sure exactly how many lines of code that translates to but it’s a lot. GPT-3 can use 4k so that’s a huge increase, the next version could be even larger and there are other ML techniques that allow for massive context lengths. Copilot already does a good job of refactoring code and knows enough about your code base to use your functions and methods. So what the other commenter said does not sound impossible to me.

It isn't anywhere near being able to diagnose anything more than off-by-one and other common errors.

Identifying hose problems will bring a LOT of value - but it isn't going to program and do general problem solving for you! It just has no signs of being able to do that.

Is there a proof that context length increase will result in a better result? It’s a possIbility, but not certain.

Yesterday’s Steve Yegge post talks about it. You can provide text as context but you can also provide a dense representation in the form of text embeddings that capture the context. Today you can manually do it by something like LangChain but in future it will be part of our text editors.

Yes it will be able to give amazing feedback to us as devs and quickly identify common problems (that I still make all the time, even after developing for a decade!) which will bring a ton of value to programmers.

But, it will be a tool - it won't be something that will solve general problems for you. It won't make an average programmer a great programmer.

The moment that happens we can all forget about working and just do arts, space exploration, and acid orgies. But that “future” is somewhere between full self driving and thermonuclear reactor.

Reply on news.ycombinator.com