Hacker News Reader

Comment by dougmwne | Hacker News Reader

Interesting. That probably means a Github copilot upgrade to a GPT-4 model is incoming.

sure hope so, especially with copilot labs brushes..

imagine highlighting some code and dumping the contents of a jira ticket into a text prompt.. these days are coming.

I’m currently slapping together something like this, though the highlighting of code is automated – in a sense – when an error is piped into the program and the error stack is used as a prompt to find a solution to the error.

The goal is to splice a solution into a file on a new branch, write a test to ensure it resolves the issue (if necessary), then create a pull request describing the change set and how the test addresses the issue.

I’ve so far worked through it manually and it works okay, but for fairly limited scopes. Like, why didn’t this environmental variable load properly? Why couldn’t we parse this input? Why did this time out? No big picture stuff.

I have a feeling GitHub will be adding this directly to repositories so initial solutions and ideas (perhaps even fixes) can be generated by an AI trained on your repo and its issues/PRs/dependencies. I’m not doing this with aspirations of building a successful tool so much as learning to think with AI as a tool for improving systems.

It’s a very tractable thing at the moment and I think future iterations of GPT will make it truly useful.

I agree, it seems inevitable that github would include this sort of auto-resolve functionality. I have had good luck with GPT outputting unified diff and LSP format. As the neighbor comment mentions, context window will be a limitation especially with large codebases - I'm thinking for that, it will be necessary to either be able to fine-tune or to summarize the rest of the code base and take multiple requests, and store the results of these intermediate summaries. I can construct this sort of thing by hand on larger codebases already, surely it can be automated to a large degree.

I think the problem will be context and token limit size. Right now, GitHub Copilot uses the current file “and related files” as context. But if you have several thousand files, even GPT-4 won’t be able to handle that. Either you need to cleverly know exactly which files to feed in for context or have a very very high token limit size.

Does GitHub Copilot use Codex model?

Yes

Actually, there is no way to be sure^. If you think about the costs + scale, it's likely to be cushman (code-cushman-001).

^ Unless you are from OpenAI, in which case I have more questions for you :)

They use similar models to the codex models, they mentioned it a couple years ago

cushman and code-davinci are similar for sure (same architecture). Perhaps that's what they meant.

was codex API used in GitHub copilot if so what are implications on Copilot as our team depends significantly on copilot for our work in neovim?

Not the API itself, but the LLM behind it was a codex model. They are essentially saying Codex era models are obsolete so that means an upgrade announcement is probably only a few weeks off.

could also just migrate to gpt3.5 turbo as it's more cost effective

what are drawbacks of gpt-4 relative to codex ?

It costs money, mainly.

any other drawbacks on codex vs gpt-4

Reply on news.ycombinator.com