They don’t do half as well in large codebases that use ad-hoc frameworks. For example, it has no idea how to retrieve the currently-logged-in user object for a new endpoint you want it to build.
The solution there is to fine-tune it on your codebase, but that’s likely a few years away for the average LLM user.
Greenfield? Unless we disagree on the definition of greenfield (no prior work), then a plane spotter app is anything but greenfield. GitHub returns 11 repos for JavaScript alone.
It knew the data model and methods that existed in code, and used these.
In this video they say "Custom-tuned models on code" are coming soon, but aren't there yet: https://youtu.be/qbxj_JDQ4Lk?si=qSTzMBpmqqJD5o9u&t=1632
The names are very similar.
Currently working on an Elixir project with Ash Framework, and I wouldn't trust a single Chat GPT output for either of them.
The wider context window and better tooling are all being worked on.
Just builds a local vectordb with your code base