“When open source developers working in codebases that they are deeply familiar with use AI tools to complete a task, they take longer to complete that task”
I have anecdotally found this to be true as well, that an LLM greatly accelerates my ramp up time in a new codebase, but then actually leads me astray once I am familiar with the project.
If you are unfamiliar with the project, how do you determine that it wasn't leading you astray in the first place? Do you ever revisit what you had done with AI previously to make sure that, once you know your way around, it was doing it the right way?
What’s most useful about the LLM in the early stages is not the actual code it writes, but its reasoning that helps me learn about the structure of the project. I don’t take the code blind, I am more interested in the reasoning than the code itself. I have found this to be reliably useful.
How does using AI impact the amount of time it takes you to become sufficiently familiar with the project to recognize when you are being led astray?
One of the worries I have with the fast ramp-up is that a lot of that ramp-up time isn't just grunt work to be optimized a way, it's active learning, and bypassing too much of it can leave you with an incomplete understanding of the problem domain that slows you down perpetually.
Sometimes, there are real efficiencies to be gained; other times those perceived efficiencies are actually incurring heavy technical debt, and I suspect that overuse of AI is usually the latter.
Coming from other programming languages, I had a lot of questions that would be tough to nail down in a Google search, or combing through docs and/or tutorials. In retrospect, it's super fast at finding answers to things that _don't exist_ explicitly, or are implied through the lack of documentation, or exist at the intersection of wildly different resources:
- Can I get compile-time type information of Enum values?
- Can I specialize a generic function/type based on Enum values?
- How can I use macros to reflect on struct fields?
- Can I use an enum without its enclosing namespace, as I can in C++?
- Does rust have a 'with' clause?
- How do I avoid declaring timelines on my types?
- What is an idiomatic way to implement the Strategy pattern?
- What is an idiomatic way to return a closure from a function?
...and so on. This "conversation" happened here and there over the period of two weeks. Not only was ChatGPT up to the task, but it was able to suggest what technologies would get me close to the mark if Rust wasn't built to do what I had in mind. I'm now much more comfortable and competent in the language, but miles ahead of where I would have been without it.
To their credit, the people on the Rust forum have been really responsive at answering my questions and poking holes in incorrect unsafe implementations, and it is from speaking to them that I truly feel I have learned the language well.