I can't think of many things that have changed in architecture since 2010. I'm not talking about fads but about actual principles.
they aren't principles as such, but they certainly play into what is important and how you apply them
Most problems are not well-addressed by shallow monoliths made of glue code. It's irrelevant what "AI wants", just as it's irrelevant what blockchain "wants".
People tried to use both as databases, put both in cars, invest in both. The vast majority of claims people make about them are just not evidenced, yet their hypist-adherents are so confident that they're willing to show you evidence that contradicts their claims, and call it "proof".
Yes, the actual technologies are very different. But nobody is actually paying attention to the technologies (an ignorance that my other comment snarkily accuses you of displaying here – I probably should've been kinder).
Linus Torvalds is a strong advocate. He even wrote a blockchain-based source code management system, which he dubbed “the information manager from hell”[0], spending over three months on it (six months, by his own account) before handing it over to others to maintain.
People complain that this “information manager” system is hard to understand, but it's actively used (alongside email) for coordinating the Linux kernel project. Some say it's crucial to Linux's continued success, or even that it's more important than Linux.
[0]: see commit e83c5163316f89bfbde7d9ab23ca2e25604af290
It does make the last step ("have a piece of software that's fit-for-purpose") a bit harder; but that's a price you should be willing to pay for velocity.
Maybe this time it’s different, maybe it’s not. Time will tell.
I mean, you're not even wrong.
I’ve been writing code professionally for a decade. I’ve led the development of production-grade systems. I understand architecture. I’m no idiot. I use Copilot. It’s regularly helpful. and saves time. Do you have a counter-argument that doesn’t involve some sort of thinly veiled “but you’re an idiot and I’m just a better developer than you”?
I don’t by any means think that a current generation LLM can do everything that a software developer can. Far from it. But that’s not what we are talking about.
It's not that I don't believe your experience specifically. I don't believe either side in this case knows the real industry-wide average improvement until someone really measures it.
What I do know though, is that ChatGPT can finish a leetcode problem before I've even fully parsed the question.
There are definitely ratholes to get stuck and lose time in when trying to get the LLM to give the right answer, but LLM-unassisted programming has the same problem. When using an LLM to help, there's a bunch of different contexts I don't have to load in because the LLM is handling it giving me more head space to think about the bigger problems at hand.
No matter what a study says, as soon as it comes out, it's going to get picked apart because people aren't going to believe the results, no matter what the results say.
This shit's not properly measurable like in a hard science so you're going to have to settle for subjective opinions. If you want to make it a competition, how would you rank John Carmack, Linus Torvalds, Grace Hopper, and Fabrice Bellard? How do you even try and make that comparison? How do you measure and compare something you don't have a ruler for?
This is an interesting case for two reasons. One is that leetcode is for distilled elementary problems known in CS - given all CS papers or even blogs at disposal, you should be able to solve them all by pattern matching the solution. Real work is anything but that - the elementary problems have solutions in libraries, but everything in between is complicated and messy and requires handling the unexpected/underdefined cases. The second reason is that leetcode problems are fully specified in a concise description with an example and no outside parameters. Just spending the time to define your problem to that level for the LLM is likely getting you more than halfway to the solution. And that kind of detailed spec really takes time to create.
You have to watch out for that, that's an AI evaluation trap. Leetcode problems are in the training set.
I'm reminded of people excitedly discussing how GPT-2 "solved" the 10 pounds of feathers versus 10 pounds of lead problem... of course, it did, that's literally in the training set. GPT-2 could be easily fooled by changing any aspect of the problem to something it did not expect. Later ones less so though last I tried a few months ago while they got it right more often then wrong they could still be pretty easily tripped up.
Most likely though, code quality will suffer. I have a friend who observes what people commit every day, and some of them (apparently plural) copy & paste answers from an LLM and commit it before checking that it even compiles. And even when it works, it’s often so convoluted there’s no way it could pass any code review. Sure if you’re not an idiot you wouldn’t do that, but some idiots use LLMs to get through interviews (it sometimes works for remote assignments or quizzes), and spotting them on the job sometimes takes some time.
LLMs for coding are definitely useful. And harmful. How much I don’t know, though I doubt right now that the pros outweigh the cons. Good news is though, as we figure out the good uses and avoid the bad ones, it should gradually shift towards "more useful than not" over time. Or at least, "less harmful than it was".
If I were using one of these things, that's what I'd do. (Preferably rewriting the commit to read Author: AcmeBot, Committer: wizzwizz4) It's important that commit history accurately reflect the development process.
Now, pushing an untested commit? No no no. (Well, maybe, but only ever for backup purposes: never in a branch I shared with others.)
Requiring that the counter-argument reaches a higher bar than both your and the original argument is...definitely a look!
And nobody ever will, because in 10 years, coding AIs will not look like they do now. Right now they are just incapable of architecture, which your supposed optimal approach seems to be optimizing for, but I wouldn't care to guarantee that's the case in 10 years. If nothing else, there will certainly be other relevant changes. And you'll need experience to determine how best to use those, unless they just get so good they take care of that too.
But I find it intriguing. Do you mean architecting software to allow LLMs to be able to modify and extend it? Having more of the overall picture in one place (shallow monoliths) and lots of helper funtions and modules to keep code length down? Ie, optimising for the input and output context windows?
LLMs can unlock incredibly development velocity. For things like creating utility or helper functions and their unit tests at the same time, an engineer using a LLM will easily 10x an equally skilled engineer not using a LLM. The key is to architect your system so that as much of it as possible can be treated this way, while not making it indecipherable for humans.
This is a temporary constraint. Soon the maintenance programmers will use an AI to tell them what the code says.
The AI might not reliably be able to do that unless it is in the same "family" of AIs that wrote the code. In other words, analogous to the situation today where choice of programming language has strategic consequences, choice of AI "family" with which to start a project will tend to have strategic consequences.
It's more of a generic system with attention on performance of specific application rather than a system designed to cater to current LLMs.