The code that LLM outputs, has worked well enough to learn from since the initial launch of ChatGPT. This even though back then you might have to repeatedly say "continue" because it would stop in the middle of writing a function.
Necessarily, LLM output that works isn't gibberish.
Especially in this context, if a mystery box solves a problem for me, I can look at the solution and learn something from that solution, c.f. how paper was inspired by watching wasps at work.
Even the abject failures can be interesting, though I find them more helpful for forcing my writing to be easier to understand.
Also, it's actually fun watching LLMs debug. Since they're reasonably similar to devs while investigating, but they have a data bank the size of the internet so they can pull hints that sometimes surprise even experienced devs.
I think hard earned knowledge coming from actual coding is still useful to stay sharp but it might turn out the balance is something like 25% handmade - 75% LLM made.
they have a data bank the size of the internet so they can
pull hints that sometimes surprise even experienced devs.
it might turn out the balance is something like 25% handmade - 75% LLM made.
In law, training is not itself theft. Pirating books for any reason including training is still a copyright violation, but the judges ruled specifically that the training on data lawfully obtained was not itself an offence.
Cloudfare has to block so many more bots now precisely because crawling the public, free-to-everyone, internet is legally not theft. (And indeed would struggle to be, given all search engines have for a long time been doing just that).
> As the arms race continues AI DDoS bots will have less and less recent "training" material
My experience as a human is that humans keep re-inventing the wheel, and if they instead re-read the solutions from even just 5 years earlier (or 10, or 15, or 20…) we'd have simpler code and tools that did all we wanted already.
For example, "making a UI" peaked sometime between the late 90s and mid 2010s with WYSIWYG tools like Visual Basic (and the mac equivalent now known as Xojo) and Dreamweaver, and then in the final part of that a few good years where Interface Builder finally wasn't sucking on Xcode. And then everyone on the web went for React and Apple made SwiftUI with a preview mode that kept crashing.
If LLMs had come before reactive UI, we'd have non-reactive alternatives that would probably suck less than all the weird things I keep seeing from reactive UIs.
That is simply not true. Freely available on the web doesn't mean it's in the Public Domain. The "lawfully obtained" part of your argument is patently untrue. You can legally obtain something, but that doesn't mean any use of it is automatically legal as well. Otherwise, the recent Spotify dump by Anna's Archive would be legal as well.
It all depends on the license the thing is released under, chosen by the person who made it freely accessible on the web. This license is still very emphatically a legally binding document that restricts what someone can do with it.
For instance, since the advent of LLM crawling, I've added the "No Derivatives" clause to the CC license of anything new I publish to the web. It's still freely accessible, can be shared on, etc., but it explicitly prohibits using it for training ML models. I even add an additional clause to that effect, should the legal interpretation of CC-ND ever change. In short, anyone training an LLM on my content is infringing my rights, period.
Doesn't need to be.
> The "lawfully obtained" part of your argument is patently untrue. You can legally obtain something, but that doesn't mean any use of it is automatically legal as well.
I didn't say "any" use, I said this specific use. Here's the quote from the judge who decided this:
5. OVERALL ANALYSIS.
After the four factors and any others deemed relevant are “explored, [ ] the results [are] weighed together, in light of the purposes of copyright.” Campbell, 510 U.S. at 578. The copies used to train specific LLMs were justified as a fair use. Every factor but the nature of the copyrighted work favors this result. The technology at issue was among the most transformative many of us will see in our lifetimes.
> Otherwise, the recent Spotify dump by Anna's Archive would be legal as well.
I specifically said copyright infringement was separate. Because, guess what, so did the judge the next paragraph but one from the quote I just gave you.
> For instance, since the advent of LLM crawling, I've added the "No Derivatives" clause to the CC license of anything new I publish to the web. It's still freely accessible, can be shared on, etc., but it explicitly prohibits using it for training ML models. I even add an additional clause to that effect, should the legal interpretation of CC-ND ever change. In short, anyone training an LLM on my content is infringing my rights, period.
It will be interesting to see if that holds up in future court cases. I wouldn't bank on it if I was you.
Yes, but I can't stop them, can you?
> But I'm glad you're able to have your fun.
Unfortunately I have to be practical.
> Doubtful. As the arms race continues AI DDoS bots will have less and less recent "training" material. Not a day goes by that I don't discover another site employing anti-AI bot software.
Almost all these BigCos are using their internal code bases as material for their own LLMs. They're also increasingly instructing their devs to code primarily using LLMs.
The hope that they'll run out of relevant material is slim.
Oh, and at this point it's less about the core/kernel/LLMs than it is about building ol' fashioned procedural tooling aka code around the LLM, so that it can just REPL like a human. Turns out a lot of regular coding and debugging is what a machine would do, READ-EVAL-PRINT.
I have no idea how far they're going to go, but the current iteration of Claude Code can generate average or better code, which is an improvement in many places.
The hope that they'll run out of relevant material is slim.
> If big corps are training their LLMs on their LLM written code <<and human reviewed code>>…
The last part is important.
You can absolutely learn from an LLM. Sometimes.documentation sucks and the LLM has learned how to put stuff together feom examples found in unusual places, and it works, and shows what the documentation failed to demonstrate.
And with the people above, I agree - sometimes the fun is in the end process, and sometimes it is just filling in the complexity we do not have time or capacity to grab. I for one just cannot keep up with front end development. Its an insurmountable nightmare of epic proportions. Im pretty skilled at my back end deep dive data and connecting APIs, however. So - AI to help put together a coherent interface over my connectors, and off we go for my side project. It doesnt need to be SOC2 compliant and OWASP proof, nor does it need ISO27001 compliance testing, because after all this is just for fun, for me.
That's what LLMs take away. Nothing is given back to the community, nothing is added to shared knowledge, no differing opinions are exchanged. It just steals other people's work from a time when work was still shared and discussed, removes any indication of its source, claims it's a new thing, and gives you no way to contribute back, or even discuss it and maybe get confronted with different opinions of even discovering a better way.
Let's not forget that one of the main reasons why LLMs are useful for coding in the first place, is that they scraped SO from the time where people still used it.
Though, I guess I do treat LLM's as a last resort longshot for when other documentation is failing me.
I don't think current LLMs take you all the way but a powerful code generator is a useful think, just assemble guardrails and keep an eye on it.
That doesn't mean the agents are amazing, but they can be useful.
instead of telling me about z option it keeps hallucinating something that doesnt exist and even says its in the docs when it isnt.
Literally just wasting my time
But in the sense that even today many people don't use compilers or static analysis tools. But that world is slowly shrinking.
Same for LLMs, the non LLM world will probably shrink.
You might be able to have a long and successful career without touching them for code development. Personally I'd rather check them out since tools are just tools.
I now understand why Altman announced "Code Red" at OpenAI. If their tools don't catch up drastically, and fast, they'll be one for the history books soon. Wouldn't be the first time the big, central early mover in a new market suddenly disappears, steamrolled by the later entrants.
99% of an LLM's usefulness vanishes, if it behaves like an addled old man.
"What's that sonny? But you said you wanted that!"
"Wait, we did that last week? Sorry let me look at this again"
"What? What do you mean, we already did this part?!"
Further, they can't write code that fast, because you have to spend 1000x explaining it to them.
Instead you’d learn it, remember it, and it would be useful next time. But it’s not.
Instead of manual coding training your time is better invested in learning to channel coding agents, how to test code to our satisfaction, how to know if what AI did was any good. That is what we need to train to do. Testing without manual review, because manual review is just vibes, while tests are hard. If we treat AI-generated code like human code that requires a line-by-line peer review, we are just walking the motorcycle.
How do we automate our human in the loop vibe reactions?
Yes, I recon coding is dead.
No, that doesn't mean there's nothing to learn.
People like to make comparisons to calculators rendering mental arithmetic obsolete, so here's an anecdote: First year of university, I went to a local store and picked up three items each costing less than £1, the cashier rang up a total of more than £3 (I'd calculated the exact total and pre-prepared the change before reaching the head of the queue, but the exact price of 3 items isn't important enough to remember 20+ years later). The till itself was undoubtedly perfectly executing whatever maths it had been given, I assume the cashier mistyped or double-scanned. As I said, I had the exact total, the fact that I had to explain "three items costing less than £1 each cannot add up to more than £3" to the cashier shows that even this trivial level of mental arithmetic is not universal.
I now code with LLMs. They are so much faster than doing it by hand. But if I didn't already have experience of code review, I'd be limited to vibe-coding (by the original definition, not even checking). I've experimented with that to see what the result is, and the result is technical debt building up. I know what to do about that because of my experience with it in the past, and I can guide the LLM through that process, but if I didn't have that experience, the LLM would pile up more and more technical debt and grind the metaphorical motorbike's metaphorical wheels into the metaphorical mud.
Code review done visually is "just vibe testing" in my book. It is not something you can reproduce, it depends on the context in your head this moment. So we need actual code tests. Relying on "Looks Good To Me" is hand waving, code smell level testing.
We are discussing vibe coding but the problem is actually vibe testing. You don't even need to be in the AI age to vibe test, it's how we always did it when manually reviewing code. And in this age it means "walking your motorcycle" speed, we need to automate this by more extensive code tests.
However, code review can spot things like "this is O(n^2) when it could be O(n•log(n))", or "you're doing a server round trip for each item instead of parallelising them" etc.
You can also ask an LLM for a code review. They're fast and cheap, and whatever the LLM catches is something you get without having to waste a coworker's time. But LLMs have blind spots, and more importantly all LLMs (being trained on roughly the same stuff in roughly the same way) have roughly the same blind spots, whereas human blind spots are less correlated and expand coverage.
And code smells are still relevant for LLMs. You do want to make sure they're e.g. using a centralised UI style system and not copy-pasting style into each widget, because duplication wastes tokens and is harder to correctly update with LLMs for much the same reason it is with humans: stuff gets missed during the process when it's copypasta.
Take a look here: https://github.com/horiacristescu/archlib/blob/main/examples... (but it's still WIP, I am not there yet)
This is funny in the sense that in properly built urban environment bycicles are one of the best ways to add some physical activity in a time constrained schedule, as we're discovering.
All channelling is broken when the model is updated. Being knowledgeable about the foibles of a particular model release is a waste of time.
> how to test code to our satisfaction
Sure testing has value.
> how to know if what AI did was any good
This is what code review is for.
> Testing without manual review, because manual review is just vibes
Calling manual review vibes is utterly ridiculous. It's not vibes to point out an O(n!) structure. It's not vibes to point out missing cases.
If your code reviews are 'vibes', you're bad at code review
> If we treat AI-generated code like human code that requires a line-by-line peer review, we are just walking the motorcycle.
To fix the analogy you're not reviewing the motorcycle, you're reviewing the motorcycle's behaviour during the lap.
My point is that visual inspection of code is just "vibe testing", and you can't reproduce it. Even you yourself, 6 months later, can't fully repeat the vibe check "LGTM" signal. That is why the proper form is a code test.
I've been programming for 15+ years, and I think I've forgotten the overwhelming majority of the things I've googled. Hell, I can barely remember the things I've googled yesterday.
Some people will try to vibe out everything with LLMs, but other people will use them to help engage with their coding more directly and better understand what's happening, not do worse.
For the people who just want to solve some problem unrelated to computers but require a computer for some part of the task, yes AI would be more “fun”.
Most of all I find what computers allow humanity to achieve extremely interesting and motivating. I call them the worlds most complicated robot.
I don’t find coding overly fun in itself. What I find fun is the results I get when I program something that has the result I desire. Maybe that’s creating a service for friends to use, maybe it’s a personal IT project, maybe it’s having commercial quality WiFi at home everyone is amazed at when they visit, etc. Sometimes - even often - it’s the understanding that leads to pride in craftsmanship.
But programming itself is just a chore for me to get done in service of whatever final outcome I’m attempting to achieve. Could be delivering bits on the internet for work, or automating OS installs to look at the 50 racks of servers humming away with cable porn level work done in the cabinets.
I never enjoyed messing around with HTML at that much in the 90s. But I was motivated to learn it just enough to achieve the cool ideas I could come up with as a teenager and share them with my friends.
I can appreciate clean maintainable code, which is the only real reason LLMs don’t scratch the itch as much as you’d expect for someone like me.
I don’t enjoy boilerplate. I don’t necessarily enjoy all of the error checking and polishing and minutia in turning algorithms into shippable products.
I find AI can be immensely helpful in making real things for people to use, but I still enjoy doing what I find fun by hand.
Different strokes I guess
I just was pushing back on the “you aren’t into computers if you don’t get intrinsic joy out of programming itself” bit.
I'm a stereotypical nerd, into learning for its own sake.
I can explain computers from the quantum mechanics of band gaps in semiconductors up to fudging objects into C and the basics of operating systems with pre-emptive multitasking, virtual memory, and copy-on-write as they were c. 2004.
Further up the stack it gets fuzzy (not that even these foundations are not; "basics" of OSes, I couldn't write one); e.g. SwiftUI is basically a magic box, and I find it a pain to work with as a result.
LLM output is easier to understand than SwiftUI, even if the LLM itself has much weirder things going on inside.
I think I can describe the principles at work with DNS, but not all of how IP packets are actually routed; the physics of beamforming and QAM, but none of the protocol of WiFi; the basics of error correction codes, but only the basics and they're probably out of date; the basic ideas used in private key crypto but not all of HTTPS; I'd have to look up the OSI 7-layer model to remember all the layers; I understand older UI systems (I've even written some from scratch), but I'm unsure how much of current web browsers are using system widgets vs. it all being styled HTML; interrupts as they used to be, but not necessarily as they still are; my knowledge of JavaScript is basic; and my total knowledge of how certificate signing works is the conceptual level of it being an application of public-private key cryptography.
I have e.g. absolutely no idea why Chrome is famously a memory hog, and I've never learned how anything is scheduled between cores at the OS level.