Hacker News Reader

Hacker News

Comment by dominotw | Hacker News Reader

Parent Full thread

dominotw

[flagged]

View on HN

And how much of pre-LLM code was just copy pasta from Stack Overflow?

Code doesn't need to be novel to be useful. There's a reason why design patterns are a thing in software.

That’s why we abstract the useful code away as libraries, frameworks, etc.

AI is not an abstraction.

Of course it is. It abstracts away the code generation to a much more compressed natural language prompt. That's the very definition of a high abstraction...

It abstracts shit mate. How many Rs in the strawberry and if you want to go to a car wash, should you walk, run or drive? They're just fucking text generators, which happen to spit out something half-usable about 60% of the time.

You generally need to wire libraries in to your service, and you may be using the library in a slightly different way than normal. AIs are perfectly capable of doing this.

Back to the original point, though: most software engineering work isn't novel. Most people are working on slightly different iterations of the same thing, but with the aim of different products. You can have completely different products that use nearly the same patterns as most other services.

To put it bluntly: we don't need AI to generate novel code for the vast majority of the software being built.

It depends on the programming. However, you're right: another Angular reinsurance accounting application doesn't pose much of a challenge for LLMs.

LLMs recently solved a major, famous open mathematical problem in combinatorial geometry:

https://www.reddit.com/r/math/comments/1tj534d/openais_inter...

> you are merely remixing whats out there

So basically 90% of programming in an enterprise environment? lol. Sounds useful to me...

Unless you mean remixing the alphabet / tokens, this is mathematically false. 2^256 gets you to unique very fast, and that’s like 252 bytes or like 80 tokens. Remember almost all numbers are irrational. Complexity is infinite.

There is nothing new under the sun.

Everything changes, nothing remains without change.

Nah, most things stay the same withing quite narrow margins. The moon gets hit by a meteor now and then, but it's been essentially the same rock for some billion years.

Life didn't stay mostly the same over that time span. Evolution is a good counter to everything remaining the same under the sun.

Anyway, Buddhists and Heraclitus aren't wrong. It's just a matter of enough time and the moon will no longer be a moon.

But there are other suns :)

There are other suns that nothing new under them

Most code written are not novel. Actually most code written should not be novel. Eg: the number of lines of code written isn't spent on writing git, it's spent on writing that landing page no one ever sees.

99% of coders don't need to generate anything novel.

then what are they doing with the time savings from llm. generating more remixing ? is there really so much demand for remix slop. i dont think so.

I don't think I've ever worked on a project where there wasn't more work to be done than there was time to do it in.

ai can remix things on the fly so lots of widgets that were being stamped on are not needed anymore.

we are in middle state where ai tools to generate on the fly widget work arent accessible in the form that most ppl need. So programmers are currently doing the manual step by managing remix into easily consumable form.

My job is different from most developers these days.

But I'm writing more code than I ever did as a developer. YMMV

I don't think you understand how LLMs work.

They're not merely re-arranging pre-existing blocks of code.

And they have been shown to develop emergent properties that weren't in their training set time and again.

They generate novel things as much as the average programmer (which works after himself having practice, exposure to codebases, and training, and reading API documentation, and such) generates novel things.

Not to be pedantic but the emergent properties are in the training set, and thus the model and algorithm. There's no magic coming from the universe.

What makes the behavior emergent is that it can't be predicted at training time.

The emergent and unpredictable output is the result of massive vector complexity being encoded.

>Not to be pedantic but the emergent properties are in the training set, and thus the model and algorithm. There's no magic coming from the universe.

You are either being pedantic or missing the point of emergent however.

Yes, it's not some novel unforeseen thing, like a magical Marvel Universe material or some unknown to humanity mode of thinking. Same way when people make something new they still recombine known words, or colors, or physical things in the universe.

It is however new capabilities that is not explicitly in the training set and can't be predicted by it. Like teaching something only calculus training materials and it figures out boolean algebra.

>The emergent and unpredictable output is the result of massive vector complexity being encoded

As opposed to what in humans? God given revelation?

My point was that if training data + encoding/training = model with emergent behaviour

The emergent behaviour is in the training data and/or encoding/training.

So while I agree it is emergent from the complexity, it isn't some unknown mechanism. Just complexity at scale.

>So while I agree it is emergent from the complexity, it isn't some unknown mechanism. Just complexity at scale.

So like humans? Like the universe?

since you understand how llms really work. show us what novel items llms have generated for you.

How about a novel theorem escaping mathematicians for close to a century?

https://www.scientificamerican.com/article/ai-just-solved-an...

The question was, what did they solve FOR YOU.

> And they have been shown to develop emergent properties that weren't in their training set time and again.

Ah yes, the famous emergent properties - like suggesting that we should walk to the car wash?

Source?

If we were doing novel things, we’d be scientists. I’m an engineer though. I don’t think I’ve been writing slop for 30 years.

they can take novel things. so my novel

Reply on news.ycombinator.com