At the moment is the significant part, how long do we expect this to last now that AI is capable of generating novel ideas like solutions to Erdos conjectures?
For example, If you are not aware of Internet, you would consider a traditional internet search that comes up with a stackoverflow answer as a machine generating "novel ideas" and answers.
It is a clever marketing trick (touting Erdos solutions) employeed by AI companies.
> Every line of code I commit is generated by LLMs...
Imagine if someone told you "Hey, this stackoverflow site is great. Everything I commit in my work is copy pasted from it!". What would you think about their work? Is that something worth bragging about?
Start with clean llm, no external previous ideas of humans inserted into it, and let it generate some wisdom on its own and then lets talk. (btw thats how I would expect we could get closer to AGI with these statistical models, but thats just my opinion)
No, it did not.
Start with clean llm, no external previous ideas of humans inserted into it, and let it generate some wisdom on its own and then lets talk.
An LLM is about as likely to do that as you are. The ability to generate "wisdom" ab initio cannot possibly be a criterion for intelligent reasoning. The ability to arrive at novel mathematics proofs, on the other hand, is good enough for me.
Intelligence means making the most of the resources and information available, not the ability to speedrun the Big Bang. LLMs are certainly smarter than humans who dismiss them as "text searchers."
It did. In the form of a pattern that humans are too incapable of recongizing. LLMs identify and repeat the pattern. That is all.
For example, if A -> B and B -> C, logic dictates A -> C. But LLMs will be able to state that A -> C, without actually using logic, if there is sufficient statements in its training data that says A -> B and B -> C and A -> C. So now if you say P -> Q and Q -> R, it will say that P -> R, when there is no explict P -> R in the training data and NOT using logic. For you, it looks like a new discovery inferred using logic when it is not. But that is how that happens..
It is just pattern recognition masquarading as logic, x, y, or z.
You're not feeling even the faintest stirrings of cognitive dissonance when a "text search" solves math problems you couldn't solve yourself?
This "super intelligent" and "capable" thing cannot even understand that your ssh keys are private and should not be sent to randos. It can solve complex math, but does not understand basic security/privacy.
What does that say to you?
When somebody posts their private keys to Github, it's usually a human. Enough said.
(And if you had ever used Claude Code, you'd know that it nags you endlessly about key hygiene.)
Oh wait, I thought these things were super smart. I didn't expect "social engineering" to work on them.
> models nowadays are way sharper than they were even a year ago.
You are missing the point. If the thing can solve complex math problems and at the same time be so dumb as to fall for "social engineering", then that means that it is not "smartness" or "reasoning" that is helping it to solve those problems. Just some form of advanced, but yet dumb, search algorithm.
No they are not. Social engineering won't work on a human security expert who knows and understands the implications of the information they are giving away. Your analogy is pointless.
Social engineering, like prompt injection, is a context attack — easy to spot if you're ready for it, but harder in different circumstances (rushed, panicked, tired, having a bad day, etc.).
Troy Hunt (security consultant, creator of HaveIBeenPwned) and Cory Doctorow have both been successfully phished [0][1]. They're both tech- and security-savvy people who "should have known better" but it happened to them anyway. But maybe you're different... you'd never fall for an online scam, right? [2]
[0] https://www.troyhunt.com/a-sneaky-phish-just-grabbed-my-mail...
[1] https://doctorow.medium.com/https-pluralistic-net-2025-04-05...
[2] https://news.harvard.edu/gazette/story/2024/09/youd-never-fa...
Are you serious? LLMs, being a computer program, should always "be ready"?
Unless you want to also claim that LLMs can be rushed, panicked, tired or can have a "bad day"!
Jesus! The mental gymnastics people will go through to justify LLMs is just absurd!
Largely agree with you otherwise, not sure why you read my comment as mental gymnastics to justify LLMs. I don't think that they have an internal emotional state that can feel rushed, panicked, so on. They do — superficially — "respond" to such cues in language, which is why they can be "threatened" [0] and "flattered" [1]. But without an internal theory of mind, LLMs do this sycophantically without any internal model of the world (hence the quotes above, to avoid fully anthropomorphizing their behavior).
The only parallel I'm drawing is that both humans and LLMs can be coerced into unintended behavior via language ("social engineering" and "prompt jailbreaking" respectively), and those attacks are more effective if an attacker is allowed to control more "context", even if the underlying mechanism of why those attacks work is completely different.
[0] https://arxiv.org/pdf/2507.21133
[1] https://arstechnica.com/science/2025/09/these-psychological-...
I'm not just trying to be snarky, but I have no idea how to read this without taking the implication that humans are advanced, yet dumb, search algorithms.
An LLM will happily say X and behaves in contradiction to X. Because it does not reason. Its behavior is not derived from things that it claim (or appears) to know.
That is literally not true and why we talk about "stated vs revealed preference" and such.
…and then wake up from the nightmare wishing the stress from the job is lower.
I can only laugh that some people truly believe that developers, one of the most ardent group at automating the tedious part of their job, would refuse to use an effective tool. You only need to look at the open source world to see people litterally scratching their own hitch everywhere.
By this logic, all non-outsourced development should be dead already.
So I think programmers who are asked to use LLMs should demand their job descriptions to be changed to that of managers, and should ever deny responsibility if the LLM stops to make progress. Thus let the organization be responsible to actually find a version of AI that works, just like how it was responsible to find competent programmers to work under the managers before.
No one said I don't use LLMs. I use LLMs daily...for search. That is its best use. That is my own judgement.
As for "certainty," that's a luxury nobody has.
Is it _faster_? sure! That is NOT better.
Before you ask, I write code w/ agents daily, I find it useful, but it's not better than I am purely on quality.
Certainly Claude/Codex's knowledge of algorithms and data structures is leagues ahead of any human programmer alive. Only its capacity for creating new ones on demand is weaker. Recent results from the mathematics field suggest that's a temporary state of affairs.
As far as prose goes, my best writing is indeed better than the best I've seen from LLMs. But that's a matter of opinion (mine.) On average the clanker wins, especially if conditioned to avoid LLM-isms.
The truth is that the models are getting better in both areas, while I'm not. Which IMHO is freaking awesome, not a reason to burn it all down.
The problem, of course, is that generating a one time solution to a problem is a much easier problem space than a many-input task with human product concerns
All i said is that one type of task is easier than another
I said nothing about a human’s capabilities, which are most certainly different than an AI’s
This is the second time you’ve said a bunch of stuff I didn’t say, and then tried to argue with me over it.